Usability Testing of Technical Communications

What is Usability Testing?

Usability testing is a tool used to evaluate a set of instructions or product. It tests to see if these instructions or product do what it is designed to do. Usability tests are a necessary part of the process of writing instructions or developing a product.

What is the Purpose of Usability Testing?

Usability testing involves test readers who read and/or use a draft in a similar situation as the intended audience would be in. After the test, you gather important information by observing and questioning the test reader. This allows you to get a good sense of the overall response of your draft so you can revise areas that need improvement. Usability testing can be simple or complex depending on the situation and importance.

Why is usability testing such a big deal and who uses it?

You may think your writing is easy to understand but more importantly, does your reader? Does your draft work the way it is intended to? The only accurate way to answer this question is to have a test subject read and evaluate your draft. Primary draft testing is the best method to increase the quality of a final draft that will be read and studied by thousands of people. Test materials range from drafts of websites, to print instructions, to informational documents.

Everybody from Fortune 500 companies to college students use usability testing in order to improve the efficiency of their final draft. Big corporations treat testing as such a big deal that they build special buildings for the sole purpose of usability testing. These special facilities can be equipped with video cameras, monitored computers, as well as one-way mirrors so testing specialist employees can observe customers' responses to drafts. College students can use this strategy with a roommate or classmates. Whatever the situation, usability testing is proven to be a valuable tool in the writing world.

Preparing for Usability Testing

Establish Your Test Objectives

Create questions or concerns that you want answered as a result of your usability testing. Be prepared for questions to arise during the testing process. Pay close attention to what your tester is telling you as this is valuable information to help improve your instructions. Be sure to annotate your observations so you can recognize any trends or red flag that exist in a particular section. Once you recognize what you want to learn, you can customize the test so it evaluates and provides the specific information you are looking to obtain. The two most general questions usability testing seeks to answer are:

How can my draft be improved?
Is my communication sufficient?

Other examples of more specific question are:

How well is the page formatted?
Does the draft influence the test readers’ attitude the way I intended it to?
Are my tasks easy to follow?

To help evaluate this criterion, try creating measurable criteria. Measurable criteria may involve a test reader reviewing an informative draft and then answering questions. Make sure the criteria is highly specific towards your objectives. Criteria that is too broad will not be useful during later assessments. If the reader gets a certain percentage correct, the draft is adequate. It can also be time related. For example, can the test reader navigate to the website link provided within twenty seconds?

Selecting Testers

Choose readers who are suitable to the testing scenario. This is a very important part of testing because if a test reader is not part of the target audience, they may influence you to make ineffective adjustments to your document. A vital rule for most cases is to pick a dentist for dentistry or an electrician for electrical instructions. Furthermore, the knowledge levels of your target readers on the subject you want to test them on should vary according to the objectives of your testing. For example, you are constructing an origami instruction manual for individuals with intermediate origami skill levels and you want to test the clarity of your instructions with a usability test. If you select test readers with no origami experience, your instructions will seem overly complicated to the readers and the outcome of the usability test will send you a message that additional clarification is needed, where in actuality your instructions might be sufficient for your target audience. Similarly, testing the clarity of your instructions with origami experts could lead you to over-simplify your instructions. The number of test readers used can vary and play a role in determining the usability of your final draft. The number of test readers used depends on the type of draft and the availability of test subjects. For example, there may be a situation where ten or more test readers are needed to determine the overall comprehension of a draft. The purpose of needing more test readers is that each test reader has a unique perspective and can contribute different opinions and ideas. In the case of a more structured draft, such as step-by-step instructions, fewer testers are needed since you are testing only for clarity of these instructions. The key here is to make sure that your test readers use your draft the same way your users will.

Major Categories for Readers' Tasks

The tasks performed by the test readers can be categorized into three categories:

Perform a Procedure (performance test): For example, navigating through step-by-step instructions.
Locate Information (location test): For example, finding information in a reference manual.
Understand and Remember Content (understandability test): For example, learning something through reading.

If you are testing for a newly designed website, all three categories are needed to ensure accurate results.

Testing

There are three main types of usability tests:

Performance Test

Performance tests are instructions that assist the reader in performing an action. Provide the performance test to a tester in an environment that has the same tools, information, and references as your target audience. As they read the instructions, observe them without interference to see if they can perform the process without any complications.

Here are the four major elements of performance tests:

Tasks: Ask testers to perform the same tasks your target audience will perform.
Location: Use the same setting for testing that your target audience will be in.
Resources: Provide testers with the same tools, equipment, and resources that your target audience will have.
Information Gathering: Gather information in ways that will enable you to observe the details of the testers' experiences without interfering with their testing of your draft.

Location Tests

Location tests ask readers to locate information in reference manuals or websites. One way to test the effectiveness of your layout is to measure how fast the testers can locate information in the text. This type of test is valuable in measuring the quality of your headings, topic sentences, front matter, back matter, and determining if your links are properly guiding the reader through the text or website.

Understandability Tests

Understandability tests are extremely important just for the fact that if your reader does not understand or comprehend what you are communicating then the writing is useless to them. Understandability testing should be utilized in every writing test. The main way to conduct understandability tests is to have the test reader read the information and then answer strategic questions in order to get responses that will help guide your editing process.

Note: Conducting many small usability test drafts is far more efficient than conducting one large test after the finished draft is complete. If you are testing a web page, sketch the design on paper to be tested so you get a sense of what is needed before starting the website creation. Testing drafts more frequently will allow for many small easier revisions instead of one big overhaul of the draft.

After Usability Testing

After the test subject reads and uses your draft, the first step is to interview the person. This is used to gain more crucial information in addition to the information you observed during the testing. The questions vary according to the type of draft and the criterion of the usability test, but here are some general questions that could be asked:

What was the biggest problem you encountered in the draft?
What do you suggest?
What did you like about the draft?
Was the format user friendly?

After usability testing, the second step would be to analyze the data you collected. A very helpful way to analyze data is to input your raw data into an application such as Microsoft Excel from which graphs and tables can be constructed. From the test and the interview, you should be able to determine specific areas of your draft that need improvement or revamping. Examples could be anything from simply rewording to make more sense or possibly even a complete revamping of your draft. For this reason, usability testing is a great tool for determining the effectiveness of your draft.

The third step would be to implement the necessary changes to your draft. After the changes are made, further testing is often carried out to see whether or not the changes worked. It is important to realize that your first draft of instructions are usually not perfect and multiple revisions and testings are often necessary.

How to Make Usability Testing More Effective

Usability testing can often be a lengthy process. For this reason you want it to be as effective as possible. Listed here are some ideas to make your usability testing more effective.

Make sure you are prepared to take notes on your testing. Have documents with charts that can be filled with text during the testing process to make data more easy to analyze
Ensure testers are part of your target audience.
- ex. If you have a set of instructions designed for adolescent men, do not perform usability testing on women or men outside the specified age group
Have all the materials for the tester be within reach, or easily available
Since you will be testing multiple people on a given day, bring enough testing material so you can perform all the tests you wanted to do
Be sure to be descriptive about what criteria you will be recording. You can make a number system to rank experience or possibly overall clarity of whatever is being tested.
Also leave room for a short descriptive interview at the end of the usability sheet to get the users thoughts and concerns. This will be valuable when you are revising your document to ensure it is of the highest quality and free of complications.

Avoiding Bias in the Testing Process

Avoiding bias is a big thing that can help save a lot of time spent revising. Test readers' responses need to be natural and must be similar to how the target audience would see the draft. Five simple things can minimize biased test results are:

Downplay the fact that the writing is yours. This will allow the test readers to be more honest because they will not be afraid of offending or criticizing you. Do not show expression when the tester is critiquing the draft because the tester may not be as honest if they see that the work is yours.
Pay attention to how you phrase questions. Make the questions so the reader has an open path to answer either way. If the format of your questions creates pressure to pick the ideal answer for the test reader, your results could be considered biased.
Only intervene into the testing phase if it is absolutely necessary. Letting the test reader struggle through issues in the draft himself will allow for pure results that are unbiased. Of course, intervention should be used if the test reader is in danger of injury.
Make sure the environment is as distraction-free as possible for the test reader. Keep your distance! You do not want to impede in the testing process distracting the subject.
Ideally, if possible, watch the test from a video camera, one-way mirror, or afar while somebody else conducts the test. However, if this person conducts the interview also, he probably will not summarize the answers the same way you would.

Cross-cultural Testing

If you are going to have your writing read by people around the world or in different cultures, you are going to have some extra issues to consider. This is because every culture has different beliefs and norms. Writing in one culture that is clear and simple can be complex and inappropriate in another culture, unless special measures are taken when conducting usability testing. For example, when Breathe Right(R) nasal strips were going global, their box contained dark coloring because the product was supposed to be used to clear up the nasal passage during the night time. However, this dark package theme did not work well in Europe. They saw it as dark and gloomy and they wanted the package to be light and clear. The key to obtaining global technical communication is to have usability testers from that culture.

Ethics in Usability Testing

Due to the fact that technical communication is lawsuit prone, appropriate measures should be taken even when usability testing is used. Technical writing testing has the same loose guidelines as research in medicine and psychology:

The volunteer must be informed and fully aware of the process.
The volunteer must have agreed and given approval to participate in the testing.

For some types of tests, this is only an ethical issue. However, there are situations where there are legal restrictions for the researcher to inform the volunteer of the situation in writing.