Summary: According to a recent analysis of critical incidents, users’ most important web tasks involve collecting and comparing multiple pieces of information, usually in order to make a choice.
Traditionally, critical incident analysis has been an effective tool for gathering user feedback about an existing interface. Users are asked to recall a representative occasion when an interface was especially useful or particularly disappointing. I usually ask for both positive and negative examples, and the responses consistently help me understand how people use the system and how it could be improved by changing the prominence of particular elements.
Unfortunately, critical incident analysis is less useful for many web projects for two main reasons:
-
The website or new feature may not yet exist, so users have no actual experience with it.
-
Websites frequently fail in ways that are critical to the company but unimportant to users, who simply leave and visit another site. Users rarely remember why they left a website even one or two minutes later. Abandoning a shopping cart because shipping costs could not be found is not a memorable event. This is why some analysts’ surveys about “why users abandon shopping carts” provide almost no value.
Xerox PARC Research on Critical Web Incidents
Researchers at Xerox PARC recently published what might be called the mother of all critical-incident studies. The central question was: What important things do people do on the web as a whole? Although a single website may not produce useful critical incidents, a user’s complete online experience certainly can.
Julie Morrison, Peter Pirolli, and Stuart Card collected responses from 2,188 people to the following request:
Try to recall the most recent occasion when you found important information on the internet and, as a result, took a significant action or made an important decision.
The obvious weakness of this request—and of the critical incident method as a whole—is that it does not examineordinary users’ web activity, but focuses only onimportant uses. For example, only 2% of respondents mentioned reading news when describing a critical incident, while another study of the same users’ online activities found that 24% regularly read news.
However, this limitation can be turned into a feature. Understanding what users consider important on the web offers several advantages:
-
Critical tasks are more likely than ordinary tasks to support value-added services for which users are willing to pay.
-
If you can support important tasks, users are more likely to seek your help with routine tasks as well.
-
By understanding what users value most, you may gain insight into what is distinctive and exciting about the web, inspiring innovation.
Primary Method: Goal-Directed Collection
The PARC researchers analyzed how users described acquiring the information needed to complete critical tasks.
-
Collecting: 71%. Users searched for multiple pieces of information. They were driven by a specific goal but were not looking for one particular answer.
-
Finding: 25%. Users searched for a specific item.
-
Exploring: 2%. Users wandered without a clear purpose.
-
Monitoring: 2%. Users repeatedly visited the same website for updated information. These visits were triggered by routine behavior rather than a specific goal.
The most obvious conclusion is that, during critical web use, users are almost always goal-directed: the PARC study found this to be true 96% of the time. Although this principle has long been common knowledge, the size of the percentage still surprised me.
Interestingly, finding multiple pieces of information was almost three times as important to users as finding a single item. Yet the entire browsing model is optimized for visiting one location at a time. When users need to collect several answers, they are usually left to manage the process themselves.
Primary Task: Comparing and Choosing
The study classified the main reasons respondents depended heavily on the web as follows:
-
Compare/Choose: 51%. Evaluating multiple products or answers in order to make a decision.
-
Acquire: 25%. Obtaining a fact or file, learning about a product, or downloading content. (Morrison and colleagues used the term “find” for these tasks, but I prefer “acquire” because it distinguishes the goal from the method described above.)
-
Understand: 24%. Developing knowledge of a subject, often by finding facts or documents.
Important tasks were therefore divided almost evenly between situations in which users were making decisions among multiple options and situations in which they were pursuing a single outcome.
Usability Implications: The 3C Test
Collecting, comparing, and choosing summarize most of the critical activities involved in web use. When planning usability studies, we should therefore ensure that test tasks address all three.
Usability research should, of course, also test simpler tasks. We should not ignore less important aspects of web use, because they occupy more of users’ time. However, given the web’s current lack of support for the three Cs—collecting, comparing, and choosing—we need to place greater emphasis on them so users can complete their most important tasks more effectively.
