Inspecting Usability of Rich Internet Applications by Extended Cognitive Walkthrough
Masahiro Hori1 and Takashi Kato1
1

Faculty of Informatics, Kansai University 2-1-1 Ryozenji-cho, Takatsuki-shi Osaka 569-1095 Japan {horim, tkato}@res.kutc.kansai-u.ac.jp

Abstract. This paper addresses the importance of cognitive accessibility and cognitive usability as critical requirements for the next generation Web applications. Information should not be said to be accessed unless its content is cognitively internalized or understood by the user. Accessibility of information, therefore, should be evaluated not only for its perceivability but also for its understandability. We proposed a new cognitive walkthrough (CW) method whose CW questions were formulated based on an extended HCI model that distinguishes between perceiving and understanding. The modified CW method is presented as a viable means of evaluating emerging Rich Internet Applications. Keywords: Usability inspection method, HCI model, Rich Internet Application, cognitive walkthrough

1

Introduction

The Web is becoming not only heterogeneous and dynamic as a universal medium for the exchange of data, but also increasingly ubiquitous as a platform of content delivery. In order to achieve the full potentiation of the Semantic Web user interaction, it is important to consider aspects of content delivery. The concept of delivery context is characterized by a broad range of attributes related to device capability, network connection, human-computer interaction, geographical location, and so on [1]. New generation of Web applications allow users for open, online participation and collaboration, as well as dynamic integration of multiple Web resources. Especially in the aspect of the human-computer interaction, emerging Rich Internet Applications (RIAs) do not necessarily follow the conventional page-by-page interaction, but realize a more responsive, desktop-like experience. However, building an application that simply uses new technologies such as mashups and Ajax does not ensure a better user experience. While focusing on the human-computer interaction side of the delivery context, this article raises an issue that may make the next generation Web applications accessible and usable.

User's Mental Activity

Establish the goal (Q9) Form the intention Evaluate the outcome

(Q1) Specify the action sequence Gulf of Execution Perceive the object (Q2) Interpret the object (Q3) Execute the command (Q6) Physical World System state changed Interpret the action (Q5) Perceive the action (Q4)

(Q8) Interpret the outcome

Gulf of Evaluation

(Q7) Perceive the outcome

Fig. 1. Extended HCI model.

2

Extended Cognitive Walkthrough

We have proposed [2] that the extended cognitive walkthrough (ECW) can be an effective means of going beyond perceivability in designing and evaluating Web accessibility and usability. We suggest that the ECW also be instrumental and useful in ensuring cognitive accessibility and usability of the next generation Web applications. The ECW is a variant of the cognitive walkthrough (CW) which is a usability inspection method aimed at evaluating the ease of learning user interfaces. Analysts attempt to give yes/no answers to evaluation questions and to indicate why they think the intended user can or cannot be assumed to successfully perform the required action. In order to meet practitioners' demands for improved ease-of-use of the method, the CW has been revised from the first [3] to the second [5], and to the third version [6] where the number of evaluation questions was reduced to four. Though apparently much simplified from the previous two versions, the third version brings about increased ambiguity in the interpretation of its CW questions. In our attempt [2] to make CW questions easier to deal with and more effective in identifying usability problems, we first extended Norman's Seven Stages of Action model [4] by incorporating two distinctions; "specifying object vs. action" and "perceiving vs. understanding" (Figure 2). Namely, the stage of specifying the action sequence, which is represented as a single process in Norman's model, is divided into

Table 1. Nine evaluation questions based on an extended HCI model.
HCI Processes (Q1) Form the intention (Q2) Perceive the object (Q3) Interpret the object (Q4) Perceive the action (Q5) Interpret the action (Q6) Execute the command (Q7) Perceive the outcome (Q8) Interpret the outcome (Q9) Evaluate the outcome CW Questions Will the user intend to achieve the right effect? Will the user notice that the correct object is available? Will the user know what the correct object refers to? Will the user notice that the correct action is available? Will the user know that the correct action should be applied to the correct object? Will the user be able to apply the correct action to the correct object without fail or difficulty? When the correct action is taken, will the user notice the physical change in the system state? Will the user know what exactly has happened to the system state? Will the user know the current system state is nearer to the completion of the task?

four sub-processes in the extended HCI model. Based on the HCI processes identified in the extended model, we then generated nine CW questions (Table 1) whose objectives are to examine if it is safe to assume that the intended user will successfully exit each of the HCI processes. This list of nine evaluation questions can be a valuable design aid to bridge the gaps between users' goals and physical states of computing systems. Any negative answer to any evaluation question is taken to be an indication of potential design problem under the evaluated context. The questions Q1 through Q6 are concerned with the processes of bridging "the gulf of execution" [4]. These six questions are expected to help analysts to detect usability problems that may stem from the gaps between what users intend to do and what and how they are allowed to perform. The questions Q7 through Q9 are related to "the gulf of evaluation" ¥cite{Norman:1998} and are expected to help analysts to identify what difficulties the users may have in assessing the state changes resulting from their actions. For examples, Q2 addresses the ease of perceiving the correct object to be acted upon by the user, and Q4 is to examine the ease of perceiving the correct action (e.g., a key press, a mouse click) to be applied to the correct object. In traditional pagebased Web applications, the availability of the correct object (e.g., a link text, an input form) and its associated correct action might be obvious to the user because a variety of standard controls is limited. However, in the case of Rich Internet Applications with advanced interaction capabilities such as drag-and-drop and inline editing, there can be no guarantee that the availability of the correct action is spontaneously noticed by the user. The questions Q3 and Q5 address the intelligibility of the perceived object. Perceiving the availability of the correct object (or action) and understanding the meaning of the perceived object (or action) are quite different matters. Consider an

example in which the user is to invoke the edit mode for a spreadsheet cell. Unless the user knows that in this particular situation invoking the edit mode requires doubleclicking, s/he would not attempt to apply a double click even if double-clicking itself is in his/her repertoire of actions. Such is a problem stemming from the difficulty of understanding the correct action (Q5) rather than perceiving its availability (Q4). In the gulf of evaluation, on the other hand, the question Q7 examines if the user can perceive any physical change on the part of the system (e.g., page transition on the browser). In addition to checking the feedback indicating something happened (Q7), the feedback indicating what happened is checked by Q8. The question Q8 attempts to confirm that the intended users understand the meaning of the system's state change. In the case of Rich Internet Applications, the display on a screen may be updated partially instead of reloading the entire page, and more detailed information can be provided on the same screen. Therefore, even if the users can perceive the partial update of the screen, they may still have difficulty in understanding the meaning of the state change.

3 Concluding Remark
The aim of this article is to draw particular attention to the importance of explicitly distinguishing between perceptual and cognitive aspects of the users' interactions in the next generation Web or Semantic Web applications in a broader sense. Since the CW questions of the ECW were of a finer grain level than those of the third version, the participants provided reasons for 'No' judgments more selectively (i.e., whether a matter of perceiving or understanding, object or action) and described them more concretely. Such selectively chosen, concretely described reasons for possible interaction failure should make it easy for developers to come up with viable design alternatives to improve the usability of that particular aspect of the user experiences.

References
1. Gimson, R.: Device Independence Principles. W3C Working Group Note 01 September 2003, http://www.w3.org/TR/2003/NOTE-di-princ-20030901/ 2. Kato, T. and Hori, M.: Articulating the cognitive walkthrough based on an extended model of HCI. Proc. of HCI International (2005) 3. Lewis, C., Polson, P., Wharton, C., and Rieman, J.: Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. Proc. of CHI '90, 235-242 (1990) 4. Norman, D. A.: Cognitive engineering. In D. A. Norman & S. W. Draper (Eds.), User Centered Systems Design: New Perspectives in Human-Computer Interaction. pp. 31-61, Lawrence Erlbaum Associates, Hillsdale, NJ (1986) 5. Polson, P. G., Lewis, C., Rieman, J., and Wharton, C.: Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, Vol. 36, No. 5, 741-773 (1992) 6. Wharton, C., Rieman, J., Lewis, C., and Polson, P.: The cognitive walkthrough method: A practitioner's guide. In J. Nielsen & R. L. Mack (Eds.) Usability Inspection Methods. pp. 105-140, John Wiley & Sons, New York, NY (1994)