Masters Project - Literature Review

To support the new task of coaching, GLISI wants to implement an online support tool to provide immediate and ongoing scaffolding for the coaches after the initial training session. A literature review was done to discover the principles and theories that should guide the design and implementation of the support tool. The principal theory that emerged was performance-centered design (PCD), but there was some concern that the requirements for the system GLISI was envisioning did not meet sufficient criteria for an Electronic Performance Support System (EPSS), the manifestation of PCD. This concern led us to the areas of user-centered and learner-centered design in search of an overarching framework on which to base our system.

This review aims to uncover enough about each area of research to lead to the discovery of which does or does not contain principles and methods that apply to the design of our system. The review’s construct is meant to reveal how different aspects of each design methodology meet the needs of the web support tool.

PCD is a design approach detailed by Gloria Gery after years of observing the implementation of her revolutionary approach to employee training called EPSS. Gery defines the two goals of PCD to be “1) to integrate the knowledge, data, and tools required to be successful in performing a task, and 2) to provide task structuring support to help performers create required deliverables.” PCD experts, Barry Raybould and Craig Marion agree this support is delivered in one of three ways intrinsically, extrinsically, or externally (Gery 1995; Marion 1997; Raybould 2000). Intrinsic support is embedded in the interface with seamless integration into the workflow, extrinsic support is also integrated but not integrated into the workflow, and external support is not integrated to any degree. Gery sets the goal for any system striving to support optimal performance to be 80% intrinsic with the remaining 20% divided between extrinsic and external (Gery 1995). Raybould suggest these support styles should be thought of as a continuum. Any system’s aim should be to provide intrinsic support, the cheapest and quickest method, and move towards extrinsic first and then external as various methods become less practical (Raybould 2000).

Raybould and Gery both have established a set of criteria or heuristics for effective PCD. What emerges from an analysis of both sets is that a PCD system must provide layered support that can be adjusted for the individual, structure interaction toward achieving a goal while reflecting the work context, and incrementally supply knowledge when the user needs it while omitting what is no longer needed (Gery 1995; Raybould 2000). Many items on their lists reflect UCD recommendations that came out of research in the field of human-computer interaction, which is the atmosphere within which PCD developed (Marion 1997). Other scholars caution about drawing the fields too closely together, strictly defining UCD methods as narrowly focused on the system whereas PCD focuses additionally on work to be done (Raybould 2000). Winslow and Bramer state, “We don't want workers to be thought of as system "users"; that makes the system itself the dominant focus. The focus, again, should be performance-centered. We want to enable workers to do their jobs, not focus on having to use a system.” (Winslow 1994) However, John Karat believes the ISO Standard definition of usability belies that notion. The definition states, “Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Karat believes this is an “acknowledgement that usability is complex and context dependent” which is closer to the motivation behind EPSS than has traditionally been assumed. Participatory design and contextual inquiry are two facets of UCD that are particularly akin to EPSS and may impact the design of the COSS.

Participatory design is a methodology with origins in Scandinavia during the 1970s and 1980s. In an effort to advance the culture of industrial democracy, computer programmers worked with would-be sidelined factory workers to create products that improved productivity by automating some aspects of the job while respecting the time-refined craft and skill possessed by the workers by leaving those aspects preserved (Spinuzzi 2005). Participatory design is motivated by three goals: democracy; efficiency, expertise, and quality; and commitment and buy-in (Muller 1997). Spinuzzi emphasizes participatory design’s aim to capture and design for the user’s tacit knowledge. Tacit knowledge is generally difficult to articulate or observe, requiring the employment of non-traditional requirements gathering techniques. To elicit this knowledge, users of the system become partners in the discovery and design effort. They help gather requirements, brainstorm ideas, generate prototypes, and refine the design. Though this technique offers unique advantages, it can also be extremely time-consuming, a drawback that customers find hard to abide. For designers, the technique can limit the solution space to an area comfortable for the users who have been accustomed to going about their work in a certain way (Spinuzzi 2005).

A similar technique that does not yield as much power to the user is contextual inquiry. Contextual inquiry is an idea introduced by Karen Holtzblatt and Hugh R. Beyer for incorporating techniques from qualitative methods, such as participant and non-participant observations, focus groups, and interviews, into the requirements gathering phase of design. It puts designers into the role of apprentice and customers into the role of master to increase the likelihood of the designer understanding the intricacies of the users job with an eye toward how a system might be incorporated into the role (Beyer 1995).

All of the above methodologies provide valuable strategies for approaching the design of an online support website. As a new program, there are no existing users who could help in the design of the website. Therefore, the participatory design and contextual inquiry methodologies will not have a place of impact in this project. However, the field of PCD and existing examples of EPSS do provide a rich resource from which the design of the support website will be drawn. The next appropriate step is to analyze the existing strong examples of EPSS systems from both a UCD and PCD perspective.

References

Beyer, H. R., Holtzblatt, Karen (1995). "Apprenticing with the Customer." Communications of the ACM 38(5): 45-52.

Gery, G. (1995). "Attributes and Behaviors of Performance-centered Systems." Performance Improvement Quarterly 8(1): 47-93.

Marion, C. (1997). "Implementing Performance-Centered Design." http://www.chesco.com/~cmarion/PCD/ImplementingPCD.html.

Muller, M. J. H., Jean Hallewell Dayton, Tom (1997). Participatory Practices in the Software Lifecycle. Handbook of Human-Computer Interaction. M. G. L. Helander, Thomas K. Prabhu, Prasad V. Amsterdam, Elsevier Science B. V.: 1582.

Raybould, B. (2000). "Building performance-centered Web-based systems, information systems, and knowledge management systems in the 21st century." Performance Improvement 39(6): 32-39.

Spinuzzi, C. (2005). "The Methodology of Participatory Design." Technical Communication 52(2): 163-174.

Winslow, C. D. and W.L. Bramer (1994). FutureWork: Putting Knowledge to Work in the Knowledge Economy. New York, Andersen Consulting