Jump to content

Tutorials

Card-Sorting and Cluster Analysis for Information Architecture Design

Monday, March 31st, 9:00 AM - 5:00 PM

Photo of Janice James Janice James
Director, User Experience, Perficient, Inc.
Voice: 703-771-9148
Email: Janice.James@perficient.com
www.perficient.com

Photo of Carol Righi Carol Righi
Director of User Experience, Perficient, Inc.
Email: Carol.Righi@perficient.com
www.perficient.com

A challenge of web site and application design is to organize the content so the user can easily access it. A key to addressing this challenge is to create the information architecture from the user's point of view. Card sorting and cluster analysis are methods that have been used increasingly in the HCI field to accomplish this goal.

This highly interactive tutorial addresses issues surrounding card sorting and provides Best Practices guidance and hands-on experience in the use of card sorting and cluster analysis methods and tools. The tutorial was successfully presented at the UPA, CHI, and HFES conferences in 2005.

This tutorial, taught by two of the industry's best trainers in user interface design, will be useful to marketing specialists, product planners, product managers, usability/UCD practitioners, interface designers, information architects, and technical communicators who would like to learn an end-to-end method for performing a card sort. Attendees will have some background and familiarity with the concept of information architecture. They may or may not have performed this activity in the past using other tools and methods. The tutorial is designed for both those new to the field of usability/UCD, as well as for advanced practitioners who are very familiar with the concept, but wish to learn a new tool and method.

Tutorial Structure

This tutorial will provide user researchers with practical guidance and hands-on experience in the use of card sorting and cluster analysis methods and tools. This highly interactive tutorial will include lecture, discussion, and hands-on exercises in card sorting, cluster analysis, and the tools used to perform these methods. It will address issues surrounding card sorting and cluster analysis, and review actual projects in which these tools and methods were used.

In brief, this tutorial will address:

Schedule

  1. Introductions
  2. Overview (session objectives)
  3. What is a card sort?
  4. Why are card sorts useful?
  5. Mechanics of performing a card sort (Exercise)
  6. Analysis of data (Exercise)
  7. Group presentations (Audience participation)
  8. Tools
  9. Limitations and caveats

Tutorial Detail

  1. What is a card sort? This introductory segment will begin with a very brief demonstration of a manual card sort using index cards. A couple of attendees will be asked to sort items into categories, and name the categories. This exercise is simply intended to "get everyone on the same page" with regard to the essence of the card sort. We will then cover a brief history of the use of the card sort in information architecture. We will describe precursor methods to the method that will be described in this tutorial (such as Bridge, and physical card sorts using index cards or sticky notes). We will then complete this segment with a discussion of the use of card sorts in current practice.
  2. Why are card sorts useful? In this segment, we will discuss the various applications of this methodology, including the following:
    • Organization of menu structures for both web-based and desktop applications.
    • Content organization for information-rich web sites and desktop applications.
    • Organization of the content of documentation.
    This section will include discussions of case studies of projects for which card sorts have been applied, such as Cabelas.com and UAB.edu (University of Alabama at Birmingham). The case studies will address the objectives of the web site, why a card sort was selected as a method to use to achieve those objectives, and at a high-level, how they were performed and applied.
  3. Mechanics of performing a card sort. This segment covers the step-by-step method for performing a card sort. We will use the WebSort, a web-based tool, to give attendees hands-on experience with performing a card sort using WebSort and analyzing the data. We will provide the data that attendees will work with through this segment. The steps of the process will be clearly enumerated and explained in their proper sequence. These steps will include:
    1. Define the audience. We will discuss why it is critically important to define the audience for the web site or application, and address issues such as dealing with multiple audiences.
    2. Recruit. As with all aspects of UCD, it is necessary to find a pool of participants who are potential users of the resulting design. We will discuss issues such as how to recruit, how many to recruit, and reimbursing participants. We will show some examples of recruiting tools.
    3. Inventory the content. We will discuss the need to develop a complete inventory of a planned or existing web site or application, and show some examples of how best to develop the inventory. We will address issues such as how to avoid bias when providing descriptions for each of the items, how to determine the depth of content to include, and how to determine the appropriate terms to use for topics to avoid reflection of the current content for existing sites/application.
    4. Input the content into WebSort. We will demonstrate how the study administrator sets up a WebSort study. We will demonstrate how the "cards" are created, the mailing list is prepared, and other aspects from the point of view of the administrator.
    5. Use WebSort. We will engage the attendees in en exercise where they will be the participants in a card sorting study. We will have prepared a set of "cards" they are to sort. All attendees will perform this activity.
  4. Analysis of data. This segment will address the second phase of this process: analysis of the data using cluster analysis. We will begin with a brief introduction to Cluster Analysis, which will include a conceptual discussion of the statistics behind the method. We will discuss how it has been used in the past, especially in marketing studies. We will contrast it to the "eyeball" method typically used with the data generated by manual card sorts.

    We will then return to our case study in progress. We will demonstrate the WebSort cluster analysis tool and explain how to use it. Attendees will then have an opportunity to use the WebSort tool to analyze the data. We will form teams of attendees. The teams will manipulate the tool to generate categories. Each team will be asked to create a basic architecture (categories and labels). We will have each team present their results, sharing the thought process that led them to their design. We will discuss how the same set of data can potentially generate multiple versions of an architecture. Other issues from the attendees' experience will be discussed.
  5. Tools. This segment will cover some tools that are used to facilitate the process of the card sort and cluster analysis. We will discuss the use of:
    • EZSort, WebSort, and others. An overview of the card sorting tools currently available.
    • EZCalc and the WebSort cluster analysis tool. An overview of the cluster analysis tools currently available.
    • Excel. How spreadsheets are used to organize and sort the information in a content inventory, track recruits, and display results.
    • Visio or other graphics applications. Simple logic flow diagrams used to depict the site architecture.
    • Wireframe. Interactive .html pages used to show the architecture "in action."
    • Surveys. Recruiting, priority studies, etc., performed prior to a card sort can be performed using survey tools.
  6. Limitations and caveats of card-sorting. This segment will cover some of the current issues around card sorting, such as variations in the methodology, limitations to the current practice, and possible directions for the future. These include:
    1. Open vs. closed card-sorts. Whether participants are allowed to introduce new items will change the manner in which results are interpreted.
    2. Renaming content. Because terminology can affect usability, it is often valuable to have users name the content as well as the categories. We will discuss when this is and isn't recommended, and how to alter the methodology to accommodate this step (i.e., getting user feedback on the labels of the items after they've completed the sort).
    3. Users' thought process. Asking users to articulate their rationales while or after they've completed the sort can provide more insight in to the user's mental model for the content, and can be helpful in developing the architecture.
    4. Multi-level sorts. Currently, card sorts allow only one level of categorization. A variation in the methodology can help formulate lower-level categories into higher-level categories.
    5. Card sorting and cluster analysis for large-scale information systems. In addition to "classic" web and application design, this method can and has been used for large systems (Dong, 2002)
    6. Remote vs. face-to-face studies. We will discuss advantages and disadvantages to each.
    7. Individual vs. group studies. We will discuss the implications to multiple individuals performing the sort to obtaining real-time group consensus by having groups perform the sort.
    8. How many participants are needed? We will discuss a recent study by Tullis and Wood (2004) that provides guidance on this topic.

Tutorial Materials

The tutorial notes will include the workshop presentation slides, detailed descriptions of key content from the lecture portions of the tutorial, worksheets for the exercises, and a bibliography of recommended readings and tools to research. The following are a few sample presentation slides:

Slide 1

Sample slide 1 from the card sorting tutorial


Slide 2

Sample slide 2 from the card sorting tutorial


Slide 3

Sample slide 3 from the card sorting tutorial


Slide 4

Sample slide 4 from the card sorting tutorial

About the Trainers

Janice James, Director, User Experience, Perficient, Inc.

Janice joined Perficient in 2006 with more than 18 years of experience in the field of human factors, usability and user-centered design and development for software, hardware, travel, Internet, financial services, education, health care and retail products and services. During the last eight years as principal of her own consulting firm, Simply Usable through Design, she has focused on providing usability engineering and interface design consulting services to companies ranging in size from start-ups to Fortune 500s, including CNet, AOL/Netscape, Hewlett Packard, Unisys, Intuit, Humana, BestBuy.com, eBay, AARP and McGraw-Hill, to name a few. Prior to forming her own consulting company in 1998, Janice introduced usability and user centered design to three major corporations: Netscape Communications, AMR's SABRE Travel Information Network and Electronic Forms Systems (EFS). Within all three corporations, she led the efforts in developing and then managing usability and UCD programs. She was instrumental in educating and raising the awareness of thousands of designers, developers, and managers about the benefits of and process of iteratively integrating UCD into the development lifecycle. Across each of the corporations, Janice actively promoted usability engineers as vital contributors to the development of usable products and services.

Janice has a broad range of experience in all types of usability and user-centered design processes. She has special expertise in establishing positive client relationships, providing team leadership and in analyzing client needs and proposing appropriate and cost-effective usability methods that are conducive to the various phases of development lifecycles and that meet specific business goals and usability requirements. Janice is experienced in effectively managing all phases of a project.

Janice's career is highlighted by her leadership in professional associations. In 1990, she recognized the need for a practical-oriented organization dedicated to enhancing the skills and professional success of usability professionals and became the principal founder of the Usability Professionals' Association. After serving as UPA's President for five years and and developing four association conferences, she continued playing an active role in the association's growth by serving on the Board of Directors through 1999. She co-authored and developed the guidelines for Chapters and directed other leaders in the development of many of the early UPA chapters. Janice also founded the Kentuckiana Chapter of STC and co-founded the Society for Technical Communications' Professional Interest Committee on Usability.

Janice holds undergraduate and graduate degrees in English and Technical Communications. She has completed graduate work in Human Factors and HCI and is a published author and frequent speaker at professional conferences. Janice was a contributing author of Usability in Practice, a book of case studies on starting and running usability programs in corporations. She is also the co-author of a forthcoming book on UCD case studies.

Carolyn Righi, Director of User Experience, Perficient, Inc.

Carol Righi is the Director of User Experience for Perficient's St. Louis Business Unit. Carol has worked in the area of user experience and User-Centered Design for more than 20 years. Carol is widely considered a UCD thought leader, is published extensively, and has presented at numerous professional meetings worldwide. She is co-author of User-Centered Design: An Integrated Approach and is a featured columnist for TaskZ.com. She is currently completing work on a second book with Perficient colleague Janice James.

Carol has been both a manager and a practitioner throughout various point of her career. Her skills include interaction design, information architecture, usability engineering, and training. Carol has designed user interfaces for the Web, and for desktop, kiosk, and mainframe applications. She has planned and conducted extensive user research studies. She has served as manager for numerous design and evaluation efforts. She has helped lead the development of the UCD competency in various organizations, including IBM. Carol has also designed and developed many standup and e-learning courses in UCD. She has trained thousands of students.

Carol is a member of the Usability Professional's Association (UPA); SIGCHI (the human-factors special interest group of the Association for Computing Machinery). She serves on the editorial board of the peer-reviewed UPA online journal "Journal of Usability Studies," and is frequently sought out to review journal articles, conference submissions, and other scholarly and professional works. Carol is Usability Co-Chair for CHI 2007.

Carol has worked on projects across the industry for companies such as IBM, Google, Yahoo, Intuit, A. G. Edwards, Mapquest, Met Life, and many others. Past positions have included consultant for IBM's Global Services; Manager of Usability for IBM's Personal Software Products division; Manager of Academic User Services at Teachers College, Columbia University; Guest Lecturer at the Weizmann Institute of Science, Rehovot, Israel; and various other positions in consulting and educational services.

Carol received her undergraduate degree in psychology in 1981 and her Ph.D. in school/educational psychology in 1988, both from Fordham University in New York City with a concentration in computer applications to education.

IUE 2008 - Dramatically improving today's web site user experience