Last month, the LDS.org team conducted an online study to find out why people visit LDS.org. On Tuesday and Friday evenings, one percent of site visitors were randomly asked about their experience with the LDS.org website.

Visitors were asked to explain, in as much detail as possible, why they came to LDS.org that day. Then they were asked to continue doing whatever they came to the site to do. Lastly they were asked to rate how easy it was to complete their task and comment on the main challenge they faced in completing the task. The team gathered more than 800 responses to various questions.

Faced with a wealth of valuable feedback from site users, the next step involved processing and interpreting the data. While it was relatively straightforward to report the quantitative data—time on task, satisfaction ratings, etc.—dealing with the large volume of qualitative responses was a bigger challenge.

To handle these responses, the LDS.org team used a brainstorming technique called “affinity diagramming.” Affinity diagramming organizes different pieces of qualitative data in a way that surfaces overall themes.

When done properly, affinity diagramming can prevent researchers from being overwhelmed by too much information. It also helps them avoid imposing their own interpretation based on just a few of the most memorable sound bites.

The LDS.org Affinity Diagramming Experience

Building an affinity diagram from the LDS.org survey results took two days for most of the study team, plus a few days of additional work by team leaders. The study team consisted of user researchers, product managers, interaction designers, and other stakeholders. It was important to select participants who had the ability to make decisions based on information from the affinity diagramming process. The participants received deep, first-hand exposure to the raw survey results to broaden their understanding of users’ motivations and biggest challenges while visiting the website.

To start the diagramming exercise, each survey result was printed on a yellow post-it note. The team gathered in a conference room with blank walls. The yellow post-it notes containing the raw data were laid out on a table in no particular order.

The team’s first task was to begin grouping the post-its together without much discussion, to avoid influencing each other. Throughout the exercise, the team members were asked to move around the data as they saw fit, even items that had already been grouped. This rule prevented the team from prematurely forming categories under which the data should fall, until the most logical and natural groupings had been reached by consensus.

Two team members arrange and re-arrange the post-it notes into groups that make sense to them. Although the post-it notes don’t have a single, correct grouping, it’s important that the groupings have a consensus among the team.

Allowing the Groups to Form Themselves

The goal of affinity diagramming is to achieve a bottom-up, hierarchical approach in grouping the data. Ted Boren, User Research lead and team participant, says that it’s necessary to “fight the desire to name or label categories too early” while building the affinity diagram. “You have to categorize early in the sense that you are grouping things right away. But you must also avoid naming the category too early. Then your grouping will be consistent with the data rather than the labels," Ted says.

In other words, the goal of an affinity diagram is to be descriptive of data, not prescriptive. This is why no headings were assigned to the groups of data for the first three hours of the exercise. Everyone was encouraged to have an open mind and get the best overall sense of each response and its relationship to other responses.

The Next Level of the Affinity Diagram Tree

After the initial groups had been formed, the team members worked to formulate category headings for each group. For each group level, the team used post-it notes of a different color.

A team member prepares the category headings for a section of the diagram. First-level category headings were blue and attempted to describe the overall message of the user responses printed on the yellow post-it notes below.

The participants worked hard to write the category headings in the same way users’ might phrase them. The labels were always worded from the users’ point of view: “I need _____,” “I am trying to _____,” “I have trouble doing _____ ,” and so on.

The Next Level

After the groups had been formed, the participants proceeded to arrange the labels together into larger groups with more encompassing labels. In this case, the more encompassing labels were green. The green post-it notes expressed the message of the blue post-it notes below them.

Finally, at the very top level, pink post-it notes conveyed the message of the green post-it notes below them. At the end of the exercise, the affinity diagram resembles an inverted tree.

A section of the LDS.org affinity diagram resembles an inverted tree. The yellow post-it notes are the individual responses, grouped by similarity. Each differently colored post-it note describes the post-it notes below it.

The Results of the Study

The process of affinity diagramming was long and comprehensive. Nevertheless, the team managed to condense the 800 survey responses into 11 top-level categories. The following were the categories that resulted from the responses:

• I visit LDS.org often for inspiration and information.
• I’m preparing a lesson on an assigned topic.
• I have an assigned topic to study for a talk.
• I’m using the site for personal study.
• I’m looking for church media for lessons or personal inspiration.
• I need more info about wards, members, and temples.
• I need information on my calling.
• Search doesn’t always help me find what I need.
• I wish the navigation was simpler and more stable.
• I’m having technical issues on the site.
• I’m having a good experience on the site.

Various stakeholders can review these high level categories, drill into lower–level categories, and eventually drill down to specific responses that constitute those categories. Often the most powerful way to experience the data is in users’ own words; the affinity diagram provides a structured way to find those words and relate them to other responses.

Interpreting the Results

The results offer insight about the reasons people go to LDS.org, the issues they experience, and how their goals align with organizational objectives. For example, one of the Church’s purposes for the website is to draw members closer to Christ and guide them with words of the living prophets and apostles. Some people may not come with that intent, but it’s one of the outcomes the organization hopes will come about.

The results from the study can help reveal the gap between the intended purpose of the site and its actual use from the perspective of its visitors. Other teams can use these same affinity diagramming techniques to improve the usability and user experience of any site or product. For example, affinity diagramming could be used to help organize notes from field visits, menu items in an application, information topics in a help system, or products in a catalog. The real strength of an affinity diagram is the ability to help teams process and interpret large amounts of qualitative data.