Most everyone agrees that web site design should be tailored to the specific needs, moraes, and habits of groups based on culture. However, some situations involve the interaction of groups from many different cultures at once. Examples might include social media users networked with people worldwide; multi-national business sites; and educational support systems combining US and international students.

I conducted a study with multiple participants (24 US citizens and 24 non-US citizens) located in the United States that elicited an inventory of the specific preferred elements (icons, labels, menus, language elements) that constitute a popular shared web interface. My study also examined how users actually perform tasks on that shared site. Additional research was conducted at the time about how the users cognitively imagine and map out websites through a card sort process but this had to be excluded from the final article. A preliminary account of the card sort part of the research is provided here.

What is card sort?

Card sorting is one common method by which usability researchers and practitioners obtain some conception about how users organize information. Users are presented with a set of cards labeled with specific topics that appear (or might appear) on a website and are asked to group them into categories that makes sense to them. A closed card sort provides overarching topic categories for each group in advance, while an open card sort allows users to decide for themselves what the categories should be called. The results of a card sort can be statistically compared and calculated through cluster analysis, multi-dimensional scaling, statistics, and other methods available in certain UX analysis software.

Method

Users in each group of 24 were given a set of 17 topic cards extrapolated from a university computing technology website. Users were asked to self-select categories or groupings for the cards and supply a category title for each group of their choosing. However, some users used an existing card from the deck as a category or made slight changes to existing cards used as category markers rather than create new categories. In these respects, while users were free to perform an open card sort, some ended up doing something closer to a closed sort. During the analysis phase, category group labels were streamlined according to best intended meaning (for example, if one user created a category called "using VPN" and another "VPN connectivity," the two categories were streamlined into one for purposes of comparative analysis.)

This card sort occurred prior to the performance of tasks and hence, the results reflect user topical preferences without prior exposure to the actual site itself during the test session. This was done in order to best determine how users would organize the information without benefit of seeing how the site actually organized the items. However, a small portion of the users indicated exposure to the website prior to the study so it is not entirely the case that all users were operating from a naive perspective about the site. Card sort results were compiled with the USort and EZCalc freeware applications by using the "average linkage" method.

Dendrogram charts show similarities and differences between the two groups of participants. The international participants generally divided topics into three categories, and the Americans divided topics into four categories. Within-category content of topics is relatively similar between-groups. This similarity is visually discernable through dendrogram summary graphs below:

Comparing Card Sorts Between US and International Users

The edit distance metric developed by Deibel, Anderson and Anderson (in "Using edit distance to analyze card sorts." Expert Systems 22 (3), 129-138) was was used to provide a more precise numerical measure of similarity between the two populations. This metric is similar to the Levenshtein method of string comparison in computer science algorithms, but the method is applied to two-dimensional matrices. The method measures the minimum number of moves needed to make two different matrices identical using insertions, deletions, and replacements of items. The edit distance can range from 0 (identical card sorts) to n (the total number of cards sorted, reflecting a completely different set of card sorts).

The average edit distance between international and American participants was 3, indicating that 3 moves were needed to make the card sorts between groups identical. The edit distance between the American participants and the actual topical organization on the website was 4, whereas for international participants, the distance was 3, suggesting that the university web designers provided a site that may have come closer to approximating the international user student population than for that of American students. Rather than being marked by sharp divergent curves, the close proximity of the dendrogram summary graph further reinforces the similarity with which both groups organized the information for the site.

Conclusion

This brief part of the published study focused on the card sort results of international and American participants and found differences between the two groups. Further research would need to examine why these results occurred. Possibly, the university IT staff responsible for web development and maintenance is comprised of individuals originally from outside the United States, or perhaps the international students have more exposure to these types of sites, which might have been a primary cultural conduit prior to physical arrival. The institution comprises 4 separate campuses located in a major urban area with a high immigrant and transnational population.

The close results might also have to do with the nature of the user participants, which compared 24 people from one culture to 24 from several other cultures bunched together. Hence, edit distances between card sorts might normally be expected to be high between American users and users of a single particular culture. When bunched with other cultures, the results may be flattened out somewhat, with strong preferences among distinct ethnicities canceling each other out. Future research is needed to compare edit distances between two (or more) distinct groups.

Finally, the edit distance method does not take into account the multi-tiered architecture of a sort (that is, how users subcategorize topics) but rather, treats all sorts as first-level groupings as it is only concerned with clusters of similarly grouped topics.