Add Focus To Your Conversion Insights
Get 14-day free trial

Card Sorting Reveals Usability Problems of US Universities’ Websites 19 December 2013 | Natalia Jatczak

As a continuation of our recent research concerning usability of 8 American Universities’ websites, we’ve decided to test them from a different angle (see the previous case study). We wanted to find out if the menu elements are located in the places where users look for them. For this purpose we’ve run a card sorting study using UsabilityTools.com.

Card sorting is a method used to help design or evaluate the information architecture of a site that helps people navigate through the content easily. It shows how particular pieces of information should be organized in order to make sense to the users.

A Card Sorting example

A Card Sorting example

This time we’ve tested the menus on ‘Undergraduates Admission’ pages. We’ve chosen 5 out of 8 websites that were previously tested. The selection was based on the menu architecture and the presence of ‘Undergraduate Admission’ page. The study was performed by 300 respondents from the USA (60 per website) and consisted of 2 parts. The first task was Click Testing, in which participants were asked to click on the element they would remove from that page. The second task was Card Sorting, where respondents were categorizing subpage’s labels.

A Card Sorting example

The results brought a couple of interesting points that you should take into account when designing a successful information architecture:

  • Firstly, you should think through the placement of subpages in the menu structure and name the categories properly.
  • It is also important not to give the same name to two different subpages (as obvious as it sounds, there were instances of this happening on the websites we tested).
  • Additionally, if you want to put the video on your website you should think about a good placement for it.
The Results

To show the general results we’ve counted two values. Firstly, the percentage of answers, in which the majority of respondents categorized subpages in the way they matched the actual information architecture of the website. And secondly, the average number of categories chosen by different respondents for one subpage title.

In the case of Princeton University, 80% participants’ categorization matched the actual website architecture, for CalTech it was 63%. Moreover, these websites were also the best and the worst when it comes to the second value (Princeton 3.37 and CalTech 6.24).

A Card Sorting example

Compatibility with actual web architecture and average number of categories

In order to explain the results we have created a list of 5 mistakes that you should not make on your website:

1) Important subpages in places users don’t expect them to be.

Princeton did perform best in our study, they didn’t, however, avoid some mistakes.

Even though logically ‘Financial Aid Without Loan’ fits into the ‘What’s Distinctive About Princeton’ category, the card sorting study revealed that participants expected it to be in ‘Cost & Financial Aid’ category. A possible solution could be adding a link to this page also to the second category.

Noone put the page ‘Financial Aid Without Loan’ into ‘What’s Distinctive About Princeton’ category. 98% chose ‘Cost & Financial Aid’ category.

Noone put the page ‘Financial Aid Without Loan’ into ‘What’s Distinctive About Princeton’ category. 98% chose ‘Cost & Financial Aid’ category.

Another example of this issue could be identified in CalTech’s menu:

‘Accomodation for Disabilities’ - 52,6% of respondents would put this page into ‘Living  at CalTech’ category and only 16% of respondents into ‘Learning at CalTech’ where it was on the website.

‘Accommodation for Disabilities’ – 52,6% of respondents would put this page into ‘Living
at CalTech’ category and only 16% of respondents into ‘Learning at CalTech’ where it was on the website.

2) Ambigous categories

On Stanford’s website there was ‘The Basics’ category. This somewhat vague category label turned out to be very confusing to people. What is more, this category included many important subpages like ‘Application Requirements’ and ‘Our Selection Process’. Only 3% of respondents matched these two subpages to ‘The Basics’ category, whereas 91,5% and 85% thought that ‘Application Process’ category would be more fitting.

Only one of the 5 subpages from this category (Request Information) was correctly matched by the majority of respondents.

Only one of the 5 subpages from this category (Request Information) was correctly matched by the majority of respondents.

This may mean that the ‘The Basics’ category may be confusing to the visitors who look for particular pieces of information. On the other hand, some visitors who entered the website with no specified goal may find it useful. This could be investigated further e.g. in user testing.

The unclear information architecture can be even more problematic to the visitors if the navigation requires clicking to see the subcategories (see the image below).

To see the subpages you have to click on the category label.

To see the subpages you have to click on the category label.

3) Names of subpages hard to understand

Once again, this issue concerned mostly CalTech. Labels’ names were very vague and seemed to use unclear terminology, for example, ‘3/2’ or ‘QuestBridge’. The card sorting study showed clearly that these particular labels created a lot of confusion as respondents sorted it to various different categories (see table below).

Name of the category Number of respondents
Learning at CalTech 12
What is CalTech? 10
Upcoming Events 8
After CalTech 5
Visiting CalTech 4
Who’s a Teacher? 4
Living at Caltech 3
Applying to Caltech 2
Affording Caltech 2

It was only after clicking on these elements the full labels appeared (‘3/2 Applicants’ and ‘QuestBridge Applicants’). If the word “Applicants” had appeared already on the label it could have been easier to understand and classify correctly.

4) The same name for different subpages

We’ve found also an interesting fact on the Johns Hopkins’ website. There was a category and subpage with identical names – ‘Student Life’. No wonder that great majority of respondents would put this page into the category with the same name. On the website’s menu, however, not only was this element placed in a ‘Fast Facts’ category, but also after clicking on both ‘Student Life’ labels, different subpages were opened.

We have already touched upon this problem in the previous case study, but it’s worth repeating – try to avoid creating different subpages with identical labels as it will only increase confusion.

5) Video on a homepage

The opinions whether the video on the homepage increases conversion or not are split. When you want to put it anyway, you should definitely think through the video placement on the page. Both Princeton and Stanford decided to put videos on their Undergraduate Admissions pages. In the Click Testing task, subsequently 46,67% and 40,38% of respondents chose the videos as elements they would remove from the websites. This data was quickly revealed thanks to Area of Interest functionality. It is enough to mark an element to get the quantitative analysis of the clicks in a given area.

Results of the Click Testing task on Princeton’s website.

Results of the Click Testing task on Princeton’s website.

Results of the Click Testing task on Stanford’s website.

Results of the Click Testing task on Stanford’s website.

As you can see, even when pages had good overall scores in the card sorting study, there were some mistakes which may cause problems with finding the pieces of information on the website. To make sure you’re not making one of these mistakes run the card sorting study and evaluate your menu!


Natalia Jatczak

Written by Natalia Jatczak

She is a Cognitive Science lover with a bag full of hobbies including reading, watching movies & sitcoms, working out and traveling. Time management is her specialty, apparently.


  • Jonathan Clive Mukwaya

    Thank you very much for sharing your incredible insights on this information
    Keep doing your thing because you will go far

Ready to empower your website's potential?