upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

Use of Card Sorting for Online Course Site Organization Within an Integrated Science Curriculum

Alison Doubleday

Journal of Usability Studies, Volume 8, Issue 2, February 2013, pp. 41 - 54

Article Contents

Discussion and Conclusions

The results of the Cohort A open card sort were somewhat surprising to the curriculum planning committee. Anecdotal evidence suggested that both students and faculty members preferred to organize material by discipline, by topic, or by medium. In fact, prior to the first card sort, the planning committee suggested that sites should be organized by discipline within each didactic topic and then by medium because this seemed to make the most sense to faculty members, based on prior experience with Blackboard course sites. However, the Cohort A open card sort revealed that they, almost universally, wanted material organized based on frequency of access.

While it may come as no surprise that students think primarily in terms of their weekly schedule, it was interesting that faculty (at least anecdotally) did not consistently share this perspective. In fact, the pre-conceived notions of the planning committee (that students would prefer organizing content by discipline and then by medium) were, essentially, debunked by the data obtained from student participants. This may simply reflect the differences in overall schedules between faculty and students, or it may reflect a difference in the mental models of novices (content users) and experts (content producers).

These unexpected results, as well as the verbal feedback (complaints) from Cohort B students during their first year in the new curriculum, encouraged the second card sort. The planning committee became increasingly concerned that, even though students provided the data from the initial open card sort, the lack of experience with an integrated curriculum might have led to an inability in Cohort A participants to reliably predict logical information architecture for the new curriculum. Although Cohort B students were able to use the initial course sites throughout their first year in the new curriculum, the second card sort allowed the curriculum planning committee to make modifications to the template site before several years of sites were built.

It is possible that differences in organization identified between the two cohorts reflect the differences in curricula experienced by these two participant pools (traditional vs. integrated). For example, the Cohort B participants, exposed to an integrated curriculum with a focus on case/scenario-based learning, were much more accepting of supplemental resources and tended to integrate materials according to case/scenario rather than by discipline. This last point is conjecture at present; a larger study with an experimental design would be necessary to determine whether exposure to different curricula impact the process of organizing content. A larger, experimental study would also be required in order to determine whether the actions of consuming or producing content (differences between students and faculty, for example) impact content organization, as well. Although this case study cannot address these more broadly reaching questions, the results presented here do reveal that experience with the new, integrated curriculum encouraged Cohort B students to request additional modifications in organization of sites. As the Cohort B participants more accurately reflect actual site users, the Cohort B semi-closed card sort results provided valuable insight into a logical navigation for this population. The Cohort A open card sort and usability test provided an excellent starting point and enabled us to put together an initial template site that could lend consistency across all the first year biomedical sciences course sites. The Cohort B semi-closed card sort and usability test allowed us to refine that template and make numerous modifications. This second step was essential to ensure that the course sites reflect the integrated structure of the new curriculum as much as possible and to provide an information architecture that is consistent with the expectations of the primary site users.

While the most recent cohort of students in the program, the cohort following Cohort B, appear to be experiencing minimal difficulties interacting with the modified course sites, it is important that we continue to monitor the performance of the sites for incoming students.


Previous | Next