upa - home page JUS - Journal of usability studies
An international peer-reviewed journal

A System in the Wild: Deploying a Two Player Arm Rehabilitation System for Children With Cerebral Palsy in a School Environment

Raymond Holt, Andrew Weightman, Justin Gallagher, Nick Preston, Martin Levesley, Mark Mon-Williams, and Bipinchandra Bhakta

Journal of Usability Studies, Volume 8, Issue 4, August 2013, pp. 111 - 126

Article Contents


Discussion

The system deployment was successful insofar as it was usable by school staff and children without direct intervention from the research team. The early problems with setup and initialization may have impacted the amount of use, but the system was still used. When these problems were resolved in later deployments, there was not a marked increase in the amount of usage. The major barrier to use of the system was not its technical complexity, but finding sufficient time in the school day for children to use it. None of the children came close to the target level of usage, and the tendency to use registration periods inevitably limited the amount of therapy that could be delivered in one session. Those children making greatest use of the system were those where dedicated lesson time or the lunchtime of the school day was given over to use of the system. The lack of system use was most pronounced around the time leading up to exams, which is not unreasonable on the part of the schools. It is worth noting that the therapy here is intended to replace or supplement a home exercise plan, rather than replace existing time with a physiotherapist and is therefore moving therapy that would otherwise have taken place at home (if at all) into the school environment.

Despite this limitation, the two-player version of the system was used slightly more frequently than the single-player version, albeit for slightly shorter sessions than in single player. Some of the reduction in therapeutic exercise time was caused by the need to set up a second player and carry out the assessment task to determine the level of assistance twice. The system may have value in schools as an adjunct to a home exercise plan—though whether this would be worth the cost (especially if only one or two children in the school were to use it for therapeutic purposes) is another matter. Our future research will focus on home deployments of the system.

Working with schools presents a range of significant practical challenges, not least the fact that rehabilitation research will inevitably be secondary to children’s education. Several issues were raised in this deployment that are worth bearing in mind for future research, particularly where a system is to be deployed unsupervised.

Firstly, schools demonstrated a marked reluctance to contact the research team and report problems. They only mentioned these problems at the end of the deployment, despite the fact that the problems interrupted use. While this was consistent with our aim to leave the system without direct intervention, it did mean that a problem that could have been corrected in the field went unaddressed for several weeks. In this case, the impact does not appear to have been serious, but it does demonstrate the value of maintaining an error log on the system to keep track of problems for review post-deployment, rather than simply relying on self-report from users in the field. In future deployments, we suggest that it would be worth having the system relay its usage and error logs back to the research team via wireless Internet on a regular basis so that problems could be identified and corrected quickly. Better that than finding that a four-week deployment had been wasted because of an unreported technical problem.

Secondly, obtaining feedback was particularly challenging. While staff were generally enthusiastic and happy to cooperate with the project, any involvement was inevitably over and above their already demanding job. As schools are busy, the systems often had to be delivered and collected either before children arrived in the morning or after they had left in the evening. In many cases, the participating staff were not present when the system was collected at the end of the deployment. This schedule meant that the only feedback available were the completed questionnaires, which often were not completed when we collected the system. It is important to note, that completing a questionnaire after a significant amount of time has passed may affect the accuracy of one’s recall. With hindsight, it would have been better to deliver the questionnaires upon collecting the system to ensure that they were labeled or coded correctly so responses could be tied to a particular school even if they were not fully completed.

Finally, having multiple teachers involved in using the same system (or using the space in which the system was located) did sometimes present a problem in terms of delivery and collection. We only gave system and study instructions to the selected supervising staff member; not all staff members that would be using the system were given instructions. Because the system was used by multiple staff members at different times (and/or the system was housed in an area used by multiple classes), this sometimes led to confusion when the system was being delivered or collected, i.e., some staff members did not have the benefit of knowing what we needed from them because they were not debriefed by us or the supervising staff member.

 

Previous | Next