xAPI-enabled Mobile Health System with Context-Awareness & Recommendation Engine for Patients


Mobile health applications (Apps) have flourished in recent years; many hospitals also use mobile portals to engage and serve patients. This case study from iCAN Lab and Classroom Aid addresses the issues for spinal cord injury patients with the mHealth solution. xAPI enables integrating data from disparate sources efficiently, machine learning enables the App to build context-awareness and modeling, predict behaviors and serve patients dynamically -- like an intelligent assistant. The same system architecture can be used for training and learning project as well.  


Current sensors (including wearables), medical devices and mobile technology enable us to collect numerous data around humans and environment, but they are heterogeneous and unstructured. Also, there is no standard, or an interoperable schema for documenting human health in a digital format. Context information is especially crucial for medical data, and it can be recorded in different formats or even totally missed. We can't make use of all heterogeneous data until time and computing power are committed to integrating and interpreting them.  In this case, we need to let the App called Smartchair respond adaptively to patients in real time to help them reduce chronic injuries, to assist them and to guide them dynamically.


Concerning the Smartchair user's needs, frequent clicks result in chronic injuries. The solution is to build context-awareness from user's history records, from the prescription and from other context data to prompt the recommended action for user dynamically. This reduces clicks and provides guidance at the same time.

Different services needed to communicate amongst each other – for example, the site for therapists and the wheelchair vendor's system need to communicate. To solve this, xAPI is leveraged for data transfer and integration.

System developers need a more efficient way to collect user behaviors to obtain feedback and to improve the system. After revision, users also need to relearn a newly-designed interface. The solution is a 'recommendation engine' to prompt recommended actions for supporting users as well as collecting feedback. Developers will update the system design only after the hit rate of recommendation is higher than a threshold.

At the center is Smartchart App, data from the motor power wheelchair and the prescription from the physician or therapist which integrate with data collected by Smartchair. xAPI is used to normalize the language describing patient's activities, with necessary context data in each statement. User interfaces are segmented and named appropriately so that machine and humans understand them. xAPI collects data from functionality, interfaces this data into building behavior models, which will be sent to context-awareness models for calculation, and then combine this with expert knowledge.   This data will be processed by filter model to sort out suggested next steps and offer this information to the patient. The recommendation engine prompts recommended actions upfront (after user login) in calculated sequence, on top of the current hierarchy. The cost-benefit analysis is to analyze if this recommendation engine really improves user experience and whether it solves the problems identified (see image).

The strategy here is the collaboration between machine and experts (therapist or teacher). First, the machine builds recommendations from the expert's prescription and user's behavior history to balance both dynamically; the output of machine learning is under monitoring of the expert.  Then, the expert can modify the recommendation if needed. The machine will always adjust its modeling to fit to that label. That means, the machine learns from its human collaborator and real data continuously.


Many medical devices and sensors can collect data around a patient, including physiological sensors, biokinetic sensors, ambient sensors — even swallowing a pill can provide pictures inside the body. With data so available, there are two things we care the most about here:

  • For medical data, the related context of a data point is very important. xAPI can record contexts in a standard way. This means that experience tracking is not only described as a number but now we can record human behaviors in a more human way.
  • When data is available in different formats, we can NOT make sense out of it or use it UNTIL the time and computing power are committed to integrate and interpret this data. xAPI data, by contrast, is highly structured in an intentional, pre-designed way. Thus, it can be integrated meaningfully as soon as it is collected. The data can be used right away for humans to read (dataviz as cognitive agent), and for machines to compute and respond (less guesswork than processing unstructured data). Services can talk to each other and work together in real time to serve humanity ASAP.

The benefit for experts (instructors, therapists) to use this solution is that integrated data can give them a better picture about their students or patients in real time; they can take actions at the moment of need.  Also, the machine's recommendation can be very helpful and the ability to share the experts' work loading and data mining findings from accumulated data can be very valuable in the future design of training or prescriptions.