Ix Design | Web Design
Physical Augmented Reality Therapy
Physical Augmented Reality Therapy or P.A.R.T. was a 20 week long design project completed as part of my capstone for my Bachelor of Arts in Interactive Media Design.
P.A.R.T. is an augmented reality assistive application for physical therapy. It aims to increase patient engagement, follow-through, and cost-effectiveness for various physical therapy settings. Through a minimally invasive AR implementation and compatibility with existing Web XR technologies, we provide physical therapists with real-time data and patient metrics, accessible from anywhere with an internet connection.
Lead Interaction Designer
Usability + Desirability Testing
Interactive Figma Prototype
Interactive WebGL Prototype
As P.A.R.T. is both a business to business (B2B) and business to consumer (B2C) product, there were multiple domains to consider. On the patient side (B2C), the domain of our project consisted of elderly individuals, those with cognitive impairments. From our research we found that elderly patients often struggle with remembering prescribed home exercises, leading to missed days and longer treatments plans. Patients with cognitive impairments face these challenges as well, resulting in a demographic of patients for which traditional treatment methods create significant barriers to recovery. On the physical therapist side, we took the time to speak with licensed practicing PT's to discover major pain points. We took the time to look through the most common communication practices between patients and PTs regarding home exercises programs (HEPS), which were either: emails and/or via written on paper. However, a major pain point of this methodology was that PT’s had no idea if their patients were doing the exercises correctly or if they were doing the exercises at all.
The stakeholders involved in our domain included elderly individuals, and those living with cognitive impairments. Also included are physical therapists who note issues with the way patient progress is monitored today—as well as dealing with a lack of resources for struggling patients.
There are many technologies that exist in the PT space, created with the intention to serve PTs. Platforms like WebPT, Medbridge, Hep2G0, quickEMR, and PreHab, all exist and take up considerable market space. However these technologies operate solely on web and mobile platforms—therefore restricting their services to those limited platforms.
We learned through our research that the current workflow was a process they developed through the years and interrupting that could be an issue. This wasn't simply because their workflow was optimized, but because they had to pick and choose different systems. These systems generally consisted of EMR (electronic medical record), billing software, scheduling software, and HEP (home exercise program) libraries. Some, but not all of these, of these were built into proprietary systems, and data visualization was also lacking.
Concerns with Available Technologies
The main concern with existing products is the requirements placed on PT patients, products like Medbridge and WebPT require patients to download proprietary applications in order to view video tutorials and provide guides. These solutions, while beneficial and useful for younger audiences provide significant roadblocks to elderly persons for whom smartphone use is difficult. A further concern with these platforms is that all the data collected is submitted by patient choice often, only binary questions and a pain scale.
Our team immersed ourselves in the domain which our project operated in. Our goal was to learn about a domain we were largely unfamiliar with, identify what is happening in that space, who the key players are, and what the role of technology is. To do this we wanted to gather interesting examples of the types of initiatives and technology related to AR and PT, as well as identifying best-of-breed sites and technology in these areas, with a specific focus on products aimed to increase accessibility. To do this we conducted a brief and expansive study of existing primary and secondary sources to better orient ourselves with the state of the market, existing products, and already stated concerns with existing AR technologies. This in turn bled into our greater secondary research, as though this process we collected numerous academic articles and papers we could later reference.
Following our initial domain immersions, the team looked at over 20 peer-reviewed academic articles and published papers discussing the intersection of augmented reality and physical therapy. By looking at these resources, we were able to ground the subsequent phases of our project on reliable, peer-reviewed research. We were able to draw critical insights from these papers and were able to discover the areas where we could innovate and experiment with the combination of these technologies.
In parallel to our secondary research, as it never really ended- we were continually updating and improving our foundations- the team conducted brief ethnographic studies. We looked at online websites and forums that PTs publicly posted on. We specifically searched for discussions amongst PTs about current solutions, technologies, and best practices. We collected a combination of over 10 threads and discussions- spanning roughly 200 separate posts- from various online communities. We then thematically analyzed these responses to develop themes and insights we could use throughout our following research.
Explorative Usability Study:
To answer our research questions, we first conducted surveys of 22 past PT patients, conducted 6 interviews with past patients, 2 interviews with practicing PT’s, as well as exploring the existing market (see above competitive analysis chart).
We aimed to identify trends and patterns. Through these interviews with physical therapists and online surveys from past and present physical therapy patients we uncovered a number of recurring patterns and a variety of pain points. This ultimately led us to the question, “How can we improve the PT experience for middle and senior aged citizens. The primary concerns for physical therapists were patient follow-through and measuring success. For patients, pain was a factor, but primarily it was a question of motivation.
The high-priority observations based on our research that our team noted were the issues regarding patient adherence. Notably, 50% of survey respondents stated that they either only casually completed their HEPs or did not often complete their HEPs. In our open answer questions and optional follow-up interviews, we were able to dig deeper into the reasons for this adherence drop-off. When asked specifically for the reasons why respondents noted that there were often many competing factors. In a clash of intrinsic and extrinsic pressures, respondents noted factors like lack of improvement, further issues with pain, family and monetary pressures, and the need for self-discipline. Respondents gave the following answers:
“Money and time. Each time I went to a PT appointment there was a time commitment and a copay.”
“I felt like it was just something I was doing to tick a box and say I went, so my workplace would approve my time off.”
“The comments that were said by the therapist about the stretches and not seeing any improvements from the feeling to the arm.”
"I wasn't allowed to take medical leave (unpaid!) if I didn't go."
On the PT side, we found a variety of interesting insights through our interviews. Though we were only able to conduct two interviews (we found out the hard way that PTs are incredibly difficult to get a hold of) we were able to gather many valuable insights and recommendations. We were able to hear from individuals who had personally dealt with many of the common themes we had come across in our earlier research. In a conversation about adherence and HEPs, one PT stated:
“The other half... is getting the patients to perform the exercise correctly. Most studies have found that if you give patients complex things, they just don’t do them. You want to make the exercises as simple as possible.”
When we discussed the potential merging of AR technologies and physical therapy, we were met with much concern about integrations and the cost to operate such technologies. While stating the foreseeable barriers, one PT continued:
“There’s potential for this project and could be valuable for [future AR] integration in therapists’ offices.”
Keeping all of these factors in mind, we began developing an idea of the current problems and potential solutions we could create.
The team knew from the start that this problem needed to be approached from both the physical therapist and the patient sides. For the PT, it had to make sense to include it as part of their treatment plan, and it had to provide real-world data to show patient follow-through and improvement. For the patient, it needed to be easy to use and incorporated into their home exercise program.
Shifting to the next phase in our project, we began by developing a feature list. To do this, we each wrote ten task flows, roughly 50 total, based on the data and artifacts we collected earlier. Once we created these, the team worked together to develop a priority matrix (shown below).
Once we had narrowed down the features that we considered “Need to Have’s” (indicated by the red and orange lines), we set off to develop user flows. We produced bigger picture product flows as well as more individualized feature flows. Shown below are a sample of the flows created by the team.
Once we had a fundamental understanding of how our AR and Web-based programs would function and interact with the humans using them, as well as with each other, we began our journey of prototyping each of the different components. We began this by sketching out the potential layouts and looks of the products. Shown below are our early sketches.
Following the early iterations of sketching, the team split into two smaller specialized groups- one focused on the web B2B product and the other on the AR implementation. We prototyped the web product in Figma. We stepped through low, medium, and high fidelities. Shown below is a cross section from each phase; please reach out if you would like to see more.
Simultaneously, the other half of the team went through a similar process designing and prototyping for the AR B2C product. Designing in Figma in the early stages of the project, they later swapped over to WebXR in order to develop a running prototype. Shown below is a similar cross-section of work from each phase, as well as a gif of the AR interaction.
As a part of the capstone course and project, our team was in charge of creating a more in-depth, Behind the Design Video, as well as a concept video demonstrating the use cases of the product. Check out both below.