The Importance of Testing the User Experience of SaMD

Beyond Compliance: Testing SaMD Through the Patient’s Eyes

As a tester, I find myself exploratory testing every piece of software upon which I place my hands. I can’t help it; it is innately part of my mindset. Also, as a tester, I have championed the customer in every one of my test projects. Therefore, it would come as no surprise that much of my personal exploratory testing is done with a heavy customer experience focus. Some of my recent and most interesting exploratory tests were on Software as a Medical Device, (SaMD) products.

Having tested, managed and consulted on the verification and validation of SaMD, I understand and have managed the complexities of testing FDA-regulated software. As I informally exploratory testing these products, I’m learning how very important testing the client interaction truly is.

Although most all medical devices require some level of interaction with the customer, SaMD requires a different and broader type of interaction; the client is interacting with both the software aspects as well as the medical aspects of the product.

Although the FDA definition of software as a medical device is “Software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.”, it is often delivered in the same ways as most software products.

When a client “meets” most SaMD products, they interact with the software in all of the usual ways that they interact with any software package. This includes the mode of delivery, usually cloud based, the device(s) upon which the software is used, perhaps a tablet, a smart phone or even a laptop. In addition, there may be devices that attach to the body or scanners to monitor body movement. Finally, there are usually some means of communication established for the client to interact with the supervising medical professional, perhaps by text or video chat. All of these must be kept in full working order, fully charged with ongoing software updates so that the clients benefit from the SaMD’s intended use and features.

It is imperative that these be thoroughly tested from the perspective of the clients who will be using the SaMD as they impact not only the user experience; but more importantly, the delivery of the medical benefit.

For example, I as I interacted with a SaMD whose intended use and features were to provide physical therapy, I found that the sensors attached to my body sometimes fail to register how I am executing the exercises. Sometimes, they fail to register movement at all. Other times, it will register movement or alignment incorrectly.

After many sessions (tests) I find that when this usually happens it is because the sensors are not fully charged. However, other times, it happens after the software receives updates.

As a client, I find it exceedingly frustrating when this happens. Sometimes, I’m unable to complete the exercise program and I don’t get the full benefit. Furthermore, if I get too discouraged with the failures, I may discontinue my compliance with the program.

Since the behaviors of the hardware and mode of delivery of the SaMD can impact how the user benefits from the SaMD’s medical intended use and features, it is critical to test the user experience.

About the Author

Gerie Owen is a Quality Engineering Manager, QE and Agile Process Expert, and an accomplished author and international conference presenter. With extensive expertise leading test teams across healthcare, financial services, transportation, utilities, and insurance, Gerie specializes in analyzing test processes, coordinating global teams across time zones, and delivering complex, high-quality projects on time and within budget.

She is passionate about mentoring Quality Engineers, coaching them in both technical testing and career advancement. A frequent keynote speaker and award-winning author—including winning Best Paper at SEETEST—Gerie has presented at major conferences worldwide (EuroSTAR, Agile Testing Days, STAREAST, STARWEST, PNSQC) and numerous DevOpsDays events. Her presentations and articles draw directly from her real-world technical experiences and process-improvement insights.

Connect with Gerie via email, X (Twitter), or LinkedIn.