Some medical device product development teams skip usability testing when developing new products. Reasons include, ‘it takes too long’ or ‘we already know how the product is going to be used’ or ‘we’ll test usability during design validation’. User feedback is frequently collected by showing mock-ups or through structured data collection, as opposed to usability testing that observes how the user interacts with the product. Early and frequent usability testing is helpful since you often finds issues that can be addressed during development. In February 2016, the FDA released a guidance on usability, Applying Human Factors and Usability Engineering to Medical Devices. From the FDA’s perspective, usability should be closely tied to risk management activities per ISO 14971 – Application of Risk Management to Medical Devices. When integrated correctly, usability testing helps ensure the product is safe and can be used correctly by the user (FDA expectation) and that the user derives value from the product (business expectation).
You’ve just spent 2-3 years working on a new product, and you launched it on time. You think the product is great. It meets the customer or user requirements, the price point is right, and you are looking for the sales to take off.
You have talented product designers and engineers, and your product managers have a great understanding of the user. You interviewed some users to get their feedback when you started the project and held several focus groups to discuss the concepts during development. During design validation, testing included some users running protocols to show the product met the user requirements.
But sales don’t take off. In fact, you are getting feedback from some customers that the product is not working correctly. When you tested the product in-house, it worked fine. The problems are starting to show up on social media and in product reviews. Instead of it being a star product, now you have a dud that everyone knows about … and you are falling short on sales. So what went wrong?
After the problems surfaced, you asked users to describe what they were doing, what buttons they pressed, and how they used the product. Nothing seemed to provide a clear answer. So you finally brought in a few users to show you how they were using the product. You gave them the product, like they would receive it (including packaging and instructions for use), placed them in a setting similar to the expected use environment, with no additional instructions or protocols. Then you observed the individual using the product without interrupting them.
The first thing you noticed was that no one read the instructions. This product is similar to a competitive product, which the users are familiar with. When the user was asked about the instructions, after the session, they said the instructions were often useless; either they were too hard to understand or they were filled with legal language that didn’t help them use the product. In addition, because the product was similar to a competitor’s product, they didn’t see the need to read new instructions.
The next thing you noticed was that the user seemed to struggle with a specific setting. The design team added this setting to provide the specific feature that makes your product unique in the market. So why aren’t they doing it correctly? During a post-observation interview, you discovered the user didn’t know that the capability existed and it wasn’t obvious how to engage that setting.
Now that you understand why your users had a problem, you have to decide whether to fix it. More cost, time, and resources would need to be devoted to fixing the product, which may have been prevented with usability testing during development. These issues and delays also translate into lost sales.
As a product developer, getting user input at the beginning of a project is the starting point. However, the product developers also need to understand how the user will interact with the product. Testing via protocols may not identify user problems which often surface during usability testing. Collecting feedback from the user during development is important to successful product development. You just need to remember that usability is part of that feedback.