Determining the number of participants for device usability trials

Usability trials are a critical step in the development and evaluation of medical devices with electronics and software. To gain regulatory compliance, medical devices need to comply with the IEC 62366 standard on Usability Engineering which sets out a systematic process to analyze and evaluate medical device usability to minimize risks associated with normal use.

A key part of this process are two sets of usability trials - formative and summative. The purpose of formative trials is to identify usability issues and to inform mitigations. The purpose of a summative trial is to evaluate the effectiveness of the mitigations and thus the safety of a device from a usability perspective. A client recently asked us how many participants are needed for usability trials compared to clinical trials? We thought we would share our answer.

Where clinical trials for medical devices are focused on proving device safety and efficacy in with large patient cohorts, usability trials needs a substantially smaller number of participants to identify and mitigate usability issues.

The formative trial largely involves site visits, task analysis and interviews with the aim of identifying potential usability issues, with the purpose of informing the design of the device and what factors need to be evaluated and tested in the summative trial. Because the device is typically in the early stages of development, a prototype or even a paper/model simulation of the device is sufficient. It is possible that the formative trial participants don't actually have to be real users in a real environment, although that is obviously desirable. Because of the exploratory nature of this process formative trials only need a small number of participants to identify the range of usability issues. Typically five to ten participants is sufficient.

Determining the number of participants for a summative trial is more complicated. Summative trials need to be conducted with the real product, real "users" and a real environment of use, although the actual trial can be simulated rather than on a real patient. The trial needs to be designed to ensure all hazards and related tasks are evaluated in all use cases, including training. The only guidance a user should have are the Instructions for Use. If the product developer has to manually assist users during the trial to avoid use errors then that is a fail.

The IEC 62366 standard provides guidelines for determining the appropriate number of participants in Annex K. This table from the standard provides a sense of the reasoning process.

Essentially, the number of participants has to be large enough that there is an acceptable probability of detecting and observing use errors.

While there is no absolute number required for summative trials, there is a "magic number" used as a rule of thumb, to guide early planning. The general recommendation is a minimum of 15 participants per distinct user group, which is sufficient to identify an average of 97% of usability problems. This article by Seda Japp explains the reasoning for this number.

The actual number needed will depend very much on the device's complexity and user diversity. For example, in usability trials, participants are not always patients. Usability testing can involve a range of users, such as healthcare professionals, physicians, technicians, or patients. For home-use devices like glucometers or blood pressure measurement devices, the sample size may need to increase to represent diverse user groups, considering factors like age, education, language group, and ethnicity.

Whatever participant numbers you settle on in your trials has to be justified. One client of ours with a surgical device succeeded in gaining regulatory approval from the FDA with as few as eight participants for a surgical device. At the end of the day you need to be able to justify and defend the data you provide as part of your Usability File you provide with your regulatory submission. Advice from experts in the application of IEC 62366 to medical device development will help provide the confidence you need. For more information, contact us.