Articles on Law, Science, and Engineering

Problems with User Facility Reporting[index]

Given that health care professionals have not previously been expected to file product-related reports, device user facilities are the weak link in the existing reporting system. They have the best information on product problems, yet they have been asked to report only incidents involving serious injuries. This is contrary to good risk management strategy, and leads to a gross underreporting of both injuries and problems. It also limits the ability of manufacturers to do effective reporting because they must depend on the user facilities for much of the primary data on product problems and injuries. Good risk management, which is really just another name for quality control, is based on identifying potential problems before they result in patient injuries. To use a traditional statistical quality control example, assume that you run a production line that produces bores cylinder holes in engine blocks. If your inspector checks the position and size of the holes in the finished block, you will probably have to throw away at least one block whenever you detect a problem. If the production line is fairly long, the pipeline effect, you may have 100 defective blocks before the first one reaches your inspector and is identified.

In contrast, if your quality control inspector inspects the machine that bores the blocks, i.e., if the inspection is upstream of finished product, it will be possible to detect drifts in machine settings before they result in an out of tolerance block. This will require more intensive monitoring because each machine must be sampled on a regular basis, but it will prevent costly runs of defective products. It will also improve overall quality because it provides more points at which to detect defects, making it less likely that an improperly bored block will be sent to the engine assembly area.

If quality control in our car factory were run on the same principles that many user facilities use for reporting medical device problems, we would manufacture the components and assemble cars with no inspections, then take them on a high speed test drive to check for problems!

Patient injures and deaths are the worst possible indicis for reporting. Most fundamentally, the damage is done and there is often little that can be done to mitigate the patient's injuries or the corresponding legal liability. It also leads to gross underreporting of problems for two reasons: 1) once there has been an injury, reporting a problem is often tantamount to assigning blame to one's self, one's coworkers, or one's institution; and 2) the background level of morbidity and mortality tend to mask all but the most obvious device and drug problems.

From a statistical perspective, injury driven reporting is ineffective because it produces samples that are too small for proper analysis. This is especially important for drug-related adverse events, where the patient's injury often mimics other medical conditions. This can allow a major, but rare, risk to be ignored for years. (It can also have the opposite effect when a few well-publicized anecdotal reports of injuries ruin the reputation of a product.) This has been considered less of a problem in medical device reporting because the FDA has concentrated on obvious device failure modes such as electrical shocks. These can be easily attributed to a device without detailed statistical analysis.

Modern medical devices have become sufficiently reliable that obvious device injuries such as electrocution are very rare. The recent identification of latex sensitivity in patients given barium enemas illustrates the problem in identifying subtle device problems. In this case, the first reports of injuries focused on the barium because the injury was a classic drug reaction. It was only after many reports were studied that the association with the latex catheter was discovered. In addition, I believe that the current reporting system misses the most important cause of medical device-related problems, human factors engineering.

As is well known to clinical engineers in user facilities, perhaps 50% of device problems are the result of poor human factors engineering. I will argue that the problem is so widespread that we often do not recognize it as a problem. It is taken for granted that medical personnel will require some training or specific orientation to every different brand (or even different model) of a given type of medical equipment. (This does not mean that you can afford to do the training, just that you know it would better to do it.) While the problems of training become overwhelming in the typical facility with a hodgepodge of equipment and constant personnel turnover, we do not see this for what it really is: as a massive failure of human factors engineering. As a result, when an injury does occur, it is often attributed to operator error rather than to poor equipment design. A reporting system geared to identify every occasion when a machine was improperly adjusted or setup would quickly identify human factors problems, well before they resulted in patient injuries.

Next - "Voluntary" Mandatory Reporting

Previous - The MedWatch Program