July 22, 2011

Webinar Recording: FDA Design Traceability Requirements for Device Development

Thanks to everyone who joined us for the FDA Design Traceability Requirements for Device Development webinar with John Avellanet, Managing Director & Principal at Cerulean Associates, LLC and Larry Nicholson, Business Development Manager for Life Sciences at Seapine Software. If you missed the event, or want to watch it again, the recording can be found below or viewed on SlideShare. Additionally, you can download the FDA Expectations for Traceability in Device Design slide deck. http://youtu.be/iB8JDuHTdIk Q&AWhere can I find the FDA draft document for medical applications? All of the recent medical device guidance documents can be found on the FDA site: www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm162707.htm. (Avellanet) Is an application that sets up datasets for infusion pumps considered a medical application? Take a look at the guidance, specifically at the criteria listed on pages 13-15, and then the examples provided by the agency in Appendix A of the guidance. (Avellanet) Is there anything relevant to web-based medical applications for desktop computers? See answer above, and look to other guidance documents on medical software applications. I suggest comparing what the web-based application does versus what issues are raised by FDA in the guidance documents that cause an application to “cross the line” into medical device technology. If the web-based application meets the criteria laid out by the agency, be very cautious about assuming the guidance doesn't apply to the web-based application. After all, many mobile applications use the web to store data. (Avellanet) How should we handle traceability if part of our product is developed by external parties? The agency will expect you to treat the external party (assuming the external party has a different quality system than yours) as a supplier. You will then want to tailor your traceability activities and records accordingly. (Avellanet) How do you recommend combining FDA traceability with Agile software development? This is a great question that unfortunately can get quite complicated as it incorporates a lot of permutations. Look specifically at chapters 5 and 6 of the book. If you look at the lifecycle diagram on slides 18 and 34, you’ll see an idea of how this will work as those diagrams are from the book. (Avellanet) It can be complicated as John mentions, but there are tools, like TestTrack, that can help support faster development methodologies, including Agile. It really comes down to vernacular (naming conventions) and data structure of users stories, ensuring you can still meet compliance objectives for verification and validation activities. Seapine’s approach allows us to be fairly flexible with development methodologies by focusing on the task while ensuring compliance activities and testing are also included through enforceable workflow and automation. The compliance piece can be transparent to daily tasks, because our tools track and document everything along the way. It’s more like a test-driven process, to verify each testable user story can be validated. The FDA does not state that you have to adopt a specific methodology, just verify and validate with objective evidence. You do not have to write all your requirements or user stories at once. You can start coding or creating specific hardware prototypes, which does allow for faster VOC feedback in clinical phases. The obvious caveat is the type of device or software you are testing or verifying with your customers/testers. John makes a great point about this in his book, which basically says design with quality in mind. If you decide to adopt Agile or elements of it, you’ll want to ensure you are approaching it with the end in mind for both for testing and what the FDA expects. It all comes down to quality. (Nicholson) How does traceability affect my suppliers? It is entirely dependent on what your suppliers do for you and the contractual arrangements you have with them. Two simple examples: If it’s just a packaging supplier who provides the cardboard shipping carton for the device (but not the actual box with the labels printed on it, etc.), then traceability doesn't impact them whatsoever as the onus is on you to ensure the carton can protect your device during shipment. If the supplier is a contract manufacturer who is producing your prototypes, you need to know if they are making changes to the production process or the device. If they make changes, then someone needs to be tracking it – either you or the supplier – and that should be defined and documented in your contractual agreement. (Avellanet) I agree with John, but in the context of suppliers or contractors who may help with development or testing, you’ll want a system that really limit them and track everything they do. We give our clients the flexibility and control to explicitly control what suppliers can actually see and do with the system, as they may not need full feature capabilities, read-only rights, hide fields, notifications, etc. For example, you may have a supplier that should be notified if a specific change happens, which you could control with a tool like TestTrack. Its security group capabilities give our clients the ability to document the permissions and explicitly control them based on what they decide. Most companies want this robust capability, because there are concerns of intellectual capital loss in addition to managing change. (Nicholson) Can you provide any forms or templates for each of the eight basic design control steps? All the various steps come with forms and templates refined and tailored over the years to specific clients. A generic form of the product concept sheet is available to readers of Get to Market Now! Turn FDA Compliance into a Competitive Edge. (Avellanet) What advice would you give to an engineering undergrad that has several ideas for medical devices? Focus on whether or not you want to be an entrepreneur. If you want to be a medical device entrepreneur, then try to find a mentor who has done this before and can help you avoid the silly mistakes that we've all made (you can see a tragic set of examples in this case study from my own personal history on my web site: Want a Good Outsourcing Partnership? Know How to Ruin One First). If you don’t want to be an entrepreneur, think about what you will do with your device design – sell it, partner with someone, or? Then take that answer and talk to one of your professors for advice on who he/she would recommend you talk to that has gone that route before. (Avellanet) You said to start traceability at project go/no go. Why not at sign off of the design input? Well, technically, the design control regulations start “Once it is decided that a design will be developed” (i.e., you've made a conscious decision based on concept and feasibility studies). So, in 21 CFR 820.30(b), planning for device design and development (including traceability) fall under design control. Design input approval, 820.30(c), comes after the planning, so by the time you’re at the sign-off stage, you are already under design control and should have started your traceability documentation. This is why I strongly suggest using a more formal “go/no-go” decision as you wind down the feasibility stage (i.e., preclinical), and thus I suggest using that product concept document. I also wrote a very well-received article on this–admittedly for biotechnology firms–entitled Speeding Time to Market with a Preclinical Stage Gate. Take a look at how and why I suggest structuring traceability to start at the “go/no-go” decision point. You may also want to look at the chapter in Get to Market Now! Turn FDA Compliance into a Competitive Edge that discusses improving innovation, the voice of the customer, and the product concept sheet. (Avellanet) I agree with John, start it as soon as you start design. It’s easier than you think, if you have the right tools for design traceability. (Nicholson) I get that the cost of non-compliance is huge, but what should the cost of compliance be as a percentage of revenues? I have done several speeches on this exact topic. Two of the most recent public ones were Are FDA’s Quality System Requirements Bankrupting Your Company and FDA Compliance on a Budget: Avoiding FDA Trouble While Surviving Budget Cuts. You can get recorded copies on the FOI Services web site, or simply contact me to see if this would be something that might be of interest to your organization as a private webinar or onsite workshop. (Avellanet) How deep into testing should the trace matrix cover? As I noted in the webinar, this is going to depend entirely on the risks associated with your device, your technology, and your production (including even distribution and suppliers). I would expect a lot of detail and depth for a complex device involving both hardware and software. On the other hand, I would expect a low level of detail and depth for a simple device such as in the Class I category. (Avellanet) I agree with John, but I have seen a trend with medical device companies moving up the classification scale. If you know you’re going to be a Class II device, based on future capabilities, three or four years down the road, start incorporating good traceability early. Instituting good traceability and testing early doesn’t have to be painful, plus it gives you better metrics and visibility. If you’re testing, but worried about documenting the testing efforts, then you likely need a tool that documents behind the scenes. (Nicholson) I am a Quality Engineer for a major medical manufacturer, and traceability resides with R&D. In your experience, who owns traceability and which group or department is typically most effective at maintaining traceability? Frankly, it’s all over the map – there is no one right breakdown. I will tell you what I do with my clients to help them think through this: Which group is going to be accountable to the FDA inspector for traceability versus which group is going to be accountable to senior management (given that they are the liable ones under both regulations and product liability statutes) versus which group is going to audit it to make sure the other groups are doing what they should? Clearly the agency is going to be skeptical if the same folks responsible for implementing and maintaining traceability are the same folks accountable for auditing their own work. Another really good approach is to ask your legal department who they’d like to see in charge of traceability. I've seen situations where everyone agreed that because engineering was closest, they were the best choice to implement and maintain it up until legal weighed in and noted that, when it came to defending the company in court or to regulators, the last thing they wanted to do was put engineering up on the stand–and thus it ended up in the hands of the quality department. It really is dependent on the dynamics of your organization and how you've set up not only traceability, but also the auditing of the traceability function and records. (Avellanet) Great question and John is right, but I also think that everybody in the organization needs to take ownership, especially if they touch an item being traced. If you have integrated system, it’s not that painful. Our paper on Six Exercises to Strengthen Traceability has been popular with clients who want to start a discussion in their company about traceability ownership and best practices. (Nicholson) What trace matrix formats does the FDA typically like to see? There is no defined answer as long as you can provide accurate and complete records. That’s why I noted that almost all of us (myself included) started out simply with a spreadsheet or even a notebook, but it can quickly get out of control without an automated system. (Avellanet) Agree, but an integrated automated system lets you have the option of different formats. Our tools include an Impact Analysis report, which is much easier to read than a traditional trace matrix in a table format. (Nicholson) How detailed does a risk analyses have to be? See the March 2011 draft GHTF guidance document Essential Principles of Safety and Performance of Medical Devices, and the July 2005 guidance Implementation of Risk Management Principles and Activities Within a Quality Management System. (Avellanet) Where do clinical trials and all the associated documentation fit into the overall device design control scheme? They are part of the overall design of the device. This documentation can be referenced in your traceability matrix depending on how you planned out how to test the device design. Note that I am not suggesting you input your clinical documentation into your traceability matrix, only consider linking to it or referencing it. (Avellanet) How many times should FDA inspectors have to ask about traceability during inspection before they expect an answer? Once. (Avellanet) Agree. The inspection process is really about instilling confidence with the inspector that you have good controls. If you can’t answer their questions promptly, that’s a red flag. (Nicholson) The reality is that I do not have perfect traceability at this point in my R&D. Do I go back and improve the past or start today and prepare excuses for the auditor? Here’s what I’ve done when I’ve arrived on the scene of a company in your shoes, whether I went to work directly for them or I was called in as an outside expert either as a consultant or as an independent observer in a consent decree type of situation. Step one is to start today with the “good” documentation. In other words, start to do the right thing going forward. Then, as close as possible to starting that step one, undertake step two, which is a gap analysis of what’s missing (where specifically did we go wrong and how). Step three is then to analyze that “how” against what you started in step one (am I doing all the right things, do I need to tweak something, etc.). And then step four is, essentially, retrospective traceability (i.e., going back and trying to reconstruct as much as possible). Here’s why this approach works – it provides crystal clear documentation that says to any outside observer (say an inspector or a product liability court), “Look, as soon as I found out that I needed to do better, here are the steps that I not only took to do better, but here’s where I went back to try to fill in all these gaps that I uncovered with my new knowledge.” (Avellanet) Start today and build an automated process going forward. If you adopt a tool that minimizes the duplication and manual effort typically required with disparate systems, you’ll be able to quickly identify gap analysis and establish traceability with links to imported data. (Nicholson) By putting issues into the traceability matrix, is there a risk that you’re more prominently highlighting defects or bugs with the system to auditors? If we assume that you are going to fix the issues or come up with workarounds (including design changes to avoid an issue in the first place), then all it does is provide the documentation that you took seriously your responsibility to design a safe, effective, quality medical product and the regulator expectation of “continuous improvement.” (Avellanet) No, I’ve never seen any development cycle that didn't have issues. Auditors are aware of issues or tasks as a result of a failed test case. I would think that a failed test case without an issue would cause more red flags with an auditor. You would obviously want to close (verify) the loop on any issue that is in an open state, so good analysis and verification needs to be conducted. An integrated system will ensure these mistakes are minimized with good automation rules. (Nicholson) You've mentioned Agile might be a nightmare for traceability analysis. Can you please elaborate? I mentioned that in the context of if you are trying to maintain traceability manually and depending on how agile you want to be. I’ve seen firms actually entertain the notion of hiring someone full-time whose sole job is to update a traceability matrix! While I think that’s great from an “employ people in your community” standpoint, I’m not sure it’s the best business decision particularly if that product might be canceled and the new hire is just as quickly out of a job. Thus, if you are going to take the agile, rapid-prototyping route (which, personally, I love), then you want to think hard about automating as much of the traceability analysis as possible. (Avellanet) I believe it could be a nightmare, if you don’t have a flexible system that tracks all these changes and still manages all the linked relationships when moving items around. Without naming names, too many of the Agile-type tools are not doing a great job at tracking and maintaining these relationships, not to mention good validation reports. I believe our configurable and flexible tools make Agile more realistic for high-risk projects. (Nicholson) So a project requires traceability after the concept has been approved and a budget is set? Or after a first prototype has been made? I guess our hardest decision is when is a project subject to traceability...when is it a “go” in other words? I think in 16 years, I’ve never seen this be an easy decision. As I noted above in my answer to a similar question, the FDA expects you to start design control–and thus traceability–as soon as you are done with feasibility studies. Unfortunately, I can’t recall anyone I've ever worked with stating, “Okay, as of this coming Friday, we completely done with feasibility, so that means on Monday, people, we are all under design control.” It doesn’t work that way in the real world. The goal is to have about 80% of the feasibility information you need in order to make a decision. I find that the product concept document (especially when combined with a preclinical stage gate review) gets everyone focused: “Look, we have enough of the feasibility data now to decide are we going to develop this as a medical device with an intent to commercialize it – or not? Yes or no.” If it’s “Yes,” you’ll need to answer questions such as what are the resources, the timelines, who’s going to be involved in planning the development (such as clinical trials and nonclinical studies), who is going to be the regulatory affairs point person, (i.e., we are now under design control and 21 CFR 820.30)? If the answer is “No”, then you have another decision to make: Are we going to cut and run on this effort, or does it need to go back into the feasibility hopper until we are ready to say “Yes?” I don’t mean to sound so cut-and-dry, but there comes a point where one either fishes or cuts bait. As one of my old colleagues used to say, “It’s hard to be a little bit pregnant.” (Avellanet) Agree with John, but why wouldn’t you start traceability early? There are several benefits besides compliance, like visibility, better communication, and historical data. (Nicholson)