Name: From 20 Weeks to 4: AI-Powered Audit Prep for Life Sciences
Uploaded: 2026-01-13T17:11:04.564Z
Duration: 53 min 8 s
Description: From 20 Weeks to 4: AI-Powered Audit Prep for Life Sciences

Transcript for "From 20 Weeks to 4: AI-Powered Audit Prep for Life Sciences": Hello, everyone. Thank you so much for joining us. We're gonna wait for a couple of folks to trickle in before we officially kick off our program today. In the meantime, if you'd like to introduce yourself, please feel free. The chat function is on the top right of your screen. Feel free to introduce yourself and let us know where you're joining us from today. K. Looks like we've got some people good amount of people jumping in. We've got a lot of content today, so I'll go ahead and get started. All right. Thanks so much for joining everyone. We're really excited for the webinar today, and we've got a great program for you. Before we get started, just for a few reminders to make the most of your webinar experience today. Again, reminder to please use the chat panel on the right hand side to ask any questions. Questions will be received and answered live or by email if we run out of time. You will also get a link to a recording after the session. And if you want to see the most content, please remember to maximize to full screen. As you hover over the bottom right hand side of your screen, you'll see a square icon, and that's how you can expand to full screen. We've got a great panel for you today, and I'm really excited to introduce them. Today we have for you Sanjeev Srivastava. Doctor Sanjeev is an accomplished physician with over two decades of experience in health care. He holds an MBA an an MD in anesthesia and critical care, an MBA in health care administration, and a second master's in artificial intelligence and machine learning in health care. He is also a certified project management professional and a member of the faculty of pharmaceutical medicine. As a signal management lead at BioNTech, Sanjeev has led roles including vaccine safety lead and a medical evaluation lead for two COVID vaccines. He has consulted for top pharma companies and health authorities on signal management processes and advanced data analytics using AI and machine learning. Sanjeev lives in London with his family and enjoys playing the saxophone. So we're very happy to have him here today. We're also joined by Michael Close. Michael Close is the CEO of Logix Pharma, a pharmaceutical and medical device company headquartered in Berkshire in The United Kingdom. Michael has extensive experience in pharmaceuticals, medical devices, oncology, sales operations, and hematology. And we also have Albert Rodriguez joining us today. He's a former FDA employee with over fifteen years of experience in medical device regulatory affairs. In addition to his role as a pre market reviewer for cardiovascular devices, he served in the compliance of FDA regulations specializing in bio research monitoring. He has held various roles at the agency, including team lead, assistant director, and division deputy director. Prior to his FDA career, he also served as a study coordinator and an engineer with NASA. So welcome, everyone. So to queue up our discussion today, I wanted to give a quick overview of the compliance trends, challenges, and opportunities we have in front of us as an industry. So we all know that traditional methods of managing compliance have their limitations. Right? Warning letters are on the increase, and go to market timelines are becoming extended, especially for pioneering medical devices. And at the same time, there's huge opportunity to make improvements with AI. Industry analysts estimate that 85% of pharma workflows can be enhanced with AI, and early results of pilots in the regulatory space show two x improvements in quality and 50% cost efficiency improvements, which are impressive. And we're seeing these results echoed in other regulated industries. There is a recent, report in the data security space that showed a 68% reduction in compliance workload and a recent report from IBM showing that organizations using AI and automation see eighty days saved and almost $2,000,000 in cost savings in responding to data breaches. And the FDA is also starting to use AI in its work. And this shows that AI adoption is not just a nice to have, but actually an urgent industry imperative today. That said, many organizations are taking a cautious approach when it comes to adopting AI. And research shows when it comes to AI adoption, the risk really comes from deploying AI without proper governance. Only 55% of life sciences companies have policies and SOPs on AI and fewer still have AI audits and cross functional safety teams. IBM's report shows that 97% of companies that reported on AI related security incidents had improper AI access controls, and 63% of organizations lacked policies to prevent the proliferation of shadow AI outside of proper controls. So it's important that in life sciences and other regulated industries, AI deployment be thoughtful, properly controlled, and deliberate in order to stand the best chance of success. So here at Qualio, we've identified important considerations for life sciences to take in mind when rolling out AI initiatives. So it's important that life sciences AI be purpose built for the unique requirements of our industry. We also believe it's important for AI solutions to have input from independent third parties, especially in the regulatory space where health authority and notified body judgment is critical. And under the hood, it's important that AI be engineered to improve consistency, reliability, and explainability along with data privacy and interoperability with quality and compliance data. So this approach is reflected in how we've built compliance intelligence, which is Qualio solution for compliance monitoring. Based on companies' quality and compliance data, compliance intelligence essentially identifies gaps against regulatory requirements. It helps teams to resolve them, and it monitors risks continuously to help teams launch faster and scale globally. So let me share a real example of how compliance intelligence delivers measurable ROI. Logix Pharma is a growing pharma and medical device company. They're already using Qualio for their core QMS needs, documents, training, events, and they were facing compliance pressure. They had an ISO thirteen four eighty five recertification audit coming up. They were launching a new lab that needed ISO one fifty one eighty nine accreditation. And like many companies, they were doing compliance gap analysis and audit prep manually using spreadsheets. So this meant heavy reliance on expensive consultants as well. And they saw compliance intelligence as an opportunity to improve their audit prep process and get ahead of risk. The results were immediate and significant, and they cut their consultant spend by 60%. They accelerated their audit prep by 80%, and they passed their ISO thirteen forty five audit with praise on their approach from their auditor. And overall, they saw a five x return on investment in just the first two months. That's eliminating manual compliance processes while simultaneously improving audit outcomes. And we're really excited that Michael is actually joining us today along with our other panelists. So with that, let's get into the discussion. I'd like to welcome all of our panelists to the stage. If you will, please come on stage. Hello, Tina. Hello, everyone. Hello? Hello? Hello? Okay. It looks like we have all of our panelists. Really excited to get into the discussion today. So let's start off with talking about this idea of readiness for AI. So we talked about the fact that there's some push and pull here. Right? There's a big imperative to adopt AI, but it's important to have the control and governance in place. And many life sciences companies say that they're not ready for AI. Data isn't clean enough. The team isn't trained. Governance isn't built. But the FDA is already using AI and regulatory work. So how do you know when you're actually ready to deploy AI in a regulated process versus when you're just risk averse? What's the minimum viable threshold? I'd love to hear from all of you. Do we maybe wanna start with Michael? Yeah. Okay. Thank you for that, Corinne. I think from our point of view, we were very skeptical skeptical about, employing, AI as part of our quality management system, a, because we were a little bit sort of scared that we didn't know what was within the, AI tool that Qualio had. But, actually, when we looked at the frameworks within the ISO accreditation that we, are looking at, it was clear that all of the components of the controls that we had were were really clear, and we could actually identify some of the areas where we needed to have a look at, our gap analysis. And it was also clear that in terms of audit prep, we would have taken a long time to get to where, we were comfortable with our compliance before, an upcoming audit. But, I think what Qualio did going through, demonstrating the compliance intelligence tool, it meant that we could see clearly that, we could accelerate our review time, our gap analysis time, and, ensure that we had tasks, and elements that we could project forward. So for me being a lean company and having twenty, thirty years experience in quality management systems, it made it really easy for me to see that AI was actually going to simplify our process and be agnostic to consultants' education and, experience and that, we would have, you know, suitable examples for moving forward. So from my point of view, a demonstration of the compliance tool, was really easy for me to see that the AI had got quite specific control over the regulatory environment and was easy, then to sort of look at adopting. Awesome. Thanks for sharing, Michael. And I think a lot of folks on the call can relate to that skepticism towards AI and really wanting to see the value. Doctor Sanjeev, you've also deployed AI initiatives at BioNTech. Can you share your perspective on this idea of minimum viable threshold for getting ready to roll out an AI initiative in an organization? Yeah. Thank you for that, you know. Based on my experience in using the my AI tools, I would say from a regulatory perspective, readiness is not perfect data or even a fully mature governance. Readiness is readiness is actually clarity on what is the intended use, ownership, and controls. If we can define in a process, where AI will support the staff for a task, identify the source system that feeds it, and agree how we will test the performance before using it in live decisions, we are ready to start. So for me, a minimal viable threshold is a narrow use case, a process owner, a validation plan, and documented acceptance criteria with a simple monitoring plan so that we check the performance in routine use rather than only at the next audit. Then we can call it AI ready. Got it. Having that control around the process is really important. Thank you, doctor Sanjeev. Albert, from your perspective as a reviewer, do you have any point of view on this idea of the minimum viable threshold for companies that are looking to adopt AI in some of their core processes? Yeah. Yeah. I can just give examples with the FDA. You know, as you mentioned, we had started to implore some, AI in in some of our areas. And one of the ways we kinda convinced those others that, you know, it was viable was to have a a human in the loop. And I think that's really important to have that human human accountability. You know? Who's gonna approve the AI? Who reviews the outputs? Who is responsible, you know, if it fails? So I think having that human in the loop will allow you to have that confidence to actually use the AI and, kind of progress in its growth. That's a that's a really great point. Thank you for that, Albert. It's important to have the process owners involved in in the process. On that note, from your years reviewing premarket submissions and auditing companies, what compliance gaps consistently delayed approvals from your perspective that AI could have caught earlier? And where did you see companies waste the most time on issues that were actually fixable? Yeah. Definitely. So interestingly, you know, it's not always those hard science, issues that came up. It it was more of a kinda like the the things that people sometimes forget in terms of, having a a submission that was complete, you know, consistent, and traceable. Sometimes it it was just simple as, like, having the proper information within the document in the right places. So I think, that's really key and something that AI can can really help a lot in terms of making sure that the application has the necessary information in the proper place. And with FDA going more electronic, all submissions now are required to be electronic. Having that information properly is going to be key in how it gets reviewed and then in terms of determining if there's any issues that come up. Sorry. And then the other part about the wasting time, I think a lot of times when there's a particular issue and FDA will reach out to the company, then I think the company spends a lot of time maybe spinning their wheels trying to trace back issues that they had and then kind of providing more of what FDA needs versus kind of just giving the whole kitchen sink and just making the whole review process more cumbersome for everyone. That's a great point. I was really interesting that you say it's not really the hard science that you. see the issue, more around the documentation and traceability. And it sounds like finding that right balance between having, you know, the right level of detail versus having that kitchen sink and way too much is is important for folks to strike. Right? Definitely. Definitely. Going along that note, would love to hear from Michael and doctor Sanjeev as folks who have gone through audits and inspections. When an FDA inspector or a notified body auditor asks about your AI system and the outputs it generates, I'm curious what, in your experience, they actually want to see. What questions have you encountered from regulators, and what answers in your experience really worked? Maybe, doctor Sanjeev, I can start with you. Sure. Thank you. So in in most of the audits and inspections, the questions on AI have become less about what is the model and more about whether the company controls it like any other regulated system that influences decisions. So early on, there was curiosity about technology. A lot of questions regarding that. Now the pattern is very consistent. Auditors start by asking what the tool is used for, what decisions it can influence so they can judge the risk. Then they go straight to evidence asking about how it is validated against the real representative data, and what were the acceptance criteria, and what you did when it failed, especially with AI model. After that, the focus is on traceability. Again, we know what traceability is. The next trend which I've seen is in change control and drift. How you manage updates and how you approve them. Finally, most important thing which Albert has emphasized very clearly, there's an increasing attention on human oversight and training. So and over the last year, I have also seen more direct questions on generative AI use, whether it is being used to draft regulated records, how confidentiality is protected, and how you prevent any text that is not supported by source evidence from entering the quality system, which is possible by using a non domain specific generated AI. So these are the trends that we have seen in coming years. Yeah. It feels like that human oversight and human in the loop has been emphasized a few times as being really important from a regulatory perspective. So thank you for for bringing that through again, doctor Sanjeev. Michael, what's your perspective on from your experience, what the regulators have wanted to see? Yeah. It's a really interesting one. I think from a regulatory inspection point of view, I think there's certainly curiosity about how AI is being used within the tool and, the the change management process. I think from from my point of view, in the last inspection that we had, we were pretty much audit ready from a thirteen four eight five ASO accreditation. However, we'd use the compliance intelligence tool within, within Qualio to just double check some of the areas that we'd had, identified with some of the gaps. And then we shared some of those, gap analysis with the inspector that was was auditing our quality management system. And I think the curiosity and the questions that, came through was was exactly what, doctor Sanjeev was saying, was that, you know, how are you using it? And when you are using it, what is the suggested text and solutions for the gaps that, the system is trying to create? And they were curious to ensure that, the information that was suggested isn't just, coming from a tool, just to tick a box of the compliance, but is it coming from a credible source and and that it actually fits to the, to the controls that you're trying to, to achieve in the system. So, I think, certainly, at this moment in time, curiosity and making sure that you've got a good validation process and credible outcomes are are really important from an inspection point of view. Got it. Thank you for that, Michael. And I think they're great points. I think the other thing that is interesting for our industry, right, most companies are dealing with, overlapping regulatory standards that they're held to. I'm curious from that overlap of frameworks perspective, what have you seen be really helpful with managing those different but overlapping regulatory requirements and demonstrating that you're meeting all of the ones that you need to meet to a to an auditor? Yeah. Sure. I think one of the key things within, the system that we've we've identified is that a number of the controls, that we're needing to review are actually identical, whether you're in the environment of, ISO thirteen four eight five, fifteen one eight nine. And that makes sure that throughout the ISO standards that we are needing to apply consistency right across the business and the the the work that we're doing. So it it it's really, really useful once you've actually got these overlapping controls because the system will overlook, will look at your your documentation and make sure that you're not duplicating but having a misaligned duplication of information between the standards that that you're trying to control within the quality management system. So, I I really, once I've now got more experience with it, I really can see the benefits of lack of duplication, but making sure that it's pertinent to the to the use of the the system that we're trying to create. Awesome. Thank you, Michael. The other thing that comes up in discourse around AI and one of the biggest barriers to AI adoption is this black box concern. So this question is for doctor Sanjeev, our AI expert. How do you make AI transparent enough that a quality leader, a medical reviewer, or an auditor can trust and defend the outputs? Or in other words, what does explainability mean in practice and not just as a technical concept? So even for the for the benefit of our viewers here, I would say, please understand, black box is not that black. It's not that dark black that it makes it very dangerous thing to even get into. It's more of a gray area, and it is all about explainability. Explainability in practice is not a model diagram. It is just the ability to show you're working in plain language. So for a a quality leader or an auditor, that means each output should be tied to the exact source text or record it's relied on, and the system should preserve that evidence snapshot. And I also want predictable behavior, meaning that the same inputs produce the same outputs for a given version. And any changes documented through change control, and that is all you need to bypass that black road. If you can provide this and most of the tools and task are able to provide this if the governance is very clear from the from the beginning. Thank you, doctor Sanjeev. That's that's an interesting turn of phrase. It's a gray box, not a black box. But I think that's a that's a good point that, again, the traceability, having the evidence at hand, these are all really important factors when it comes to successfully proving that your AI is explainable. With that, I have a follow-up for Albert. What do you look for as an auditor when it comes to explainability of outputs? Is it pretty much mesh? What's your perspective on that? Explainability? Yeah. I would say more in terms of kinda understanding the the expectations. One of the things we've seen, just, you know, people talk about AI and kinda like, well, FDA didn't suddenly just become, you know, more stricter about documentation and and traceability, but kinda like as they want more explicit, more structured, kinda like looking at more of the the we call it the, I guess, TPLC approach, total product life cycle. And just in terms of I think you you'd you've heard that FDA's the QMSR rule is gonna align more closely with the ISO thirteen forty five. So I think that's a big, you know, change in terms of how we're gonna be doing business moving forward. So I think that's, just something key for the audience to obviously be aware of as well. Awesome. Thank you, Albert. Follow-up for Michael. What did your audit prep look like before and after compliance intelligence? We've touched on it a little bit, but would love to hear a little bit more about that in your own words. Yeah. I think the the audit prep was can take quite some time just to ensure that, you you've made sure that you've done a full gap analysis of all of the key components of what an order will or an auditor will look at, since the last twelve months, of, you know, work that has been conducted in the area of the medical device, assays that we've been been utilizing. So it's like a full overview of not only the work that's being done, but is it in line with the expectations of the SAP's work instructions and so forth. And, you know, I in the past, we've spent quite a a huge amount of time just ensuring that we've got the documentation up to date. And I think the the, the ability to use the compliance intelligence tool actually removed a lot of that time almost instantaneously, certainly within a couple of hours, if not overnight, in actually performing that independent robust gap analysis on our behalf and then coming up with quite a number of gaps, to really try and focus in on and look at the evidence that would be required to satisfy requirements for the thirteen four eight five, audit. And there was a number of gaps that actually we knew, from the compliance tool that going into the audit, we hadn't actually looked at the the complete level of evidence that was was required for, for the inspection. However, because of the way in which, we had identified probably the most major gaps that we we believed going into the audit that needed to be addressed. We were able to to demonstrate to the auditor that we've done a thorough job, over the year and ensure that we've got the right level of data and and evidence in the SAP updates and training and various other elements, to actually ensure that the the audit process went smoothly. But in doing so, the auditor, for a number of occasions where there was, some some areas of observations that we were going through discussions, I checked with the compliance tool, online as as we're going through the audit just to just to check my understanding of whether the compliance tool had picked it up as well as the auditor. And in every case, the, the compliance tool actually picked up, the evidence that was missing that the auditor was was looking for. So in my experience, I think, you know, the the way in which the the tool is working and the way in which the auditor actually said they were happy that, this type of tool would identify gaps on an ongoing basis, was was very pleasing indeed, really, to, to get a you know, we had four minor very minor nonconformances, which just needed documentation update. So we were really pleased that we could to, to close out, and would have the the compliance intelligence to suggest what changes that we needed to make. So it was great from that point of view. Awesome. And I'm sure it was great to get that feedback from the auditor as well. I believe that you guys were also able to bring more of your audit prep in house as a result. Can you speak to that? Yeah. Absolutely. So when when we started looking at, doing our thirteen four eight five a couple of years ago, to get to we've gone we're now in our third year of, complying with the thirteen four eight five standards. We spent a huge amount of time looking at from from a consultancy external consultancy point of view, not only training us individual individuals within the business about the framework of the thirteen four eight five and the cost management system we needed to to deploy, but the ongoing sort of gap analysis and then, shaping some of the, the work, and outputs that we needed to create, like management review, reports, and, and various other elements. And, as a result of looking at, compliance intelligence, we've actually got no external consultants at all this year, because not only have we got a good quality quality management system that we had from last year, that on looking at the review, we didn't have any gap analysis done externally. And, the work that we produced over the last twelve months has been completely in line with the standards. But the compliance intelligence tool also checks those, those working, documents, to ensure that you're you're actually delivering against the components of the thirteen four eight five controls. So we're really confident that from a quality management system, the AI tool is actually supporting the business moving forwards and and can actually look at documents as an ongoing basis to ensure that you're you're remaining within compliance in a real live live, state. So that's fantastic from that point of view, which we wouldn't necessarily have done before. Awesome. Thank you so much, Michael. Those are great results, and thank you for speaking to them a little bit more. So shifting gears slightly, this is another one for doctor Sanjeev. So we've talked a little bit about the importance of having a validation process in place for AI. When you're validating an AI system for a regulated process, like pharmacovigilance, for instance, I'm curious what you're testing for. How do you validate that the AI won't degrade over time or behave unpredictably as it encounters new data that it wasn't trained on? As as you saw over even Albert said that the system should be more about believable or controlled and something repeatable. So validation strategy differs from place to place depending on the objective. I'm a safety physician and a data analyst. So I validate against specific task and its impact on decisions, not against the general notion of intelligence. It has to be task specific. For example, PV. If the AI supports signal triage or case clustering, I test whether it reliably surfaces known issue, how often it produces false positive, and where it fails in ways that could mislead reviewers. So basically testing for consistency across product areas and data sources. And for long term control, we set monitoring that checks the output stability. And there are any model update or reference library update goes to a change control. And from audit perspective, I would tell you that it is best to create when you have created something like that, you create create evidence packs. And in evidence packs, you create all the documents that are needed for the review. And the review should not be it should be confirmation of what you did, not reconstruction at the time of audit. And that's why any AI tool that can help to organize these evidences would be a great help. Awesome. Thank you, doctor Sanjeev. With that, this one's for Albert and Michael. Can you explain why domain specific AI is important? What breaks, for example, when you try to use ChatGPT, for instance, for compliance tasks or pharmacovigilance or other sort of life sciences, areas? Maybe, Albert, we can start with you. Sure. Sure. Yeah. Definitely. Yeah. That's a good one because I've I've used that myself. And I think, you know, as a lot of us know, regulated work is about being right in in context, you know, not being possible in language. So it's really important to get that terminology down and to have that interpretation of what those wording is. In in chat GBT, it could be one thing. But if you have a specific AI system that'll provide the the context that's needed for that area of regulatory science, I think that's really key. You know, just general AI doesn't understand, I guess, regulatory intent, only patterns. So I think having that, you know, domain specific AI is really important. Awesome. Thank you, Albert. Michael, do you have any thoughts on this? I think exactly the same, really. I think you you do need to have a domain specific AI, as it offers greater accuracy. It offers greater relevance and and actually safety than a generic AI system, which you may need to sort of revalidate the the source of the information from where the the the suggestions are coming from. And I think in this area that we're working within within quality management systems, with medical, systems, you know, a lot of the work that we do is around some of the medical words that are not in the general AI search ability. So we do need specific knowledge, you know, gaining from, credible sources, and and that will avoid confusion and streamline streamline workflows in making sure that you know that the information that you're getting is from credible sources. Yep. Just that idea of of context in your workflow. Thank you for that. I think we've all talked about this idea of human in the loop a little bit throughout the conversation. I'd love to hear from everyone your thoughts on where in your workflows humans really still need to make judgments versus where AI can safely operate, and what you've learned about getting this balance right. Maybe, doctor Sanjeev, we can start with you on that one. So, again, from the very beginning of this webinar, we have we are relying on humans, and we should definitely. In in regulatory landscape, would say human must make judgments where clinical or regulatory context matters. Like, benefit risk assessment, deciding whether in in my signal management, a signal is confirmed and deciding what corrective action is appropriate. AI can safely support work that is structured and repeatable, such as organizing evidence, identifying missing records, suggesting mapping, citing supports, and prioritizing. Balance works when AI accelerates preparation and screening, and humans remain responsible for conclusions. The workflow should make that explicit with clear review points and recorded rationale. That is the tricks of any good AI model that will pass through regulatory inspection smoothly. Awesome. Thank you for that. I think that's a great overview. Albert, any thoughts from your perspective? I was gonna say Doctor. Sanjeev said it right on, but I would just say, you know, what I saw within the FDA kinda like, you know, we have to make sure we have the the proper oversight and and the guardrail, so to speak, just to make sure that the system is is operating properly and just ways to flag issues that come up and then how to go about, you know, resolving it for the future. Awesome. Michael, any any thoughts from you on this one? I think from a from a business point of view, you know, the you know, I am really the the person that ultimately responsible for that quality management system and safety that patients that may be using our medical devices or assays. So we need to make sure that we do have that human oversight from a business point of view and a human point of view to ensure that we're not just complying with with, you know, relevant text and SOPs and working instructions, which are, you know, compliant from a written point of view, but we can actually action them on a personal level and that that, the training that we provide our employees are actually regarding safety and the efficiency of efficacy of of the of the devices or or services that we're providing. So human oversight on the quality management system is actually essential, actually, especially if you're in a responsible quality management position. Yes. Agreed. I think all of our our experts agree that humans are still important, So great to hear that from you guys. I think I think there might be folks on this webinar who are interested in how to go about securing buy in and sponsorship for AI initiatives when there are multiple stakeholders that might need to weigh in. Doctor Sanjeev, I think at Biotech, it's a large organization. Can you speak a little bit to some of the processes that you've encountered there with getting AI buy in? Even even with big companies and the small different projects, you need to look for a sponsorship where within the company as well, you know, from different departments. So sponsorship follows when you can show two things. First, the control that can keep the system defensible. Second, the operational impact on cycle time or rework. It also helps to be explicit that AI is a decision support and to specify where review sits on the workflow so the stakeholders can see accountability is unchanged. Once that model is presented, in practice of what I would do, I would start by aligning the initiative to an audit or inspection pinpoint that everyone recognizes. And that is a very good way to start thinking from that end. Usually, time spent assembling evidence and the risk of missing something, then I bring stakeholders in early with a short pilot that runs alongside the current process. So the discussion discussion is based on outputs and measurable effort that is saved. This a very simple policy to how to approve their sponsorship where they get satisfied most of Awesome. Thank you, doctor Sanjeev. Finally, for all our panelists, what's your message for life sciences companies who are thinking about adding AI to their quality and compliance processes? I don't know. Should I go first here? I think from from a company point of view, I think life sciences should embrace it as I've done, and as I've got to know it a little bit more, and and look at the the the last twenty, thirty years of experience, I think it could save a huge amount of time. And I think but what is what is clear is that that quality managers should be very vigilant along the process. They should check, you know, whether the the system actually applies to them, and that human oversight is there to control the quality management at all times, as I suggested before, using the correct standards. So so I would suggest that that businesses do embrace it as much as they can. Wonderful. Doctor Sanji, what are your kind of your final message for life sciences companies who are thinking about adding AI? first thing that we all know that AI is capable. AI is here, and it can move organizations, especially from the audit perspective, from a post episodic audit preparation to continuous readiness, where gaps can be surfaced earlier and addressed with less disruption. And the life sciences companies want that. We just have to figure it out how to use AI now. So my my final suggestions would be based on my own personal experience that start with a narrow use case that removes a real manual effort and insist on traceability so outputs can be defended. Treat each implementation as a controlled process with documented review steps. It is not the AI that fails the inspection. It is the document that made the AI value less important than the AI as a task itself. So it's the documentation that makes you fail and validate to the level of risk and then monitor performance. Start small. Yep. Starting small and then kind of building on that, I think, is a great final. message to leave. Awesome. Albert, maybe we'll close on you. Sure. Sure. Yeah. I would just say, you know, I agree with everything Michael and doctor Sanjeev said. I guess I would just say from the FDA standpoint, kinda you what I mentioned at the beginning, most of the delays are not science based. You know, they are document traceability consistency problems, and these are things that AI can do and do well, you know, with the proper system in hand. And so I think, you know, showing that impact it can have in in those areas, to the various stakeholders, I think, will really show where where where AI is gonna be, kinda really making a difference moving forward. Thank you for that, Albert. So thank you so much to our speakers for these insights. We so appreciate your time. We'll be taking questions from the audience in just a minute. For now, I wanna pass it over to my colleague, Colin, who will do a quick demo of compliance intelligence and, what it can do. everybody. My name is Colin Feeley. I'm just going to share my screen. I hope everyone is doing well in whatever part of the world you're in and whatever sort of weather you have. Can you see my screen? I just want to double check. Yes. We can. Yes. Okay. Thank you, Michael. So thank you for your time, everyone, today. I'm just going to briefly give you a quick overview of our compliance intelligence solution that Michael has been talking about. I've been told I've been given five minutes. I've tried this a couple times today, and it's taken a little bit more. So what before I keep going, of course, if I go a little bit over, it'll be okay. So our compliance solution, which which we're looking at here, we built this because we are traditionally a QMS provider. And what we found is a lot of people use their QMS within compliance activities, and that works well to a degree, but we've never had any specific solutions to help you within that. So we decided to build a structured, repeatable, controlled approach to your compliance activities and bring in AI at key points in that workflow for very specific use cases. So what I'm going to do is I'm going to step through it now and show you where they come in. And we did that in a way by keeping human in the loop, traceability, explainability, top of mind. And we took on, I suppose, the job of understanding prompt engineering, understanding of how AI works and how it can be made to be more consistent in its responses and do that in a way that's designed for life sciences. So what we have here is, again, our compliance solution, which is built around frameworks and control. So if any of you know the likes of, say, Vanta and Drata, which are in the InfoSec space, very similar structure to that in terms of frameworks and controls. But our controls are built specifically for life sciences, for our internal SMEs, and are reviewed and evaluated by external SMEs as well. So you can see here we have the thirteen four eight five framework. We have a progress to see how well we're doing relating to that framework. And then we have the requirements and the controls as you can see here. Now for the purpose of this demonstration, I'm going to focus on the one control, which is the complaint management control. So we have a requirement here and the complaint management control is how we're going to meet this requirement. So what is a control? A control essentially is, yeah, exactly how we're going to meet that requirement. But the structure of it is there's two aspects to it. There's a process definition. So what are the policies, procedures, work constructions pertaining to the complaint management process as well as the evidence. Now the first place we have injected AI into the workflow is organizing your policies, procedures, work constructions against this control. And we have this button here called fetch, which essentially is an intelligent search of your QMS to try and find those relevant documents. And we can see here anywhere there's a star, this is where AI has returned something from your QMS. This means it's easy for you to review and decide if you agree with it or not. If you do not agree, you can remove it. And you can also see here isn't perfect. We manually added some as well. Now additionally to this, everything that's added, we also do a relevancy check to make sure that it is relevant to the given control, and we put this in a table for you so that you can review this, making sure that you're able to see that traceability of what the AI was doing underneath the hood. Okay? We also have the ability to organize and collate your evidence. So that would be the output of a process whereby you can set up rules that every time a complaint is closed, it will automatically be collected here so that you don't have to do that. Okay. The next piece of AI we have is the gap analysis. And this is where really we see big, I suppose, gains in terms of efficiency and time because reviewing all of this information against the relevant requirements can take a lot of time. And in this case, complaints management, this control is mapped to many requirements in 9,001, thirteen, four, eight, five, and many other standards. So checking these policies, procedures, work constructs against all those requirements can be extremely time consuming or very difficult. So we have this gap analysis here That will if there's any issues, it will create gaps, which I'll go into in a second. Then we also have the analysis of your objective evidence. Now in this case, we are going to assess the output of the process against the rules that you have set out in your policies, procedures, work constructions. And how can you see them? We extract them and we make them available for you to see here as well in your analysis rules. So for in this case, it says that if anyone receives a complaint, they should log it within seventy two hours. That would be something extracted from the SOP. And again, you can see that it's traceable. It's easy for you to check what the AI is doing. So if there is any issues found, we produce gaps. And these are logged here. They have an ID. They have a number. So again, there's traceability, human in the loop, and I get to decide what to do. If I don't agree with, which can happen in some cases, I can dismiss it with a reason. And the next time we run an analysis, it will not come up. I can resolve it, and I can also choose to review it with AI. And this is whereby we can learn more about it. We can ask it to give us more information or we can chat with it. We can kind of get more, like, explain this a bit more or help me understand this or what options would I have to fix this, and we can also remediate. Now remediate is where the AI assistant can take actions, but only in certain instances. So for example, it could take it say, for example, if it's if it's recommending that we update here SOP 17, the AI assistant can create a new draft of that document, but only if I'm the owner of that document. It cannot do anything in the system that I cannot do myself. It is an assistant to help me be more efficient in the resolution of those gaps. Okay. So as I've only given five minutes and I think I'm probably up in my five minutes. It's a very brief demo. So what you can see here is, as I said, a systematic controlled repeatable approach to compliance activities. And there is AI injected at three key points, the organization and collation of information against your control, the analysis of that information against the control and the mapped requirements across different standards. And then finally, the support in resolution of gaps, be it giving you ideas of how you can do it, helping you draft information for put into an SOP. And once you do that within the given SOP, you can come back in and reanalyze and make sure you're in a good place. And overall, you have your tracking to see where you are against the given standards. You know you're in a good place before any audit. Okay. So I'm going to stop there and I'm going to hand back to Tina. And I believe you're going to jump into a q and a. So thank you very much for your time, everybody. Bye bye. Wonderful. Thank you so much for that, Colin. We have a couple of questions from the audience, so we'll get into audience q and a here for a couple minutes. So to the panelists, how can you ensure the currency of results so as to ensure validation of the process based on AI? Real examples will be highly valuable. Also, can you share how you ensure data privacy in business AI? Maybe, doctor Sanjeev, I'll I'll start with you on this one. Okay. So the question is mainly, how would you ensure the reference of results? Again, it goes back to case ability and the optimization. So you log the full configuration, model version, from template, retrieval sources, ranking rules, even thresholds at any post processing logic, all you needed for the version control that has to be locked in order to recover the exact point, and you feel the evidence base. And I'll just let me give you a real example here. So let's say you are taking an audit preparation with this. An AI assistant will review an ISO clause and your internal SOP set then produces a gap statement and an evidence checklist. You validate it using a fixed set of, let's say, historical audit scenarios. You expect identical output for each scenario because the coding is deterministic in in its retrieval form. What I'm trying to tell is that you put the guardrails in a way that the difference of the results is estimated with the same parameters and same point how it is to be calculated. And that can be defined again within the governance structure. Wonderful. Awesome. Thank you so much for that, doctor Sanjeev. I have a couple more questions here. So from a regulatory perspective, how important is it to review any AI logic used, especially around traceability? Albert, I think I'll throw this one over to you. I'm sorry. Can you say that again? I missed the From a regulatory perspective, how important is it to review any AI logic used, around traceability? oh, yeah. Definitely. It it's key. I mean, if AI touches it, touches traceability requirements, risk verification, then, you know, it's part of your quality system. And so, yeah, I think it's it's essential. It's, yeah, without a doubt. Gotcha. What are some of the things that you would look at, to, kind of assess traceability? Like, what's the threshold that you would that you would want to see? I think what you know, one of the things, obviously, we said is the human in the loop, obviously, ensuring that it's being properly verified, validated. That's all that's coming to mind, Myra. Sorry. My mind's a little miss. Owen. That's great. That's great. Let's see here. Do we have to create any requirements or upload the guidelines, for example, ISO 9,001 in Qualio? So we have an ISO 9,001 framework in compliance intelligence. We also have a number of other frameworks that adhere to different regulatory requirements, and we can follow-up offline with more detail on that. Let's see here. Do we have any other this is the same question on recurrency of results that we went through. I think that's all of our questions from today. Thank you so much, everybody, for joining. I think we're gonna end it here. And a huge thank you to our panelists for a great discussion. Thank you so much for your time, and thank you, everybody, for joining today. Thank you. Bye bye. Thank you. Thank you, Tina. Thank you, everyone. Thank you, Tina. Take care. Bye bye.