How REI Systems digitized and automated a paper-based sampling process to more rapidly identify dangerous consumer products.

CHALLENGE

The Food and Drug Administration (FDA) is a U.S Department of Health and Human Services (HHS) agency that is responsible for protecting public health by ensuring the safety, efficacy, and security of human and veterinary drugs, biological products, and medical devices; and by ensuring the safety of our nation’s food supply, cosmetics, and products that emit radiation. The FDA’s Office of Regulatory Affairs (ORA) is the lead office for all agency field activities. The ORA inspects regulated manufacturers, conducts sample analyses of regulated products, and reviews imported products offered for entry into the U.S. As the lead office for all FDA field activities, ORA professionals work in a range of program areas and locations, with 227 offices and 12 laboratories throughout the United States.

The ORA’s laboratories play a major role in protecting consumers from unsafe, ineffective, mislabeled, and poor-quality products. They provide a scientific base to support ORA enforcement and regulatory activity. The laboratories test thousands of product samples annually. In the past, most of the sample and regulatory analysis work had been manually compiled into cumbersome paper-based reports. The reports were then uploaded to the ORA Compliance Management System (CMS) and the data shared with two other large FDA databases.

After a successful pilot, the ORA Office of Regulatory Science (ORS) worked with ORA’s Office of Information Systems Management (OISM), ODT’s Division of Application Services (DAS) and the REI Systems to begin the journey to the cloud and incorporate automation for the processing, sharing, and compliance of lab analytical data and lab work packages. The new solution, the Automated Laboratory Information System (ALIS) aligns with FDA’s goals to modernize field office operations and information sharing across the agency and move solutions into a cloud-native environment.

REI SOLUTION

The ALIS team combined program expertise, best-of-breed technologies, and hyper-efficient processes to develop ALIS. End users and stakeholders collaborated throughout the project and REI took the time to understand the enterprise problem and implemented user design, educational sessions, frequent communications, and continuous user acceptance testing which vastly accelerated ALIS adoption and increased the productivity of the first module for ALIS, the Pesticide Analytical Program. The solution included clear purposes and goals, cloud native development, human-centered design, Agile and DevOps processes, microservices architecture, and emerging technologies.

Clear Purposes and Goals for Modernization
ORA launched a modern ALIS application by defining four clear purposes for which it sought to modernize ALIS:

• Increase the speed, improve accuracy, and reduce the effort to aggregate data gathered from multiple laboratory locations – to capture more value from data.
• Move to more secure, reliable, and efficient cloud hosting, away from an on-premises infrastructure.
• Improve the communication of lab results, helping the agency leverage the power of information to increase voluntary and enforced compliance with laws and regulations.
• Reduce IT development and operations/maintenance costs by transitioning to modern technologies, microservices modules that can be incrementally upgraded, and a DevSecOps platform that helps speed future innovation.

The ORA identified its need for ALIS because it wanted a much better tool to gather information and support its communications – within the FDA, outward toward producers and manufacturers, and, for some purposes, to inform members of the public who consume regulated products.

At a deeper level, the ORA also envisioned objectives for capabilities it would seek from ALIS, including, for example, systems that could be easily upgraded to support predictive analytics. The ORA recognizes the value of test results and connected data that ALIS will provide as it is modernized. Looking forward, the agency wants a modular tool that can be upgraded incrementally – able to support several important analytic purposes now, but which, in the future, can integrate a wider range of artificial intelligence and predictive analytic uses that will help it better address the risks/threats which the agency wants its laboratories to be able to predict and target. The agency’s well-defined set of purposes and direction was consistent with REI’s recommendation of a Mindful Modernization™ approach.

Cloud Native Application
Many of the ORA’s processes are unique, so the team developed a completely new and custom cloud-native, web-based application on agency’s GovCloud platform. The cloud native orientation encouraged our team to focus on operational and hosting efficiency and security, as well as support and coordination across many ORA laboratories in dispersed geographies. It also allowed our team to integrate important security tools and sped our ability to obtain Authority to Operate (ATO) quickly and efficiently.

Human-Centered Design
A customer experience (CX) team conducted product research via multiple interviews with what would be the systems’ end users and stakeholders to document the product user requirements. The team created user personas, user stories, and storyboards to help the CX designers create the specifications for an easy-to-use and intuitive application. With REI’s encouragement, the agency’s Human-Centered Design teams are working to focus on a wider range of “users” than just FDA’s own staff (laboratory scientists as well as administrators and regulators). That wide range will include:

• Partner agency teams (e.g., The Centers for Disease Control and Prevention (CDC), Customs and Border Protection, the Environmental Protection Agency)
• Members of regulated industries (e.g., food producers and pesticide manufacturers)
• Interested third-party partners, such as academia and state and local public health partners.

Agile and DevOps Delivery Framework
Once the CX roadmap was produced, we applied an agile framework to incrementally build the business functionality of the application. A DevOps pipeline was stood up to leverage automation and continuous integration and continuous deployment (CI/CD) of capabilities.

A Jenkins DevOps pipeline was configured to build and automate blue-green deployments to FDA’s GovCloud platform. The blue-green deployments created two separate and identical environments to increase application availability and reduce deployment risk. The team could do deployment any time of the day, and the end users won’t notice any outage.

Microservice Architecture
The ALIS is a microservice-based application with a dedicated database. The microservices are deployed in docker containers to an Elastic Container Service (ECS) cluster, a logical grouping of tasks or services. The ECS, a serverless architecture, was used to leverage Lambda functions to seamlessly integrate our application with another third-party system. The ECS also enables cloud-native capabilities like auto-scaling, managed storage, maintenance overhead, and Infrastructure as Code, with minimal code changes.

Emerging AI Technologies
The REI team implemented emerging AI technology into ALIS to collect data about requests that the ALIS application serves and provides tools to view, filter, and gain insights into that data to identify issues and opportunities for optimization. More emerging AI technology is being evaluated now for future enhancements.

PARTNERSHIP IMPACT

The ORA implemented the first phase of ALIS, a completely new cloud native application, in under eight months and is now using ALIS to increase its mission agility, flexibility, and to reduce costs.

ALIS has been extraordinarily successful to date. A particular illustration of this, ALIS has made test results available 55% faster (average 2.1 days versus 4.7 days). Further, since 2022, ALIS helped the agency aggregate the results of sample tests from multiple labs across the country, enabling ORA to better detect counterfeit, hazardous, and mislabeled products, particularly where those products have appeared in separate geographic locations. As well, the ALIS makes the ORA’s IT infrastructure and applications more efficient and reliable because it is the first cloud-native application implemented in ORA.

Because ALIS has been so successful, ORA is considering it a one-stop shop for all sample-related programs. The ALIS is the first full-stack ORA application to be stood up in the agency GovCloud platform using ECS and other technologies. The ALIS digitized analytical packages and automated sample accounting, data analysis, document control, report generation, and other lab operation functions. It also systemized record-keeping, sample tracking, and report activities. E-signatures were added to streamline processes and get reports out the door more rapidly.

Further, the application was designed to be extensible – so that it can easily and incrementally be improved to help the ORA and the FDA proactively identify risks and quickly respond to public health emergencies to safeguard our national supply chain of FDA-regulated products. These improvements will allow the ORA to smoothly integrate and leverage new analytical tools and methods that it is not yet ready to implement until they have been proven and received regulatory approval.

ORA analysts can now work more efficiently and the ALIS system has supported the ORA effectively. As well, there is every indication that this increased level of reporting has been accomplished with fewer errors, according to stakeholder feedback. We are proud of the workload efficiency improvement success at this stage of early implementation, as one ALIS user said, “The ability to generate work packages in a click of a button is great. It took hours before to do this manually.”

ORA is very pleased with the ALIS technology it has developed together with its partner REI Systems. ALIS enables a quicker response to public health emergencies, automating much speedier dissemination of test results from multiple labs, so the agency can address potentially dangerous products more effectively, and better focus its enforcement resources.

ALIS is a FedHealthIT 2023 Disruptive Tech Program Award Winner!