Background:

The Department of Defense (DoD) maintains a center that manages and protects a massive trove of personnel, manpower, training, financial, and other data. These data catalog the history of personnel in the military and their family for purposes of healthcare, retirement funding, and other administrative needs vital for them, their families, and the larger DoD community.

The center’s responsibilities and impact are vast. It supports 14 government agencies, maintains over 35 million records in its Personnel Data Repository, and processes about 5 million transactions daily to verify identities, benefits, and entitlements. In addition:

• It regularly surveys active, Guard, reserve component members, their families, DoD civilians, and Service Academy students, gathering key data to inform policy and program decisions. It also issued 2.8 million Common Access ID Cards last year alone.
• Since 1990, 4 million military applicants have taken the computer-adaptive version of the Armed Services Vocational Aptitude Battery (ASVAB) exam, facilitated by the center, highlighting its contribution to the military recruitment and evaluation process.
• It oversees 230 current and historical databases, websites, and programs, each a critical piece in the puzzle of DoD’s data-driven operations.

The center spearheads developing, refining, and maintaining sophisticated, secure technological solutions to support human resources (HR) and wider DoD missions. Central to this mission is its Software Verification and Validation (SV&V) team, which confirms software requirements are accurately defined (verification) and that the systems under evaluation correctly implement the required functionality (validation).

The team’s work ensures that technical standards are set, security is integrated into system design, and solutions are developed that support HR initiatives within the DoD. As a result of these processes, automated governance activities carry out tasks to further certify products and services, assuring mission partners that standards have been met.

Challenge:

The center’s mission goes beyond just keeping the DoD’s IT backbone up and running; it’s about ensuring everything works—securely, seamlessly, and without a hitch for everyone relying on it. Meeting this mission is a colossal challenge, especially from an SV&V perspective.

The SV&V’s Quality Assurance (QA) team tests over 236 applications and manages up to 800 releases annually. This involves running roughly 50,000 test cases—both manual and automated— supported by a dedicated team of 94 professionals. It’s a Herculean task that underscores the absolute necessity for QA and SV&V processes that are effective, reliable, and can scale up to maintain the high standards expected of the center. With such a heavy workload, the organization faces an increased risk of production issues, missed schedule commitments, and a rise in the total cost of ownership.

Approach:

In response to these hurdles, the center turned to REI Systems, seeking not just solutions but a transformative vision that integrated automation and lean principles. REI took charge of the SV&V program to provide independent verification and validation services, offering an unbiased, third-party perspective on the software to ensure efficient and assumption-free testing aligned with government requirements.

We improved the center’s testing efficiency by swapping out slow manual tests with a custom automation framework, drastically reducing testing times and boosting precision. We integrated continuous testing directly into the Continuous Integration/Continuous Deployment (CI/CD) pipeline, ensuring all code updates undergo automatic testing for rapid issue identification and resolution.

Our approach emphasized transparency and real-time monitoring, demonstrated by setting up an SV&V metrics dashboard for instant access to key quality metrics. This enabled the government to assess Key Performance Indicators (KPIs) for measuring software quality and reliability. Furthermore, on-demand generation of quality metrics allowed for tracking progress against testing objectives.

We adopted agile practices for more flexible, team-oriented project management, moving away from traditional methods to improve collaboration and project flow. Beginning with a successful agile pilot led to the implementation of the agile methodology for the entire project, setting a baseline for other projects in the program to follow the criteria and reap its benefits.

By implementing lean management principles, REI significantly improved its operations by discarding outdated practices in favor of adopting more efficient tools. This transformation led to enhanced efficiency in managing test tasks and increased project adaptability.

• REI has developed an in-house Automation Testing Framework called JTF and JJF to cater to the unique requirements of the center, which are not available in the market. This automation supports various types of testing, including release testing, regression testing, webservice testing, batch testing, and backend testing, ensuring that new changes do not introduce unintended issues. These efforts have resulted in a 50% conversion of manual test cases to automation testing, significantly reducing release testing time.
• Our Risk-Based Testing strategy focused on identifying and testing high-risk areas of software, optimizing testing resources, and enhancing software quality and reliability. We established strict standards for functional and regression testing, which verify each function of the software operates as specified, with a focus on high-risk areas, and ensure new updates or changes do not disrupt the software’s existing functionality, respectively.
• Exploratory testing, a flexible testing approach where testers dynamically explore and experiment with software to uncover hidden bugs, has become key. It strengthens our software and sharpens our team’s testing skills.
• We’re also building a detailed repository for all process and testing docs to back a new test case management system on the horizon, aiming to make the QA process quicker and more thorough, enhancing both manual and automated testing.

Impact:

Our partnership with the center significantly accelerated operational agility and quality across key sectors, improving quality assurance, productivity, process optimization, and automation. It serves as a stellar example of our Mindful Modernization® approach to projects by focusing not only on technology and processes but also on people and culture. The impact of the program to date includes:

Quality Metrics and Reporting: REI’s approach helped the SV&V team achieve 100% quality in its output for nine months, far exceeding the industry’s typical defect escape rate of 4.3%. This is a major improvement from the past when over 200 defects found through each month before REI’s intervention.

QA Team Productivity: By adopting agile processes, REI boosted the QA team’s productivity by 270%. This jump showcases the team’s increased efficiency and establishes a new standard for future productivity.

Cost and Time Savings: REI’s efforts significantly reduced rework, saving both time and money. Before REI came on board, release rework cost about $100,000 a year. Moreover, REI’s agile methods are set to reduce the release testing cycle by 15%, improving efficiency in terms of both costs and time.

Process Improvements Via Lean Principles: REI’s process improvements significantly boosted operational efficiency by:

• Speeding up the QA team’s vetting process by over 300%, cutting onboarding time from four months to three weeks.
• Implementing the RRF process, boosting deployment efficiency and fostering better stakeholder collaboration and transparency.
• Shifting to remote work for weekend application testing to increase security and productivity, removing the need for travel for production release.

Automation: REI plans to automate 75% of its manual testing, targeting around 58,000 cases. To date, the center’s Automation team has automated 56%—over 26,000 cases—greatly improving test coverage and speeding up testing processes.

Ultimately, REI’s collaboration with the center showcases the significant benefits of our Mindful Modernization® approach, in which focused technology upgrades and process enhancements drive major gains in efficiency, quality control, and overall operational performance.

Copyright © 2024 REI Systems. All rights reserved.