US Marines use analytics to save billions

Predictive modeling with JMP^® Pro forecasts equipment losses

Four US Marines who used predictive modeling to enhance a system for forecasting equipment losses during combat have won a prestigious military award. Their analytics-enhanced Combat Active Replacement Factors (CARF) system is expected to save the corps billions of dollars.

Lt. Col. David Scott, Capt. Aaron Burciaga, Cynthia Cheek and Janice Scoggins received the 2013 Rist Prize, given annually to honor the best example of implemented military operations research. The award recognizes the work they have done to update the CARF, the US Marine Corps’ system for forecasting equipment losses during combat.

“This is a good-news story,” Burciaga says. “This is the Marine Corps thinking smarter, using best practices.” In this case, thinking smarter is leading to substantial increases in efficiency. The work of Burciaga and his team could save the corps as much as $2.3 billion in requirement costs by 2017.

An analytical, data-driven approach is essential for ensuring that everything – from radios to tanks – is provided in sufficient numbers without creating waste. But when Burciaga took the project, the CARF system was dependent on a 15-year-old methodology. Many of Burciaga’s colleagues at Headquarters Marine Corps simply didn’t trust the system.

“It wasn’t that people hadn’t tried to recalculate CARFs,” he says. “Analysts had made many attempts in the past.” But Burciaga’s team overcame the challenges to make the impossible possible. His arsenal included JMP Pro statistical discovery software from SAS, which brings predictive modeling and other advanced analytics methods to the desktop and displays analysis results graphically.

One of the main obstacles to the CARF overhaul was a lack of useful data. Using an ensemble method and recursive partitioning, Burciaga and his team were able to assign values to 100 percent of the Marine Corps’ most critical equipment. “With recursive partitioning,” he explains “we were able to save a lot of time and a lot of brute-force programming by using higher-order, proven and validated methods.”

Team members chose bootstrap forest as their method of partitioning, allowing them to narrow the environment in which they looked at the variables. Bootstrap forests produce results that remain stable with minor data changes and are resistant to overfitting when a number of predictors are used.

Bootstrap forest was an effective tool, Burciaga explains, because the Marine Corps has hundreds of thousands of types of equipment in its inventory but complete data for only about 300 varieties.

This method allowed the analysts to narrow their view to the smaller sets of equipment for which they had the data. They were then able to extrapolate those findings to the larger inventory.

Team members were also able to address the issue of missing data.

“Sometimes you can’t appreciate what’s missing until you start breaking it down, as the descriptive statistics and distributions ... allow you to do,” Burciaga says. “That’s helping us see where there are gaps in our processes and systems.”

The process uncovered a need for data filtering, cleansing and profiling to ensure data was getting captured in the most effective way. The new methodology has led to a total of 34,818 new CARF values, roughly three times as many as before.

Feedback on the new tool, known as the CARF Statistical Analysis Tool (CARF-STAT), has been positive. Lt. Gen. William M. Faulkner, Deputy Commandant for Installations and Logistics, nominated the team for the Rist Prize.

“Despite immense challenges, these nominees applied novel operations research techniques in the development of the policies, processes, and planning factors necessary for the development of a comprehensive strategy to support our warfighting to 2017 and beyond,” Faulkner wrote in his nomination letter.

For Burciaga, CARF-STAT is just the beginning of enhancing efficiencies through the use of advanced analytics. “There’s so much more for me to explore,” he says. The next step is to apply the new methodology and tool to the resupply of stateside troops – and extend the practice to other branches of the military. “Having common tools with the best methodology,” Burciaga says, “will help us all develop good ideas into better ideas and help us arrive at more consistent answers and shared solutions.”