Customer Story

A winning design of experiments game plan

The design and analysis capabilities of JMP^® and JMP^® Pro have US Synthetic engineers on the path to faster, more reliable findings

In football, a desperate, long-distance throw into the end zone — a Hail Mary — is occasionally the best option. But in research? Never. That’s a lesson Jed Campbell conveys when expounding on the virtues of a design of experiments approach to problem solving.

Campbell is quality manager at US Synthetic, based in Orem, UT. The strategy at US Synthetic is to empower all employees to see and resolve quality issues, and Campbell teaches techniques that are most likely to lead to optimal outcomes. The Hail Mary is not among them.

Here’s the gist of Campbell’s Hail Mary analogy: Experimenters may sometimes feel they lack sufficient time or resources to conduct a properly designed experiment, instead opting for a Hail Mary approach, executing a few experimental runs they speculate might include the best solution.

Those experiments may appear promising at first glance. But the overwhelming odds are that they’re not going to withstand scrutiny – certainly not the scrutiny afforded by JMP, US Synthetic’s go-to statistical discovery tool.

With JMP, a more promising strategy inevitably emerges. In rebutting the Hail Mary, Campbell applies JMP Graph Builder to examine experimental runs over time and make scatter plots of the design space. He uses stepwise regression to build models. The data now reveal a better set of factors, and in a location that wouldn’t have likely otherwise been explored.

“With JMP, you can do a definitive design run and get great results in approximately the same number of runs,” Campbell says. “A properly designed experiment will yield better results every time.

“This is a lesson I use when teaching design of experiments. There’s a strong tendency to vary one factor at a time, and to think you’re doing real science, when in fact you’re just basically hunting and pecking to find something that works. Without formally designing an experiment, it’s easy to say, ‘Well, I guess we’re at the sweet spot; let’s move on.’ And you end up with an inferior product.”

That’s not how research is conducted at US Synthetic, where a DOE culture is fully ingrained. It’s a founding principle of the company’s problem-solving curriculum.

With JMP, ‘dive in, decipher, design an experiment’

US Synthetic designs and manufactures diamond inserts for applications in down-hole drilling tools, primarily for oil and gas exploration, and Campbell and his research and process engineer colleagues are the organization’s frontline problem solvers. When a problem arises, they call in JMP to, in Campbell’s words, “dive in, decipher it, design an experiment to solve it and move forward. With JMP, there’s a graphic and number for everything, and it guides you right through the data exploration. It’s so easy to use.”

Tyler Dunaway, a process engineer, concurs. Dunaway arrived at US Synthetic having only used spreadsheets for statistical analysis. “The thing that was most enticing to me,” he says, “was the user-friendliness of JMP. The platform is so easy to navigate for someone who doesn’t have a ton of experience with statistics. It’s really helped me develop my skills.”

From basic screening designs to implementation ‘screaming fast’

Campbell offers an example of how those skills are deployed. The US Synthetic team was working with a customer who had a product with a defect, but no one was certain whether the defect was cosmetic or structural. Through the DOE process, they were able to prove that it was purely cosmetic. “Halfway through the presentation, the customer said, ‘OK; let’s move on,’” Campbell recalls. “They saw the end of the story already.”

Process engineer Matt Brown relates another predicament, in which he and his colleagues were trying to dial in a process without initially knowing which factors to control. They began with some basic screening designs, which led to a more definitive design. They then conducted a series of DOEs.

“We were able to identify the three factors that were most important, and we really dialed in the process from there,” Brown says. “We created a machine based on those results that we’re now using on the floor. It took about six months from the beginning of the process to the implementation of the machine, which is pretty screaming fast for that kind of project.”

The ease with which the team was able to set up a DOE is “just an awesome feature of JMP,” Brown attests. “Trying to conceptualize how to do what we did in any other program than JMP would have been daunting.”

Jason Wiggins is among the group’s JMP Pro users. Wiggins says JMP Pro provides him with an even more powerful arsenal of modeling tools: “Features like model comparison, mixed models, neural modeling and stepwise regression with validation are way cool!

“One of my most used and favorite features is the ability to bootstrap almost every statistic generated in analysis. It’s super-useful for estimates and to visualize sampling distributions.”

Of JMP in general, Wiggins says, “We all really like that the software itself is consistently more capable than each of us is individually. This means that we’re always presented with an opportunity to push our learning.”

Interactive data to share

The engineers have written numerous scripts for US Synthetic’s testing department that allow its staff to reduce repetitive tasks by automating report generation – a tremendous timesaver. “We used to have three people doing it full time,” says Jason Lott, a research engineer. "Now we’ve got two people doing it part time, freeing them up to do additional testing.” They then send the data tables to the customer, “and the customer can do their own custom analysis if they’re looking for something else,” Lott says.

Brown writes scripts for his own use that, he says, “I could think about writing in Excel, but the time it would take would probably make it not worth doing in the first place.” And, he adds, he and his colleagues are always discovering something new in JMP: “We’ll be in a meeting, and somebody shows something, and you’re, like, ‘Do that again!’”

“Every Monday morning, we have an all-engineers-on-deck meeting on various topics,” Campbell says, “and there are usually two or three executives in the room, and they’re able to see what we can do with JMP; it’s definitely recognized. The VP for R&D is a JMP user. It’s used throughout the organization.”

That’s clear recognition of the power of process. No Hail Marys in the US Synthetic playbook.