Customer Story

Performing under pressure, again and again

A statistics-savvy R&D team at Medtronic develops a new, easily replicated methodology for evaluating medical device performance

Surgery is a precise art, but even the most fastidious surgeon is only as good as the tools they have before them. That’s why the scientists and engineers at Medtronic who have dedicated their careers to developing better surgical technologies value precision. And nothing says “precise” quite like hard data.

At Medtronic, extensive laboratory testing enables researchers to generate data sets that provide crucial insight not only into device efficacy, but into each individual parameter that affects device functionality. Think sophisticated science with statistically assured reliability. That means safer surgeries and better patient outcomes.

Take, for example, LigaSure^TM technology, a series of devices developed by Covidien-Medtronic to replace traditional sutures and make surgery safer. LigaSure devices apply a precise combination of pressure and radio frequency energy to compress and fuse blood vessels during surgical procedures, thereby reducing operative blood loss and, ultimately, patient recovery times. These instruments are designed to sense changes in tissue impedance – that is, how the resistance of the tissue itself changes as vessel walls are fused. 

Analysis is the cornerstone of continuous improvement in R&D

Though LigaSure is now the industry standard for general, laparoscopic and gynecologic surgery, Medtronic continues to devote millions of dollars annually to improving and advancing the technology through new scientific efforts.

“Advancing statistical methods is a very important part of continuous improvement,” says Jim Pappas, Industrial Statistician at Medtronic. That’s why Pappas has partnered with biomedical engineer and Research Manager Susan Roweton. Together, they aim to continuously improve data analysis methods for the Tissue Effect Research group, a team devoted to advancing vessel sealing technologies like LigaSure.

The group completes testing to evaluate several important performance parameters, including burst pressure and thermal spread outside of the device jaws. “One of the ways that we test device performance is by looking at burst strength to determine at what pressure a seal fails,” Roweton says. “We want that figure to be well above normal human blood pressure – and, when we apply energy, we want to be sure not only that we’re creating a strong seal but that that energy doesn’t affect the tissue beyond the point of application. If a surgeon is operating near nerves or an organ that they don’t want to damage, they don’t want that energy to travel beyond the local point of application. Hence, we look at thermal spread with histology.”

“It’s essentially a reliability test,” says Pappas. “We’re trying to establish a margin that’s safe.” To confirm that margin, the team explores every individual factor – or combination of factors – that could affect device performance. And the result is a series of large, complex data sets that can present real logistical hurdles. 

“We create thousands of seals per year, each between two and eight seconds long – that’s potentially tens of thousands of seconds of electrical variable data alone,” says Roweton. The challenge, Pappas elaborates, is: “How do you manage that data? And [from the complete data set], which data do you actually need to answer the question? We have to condense it down to a manageable size to understand which variables we need to measure.”

To solve these problems, Roweton and Pappas developed a new method of analysis for tissue effect research that emphasizes powerful graphical and predictive modeling techniques for performance data – a discussion of which was published recently in the Journal of Testing and Evaluation.¹

Multivariate methods provide for a more nuanced understanding of both performance and experimental noise

The concept behind Roweton and Pappas’ new method uses multivariate modeling to help understand which factors cause variation – and to what degree. Their innovation? Previous approaches relied on univariate, single-factor inferential statistics as a primary means of understanding burst pressure and thermal spread. But such methods were limited by what Roweton and Pappas describe as a potentially problematic conflation of known and unknown variability.

“We used to analyze our data using traditional t-tests to see whether there were significant differences in groups,” Pappas explains. “But now we’re looking at it from a higher level, asking: What are patterns? What are the trends we can observe over time? What is the reliability? What are the components of variability? Do you see variability increase over the usage of a device? Do you see variability between devices?

“A couple of years ago, I built a tool in Excel [to address these questions], and it was able to create plots alright, but not great. Then JMP came along and I was able to mimic that graph. Not only mimic it, but greatly improve on it. It was so flexible. We were suddenly able to do data mining very quickly. We were able to slice and dice graphs almost instantaneously. And now we’re able to do the simulations instantly versus the 10 seconds my original tool required. JMP has really expanded our abilities.”

With a tool now capable of handling multivariate analyses quickly and efficiently, Pappas wrote a specialized script in JMP^® to automate analyses of performance data. The app – called the Seal Performance Tool – enables researchers to compare burst pressure with other device parameters. With this tool now available to the whole Tissue Effect Research team, they are able to more objectively assess trends and overall reliability.

“The beauty of the Seal Performance Tool is that everybody can use it,” says Pappas. “In fact, everybody [on the team] is trained on it. And consequently, our consistency in reporting is much better. Our analyses are more defendable, and with visualization, we’re able to take some fairly sophisticated analyses and bring them down to earth for all our engineers.”

User-friendly JMP^® applications systematize analysis across cross-functional R&D teams

To take full advantage of the Seal Performance Tool, nearly everyone in the Tissue Effect Research group has JMP, and Roweton and Pappas use JMP Pro. “JMP has really helped ensure consistent [methodologies] across the group – and consistent reporting of results,” Roweton says. “It has really bolstered our evidence generation capabilities.

“It goes without saying that we require significant analytical rigor built into all of the protocols and reports that our team writes. Our group is highly aware of proper statistical methods and their use, because all of the data that is submitted for FDA approval is dependent on [accuracy]. We’re demonstrating evidence that our products are safe and effective.”

Moreover, much of the time, members of the Tissue Effect Research group are called upon to act as translators on cross-functional teams, she says. “Our engineers sit on new product development teams not only with other scientists and engineers but also marketing, regulatory and quality associates. It’s really valuable to them to have the ability to show a picture [in JMP] that explains some pretty complicated stuff. They can show how testing went or how the device performed in comparison to a predicate device.” Take a comparison of thermal spread versus vessel diameter, for example. “Just by looking at the data visually,” she says, “you can see the bands where 95 percent of the values should fall. We’re able to see at a glance if performance is consistent across the range of vessel diameters.”

Moreover, she says, regardless of which engineer is sitting on a product development team, “our regulatory team knows they’re going to see the same information and trust the data for our submissions. That’s really important both to our team and the culture of our whole group.”

The most bang for your buck

“[Medtronic] invests a lot of money in R&D to generate the data [we need to build that confidence], with all the testing needed for new product development,” Roweton says. “So we want to make sure we’re using those resources well. We don’t want to waste any part of that data. And with JMP, we’re able to extract more information than with other methods.”

Pappas and Roweton are already at work on a second application that will automate predictive analytics for the group. “If we can put [this new app] in the hands of people doing algorithm development for our devices,” Pappas says, “they’re going to be able to generate another layer of understanding about this device – not only just how burst pressure is related to vessel diameter, but also what the relationship is between electrical parameters and burst. And what we need to do to tweak a device so that we get the ideal profile.”

“The value of JMP to Medtronic lies in being able to understand [and make use of] all of our data,” Roweton says. “More [efficacious analysis methods] mean we’re able to better understand the performance of our devices over time and ensure that we’re designing safe and effective devices. That’s our primary goal. And JMP helps us with that goal by enabling us to get all we can out of our data. It’s something we couldn’t do before.”