Dr. Shawn St. Peter - Equipoise, Innovation, and the Role for Comparative Studies

Thank you. Great. Well, good morning, everyone. Is this microphone on? Can everyone hear okay? Yeah. Can you hear now? I just want to make sure it's on. Great. Well, good morning. So this morning we have a very special guest, Sean St. Peter, from Children's Mercy Medical Center in Hospital. We'll be our second annual visiting professor. I'm usually pretty loud. Can you hear me now? Says it's on. Is there IT folks back there? Steve Fishman's back there. Is our audio guy he's trying to figure it out. Let's see. How about now? Okay. All right. So again, good morning. So today we have a very special guest in the way of our second annual visiting professor, Dr. Sean St. Peter from Children's Mercy Hospital. Before I get into a British, Sean's background, I didn't want to quickly acknowledge Stuart and Jane Whiteman. And as many of you know, the Whitemans have been incredible benefactors to the hospital, both in terms of the hospital itself, as well as our department. And it's really through their ongoing generosity that we're able to hold our Whiteman lecture and invite Sean to visit Boston today. Sean is actually the surgeon in chief at Mercy Children's Hospital in the director of their pediatric surgery training program. He's also the director of their clinical trials. They're center for clinical perspective trials. And it's really this really latter aspect that why we invited him today. I think many folks in the room, as well as a throughout pediatric surgery would agree that Sean, probably perhaps more than anyone else, has made a huge impact in our field in terms of really changing how we care for pediatric surgery patients through high quality events and particularly trials. So Dr. St. Peter, he's been the recipient of multiple teaching awards and mentorship awards, mostly around his training of the next generation of trialists. He's also been the author of over 350 peer reviewed publications. Many wish have to do with high quality events and clinical trials and pediatric surgery. And again, I think perhaps more than anyone else in pediatric surgery in general surgery. Otherwise, he's really moved the needle in many people, you know, really have to say you've actually done that. This is really where he's walked the walk and talked to talk. So today we're going to hear him talk about clinical trials and integration of innovation. And again, we're really proud to have Sean as their second annual busy white man professor to Sean. Thank you for the nice introduction, Sean. A bit exadurant, but I appreciate it. This is really an honor. I'm excited to be here at Boston Children's. This is something I would have never imagined what happened, but here we are. So thanks again. So this is now at the four brain of most practicing physicians. This is at the four brain of most practicing physicians and it's a common vernacular. It used to be something that we'd have to review, but we don't really need that now because it's part of the basic curriculum. But it emphasizes the evolution from expert opinion at the lowest level of evidence to randomization at the highest level of evidence. And we've come a long way, even since I finished training where expert opinion is what really dominated surgical training. Even when therapy, even when there was evidence that existed. So I got a tip in my hat to my hero, Archie Cochran, who recognized this 30 years ago, saying that even therapeutic procedures should always be supported by randomized controlled trials. When I was a chief resident in general surgery, I had matched in the pediatric surgery. I was told that randomized trials cannot be done in pediatric surgery. Now, I guess having said that, based on the evidence or based on what I was told in medical school, I drove off to residency having only ruled out two things, pediatric surgery and academics. So the reason I was told you couldn't do randomized trials in pediatric surgery was because parents would never consent and surgeons could never have equipoids. For the first part, like anything else, you just have to ask. But that second statement, I think there's some truth to that. Surgeons may not be able to have equipoids. The concept of equipoids was introduced by Charles Frieden in 1974, the idea that a physician could find a condition in which they were truly indifferent to the therapeutic value of an experimental treatment versus a control. That sounds nice and theory, but think about that for a second. How is that possible? Particularly for those of us whose actions influence the outcomes, we don't turn the outcomes over to a pill. How could we possibly reach a state that we think that there's no difference in what we do pathway A or pathway B? Ben Friedman recognized that and he said that even the slightest accretion of evidence is going to sway someone away from equipoids toward one path or another. And we know that because we all have the way that we practice. Here's something that you're never going to hear said. Surgeon finishes a case, says to the resident, when you put in the post-op orders, all your binary choices, feed, don't feed antibiotics, NG2 fully, just flip a coin because I have true equipoids. I don't care. And honestly, I don't think the way I manage a patient matters. That's never going to happen. We practice the way we do because we've lost equipoids. So how can we conduct a trial if we can't have equipoids? And equipoids is supposed to be the basis on which we do a randomized trial. And the answer is, if individual equipoids is not possible, then let's move away from ourselves, look at it from a hospital level or a health care system level. And if we have enough justification to support both arms, and there's no proof of superiority, well, then there's your equipoids. And then you can carry on. Such that I would argue you don't need to have personal equipoids to even design or participate in or to support a trial. And I'll give you an example that came from our institution. When I started fellowship, a patient with an empire, a scene on the floor, could be seen by a hospitalist who had read the paper that primary fiber analysis is better than putting in a chest to see how they do. So they would call IR and go down that path. Then the next day, another patient with an empire would be seen. And they would have read the paper that said primary vads is better than putting in a chest to see how they do. So then they would call us. So they were being sent down both pathways simultaneously with no comparative evidence between the two. We were a house divided. In that circumstance, our interventional radiologists there were convinced that fiber analysis was effective. But on the basis of retrospective data, we knew for sure that the orcoscopy was better. So how can we conduct a trial in that environment? The answer was there was no comparative data. Our assumptions require proof. And if we have a fixed management protocol, then we don't influence the way that the patients do. The therapy will influence the way that the patients do. And that right there was the crux of the whole thing. And that's why we were so convinced that vads was superior. And that's why our sample size was so small. Because we gathered our retrospective data and the surgery patients were getting out of the hospital over two days sooner. And that's because we pulled chest tubes in real time on a 24-hour clock and were comfortable sending patients home as soon as we pulled the tube. The patients on the medicine service would sometimes keep a tube through the weekend and after a tube they might stay for a day or two of observation. When we first proposed this to the IRB, they said no, because there's two heterogeneous of a population. We wouldn't be able to find what we were looking for. We countered with the argument that right now in real time, at least at this point in 2005 and real time, patients were going down two pathways. So we have an ethical obligation to do this study. Because right now we're sending into fiber analysis or we're sending the surgery and we have no comparative data between the two. So we included patients that had this in their pearl space. And for those of us who take this out, you can see why we found it entirely incredulous that someone could suggest that this could be treated with a chemical. In the fiber analysis arm, they got three rounds of TPA. That's what standard thoracoscopy. During the conduction of our trial, the London trial-immers, this came out of great ormen's street. We didn't realize the study was going on. Interestingly, the protocol was almost superimposable on top of ours except they used Euro-Kinase instead of TPA. And in 60 patients, they found no difference in length of stay. They found that that's was more expensive. And basically one in six failed by analysis. So we completed our trial and despite the concern about the heterogeneity, the randomization word, and there was no difference at consultation in any of the measurable surrogates for illness. And of course, if our protocol is the same as great ormen's streets, results should be the same. And they were right down to the number. We also had the same length of stay. There was no difference in recovery, days of oxygen, days of fever, analgesic doses, but the procedure charges were greater and there was a one in six failure rate. So we realized there was no advantage to primary vats. And having gone into this with total conviction that that's was superior, we were getting toward the end of this trial and my fellow asked me what I would do if my daughter came up with an impalignment. And I said I would just do the fiber analysis. I wouldn't enroll her in the trial. As the PI of the study, I wouldn't enroll her in the trial because nobody was getting sicker. And so having seen them both in real time and managing them both in real time, that's what swayed the management. So then we went on and the way we treated all the patients was with the three doses of TPA and that they continued to be ill after four days, then we would check to see if they had persistent pleural space disease and then they would go into vats. If they had no pleural space disease, then they would continue antibiotics. Obviously 18 observations is not enough to be very conclusive, but this is this is the right pathway. But after we finished the first trial, these guys were happy because they were right and we were happy because we felt like we could treat everybody in a smarter way. So we continued with the prospective observational study after that trial was done. And in the next hundred patients, what we found was the exact same. They performed just as well with the fiber analysis as they had done in the trial. And one of the concerns was that if you go to vats after fiber analysis, you're going to do a lot worse. That wasn't the case. The patients who got vats after fiber analysis actually did better than the patients who got primary vats with a length of stay that was a full day less once they had got treated first. Once again, there was about a one and six failure. Then we realized we were kind of jumping because of this previous protocols. So now we know in a third study's just been done in Spain that if you have a protocol in place that has a fixed endpoint that you declare failure, then you're going to have about a one and six failure rate. But now we're redefining failure because we realized you can just give another round of TPA because again, they're not going to get sicker. We've all done vats on somebody and you have a mechanical degree of the bad long, you have barrow trauma to the good long and they end up getting sicker and they go to the ICU and they run on the ventilator for a while. You don't see that with with the fiber analysis. So we started becoming more patient. And in our last five year experience, we've only had a 4% rate of vats and both of those patients were in the first two years of that cohort. So it's been three years since we've done a vats and our institution. So now if we have a patient that continues to act ill, we be patient or we repeat the fiber analysis, particularly if they have a locally-loculated disease because almost every time it's the perincomal disease is continuing to drive the illness. And even when it's continuing to drive the illness, the consultation is typically prefer. So if the patient's eating in their op oxygen, then we're done. We don't have to pursue that any further. So now we're starting this journey thinking that we were just going to go through the mechanical steps to prove that our vats was superior. Now they don't need us anymore. I may not do another vats again. The irony of that story is that you can imagine the fact that I took from the surgical community when we started to bring this data forward because it wasn't what people wanted to hear. And on several occasions, I've actually had reviewers say that I'm clearly biased when trying to write anything about this topic. They're right. But in a 100% the opposite direction. I do sound like I'm clearly biased now, but they don't realize that my bias was completely in the opposite direction. I never came at this with a biased or bivornoluses. I face that data with scientific certainty but clinical humility. So if we're going to evolve in our practice of surgery, it's going to require stepping over some sacred cows or tipping sacred cows as they say in Kansas. And the enemy of practice evolution is surgical dogma. Surgical dogma has been traditionally what heavily influences our surgical training. But you know what they say about tradition. Now thanks to the QI projects that are currently pervasive throughout the hospital, the QI, the PI, the SPS, the lean transformation that a lot of hospitals are going through. It's starting to become common now that if somebody says that's the way we've always done it, it raises a red flag because people now see that as an opportunity for improvement. But that hasn't always been the way in surgical training. So as an example, with the fundo-playcation, when you wrap the top of the stomach around the bottom of the esophagus, what was traditional teaching and a rolled off the tongue of any general severe resident is that you have to have two centimeters of intro-dominal esophagus to have an adequate wrap. And so you have to separate the esophagus from the diaphragm, bring it down to the abdomen, and then do the fundo-playcation. Get that effect right there. The problem is when you do this on babies and little kids, they get a herniation where the fundo-playcation goes up in the media stymum. That's the G.J. there. There's the stomach up in the media stymum. That's a bad problem. And that can lead to long sequences of operations. That's why there's a lot of folks in the G.I. community that don't believe in fundo-playcacers because they've seen patients go down this path. And they're afraid of going down that path. So at our center, when I was a fellow, Whit Holcomb had begun feeling that fewer less dissection in placing stitches between the esophagus and the crews would lead to fewer herniations. So he was moving down that path. And then Keith Jorgensen and Matt Carman at University of Alabama were bipolar on the topic. Where Jorgensen felt you needed a complete dissection. And Matt Carman felt that you didn't. And the way this study happened was the senior fellow and doing a fundo with Whit and Keith Jorgensen was visiting and we're chatting in the operating room and we're having this debate. And so, ask the question, would you consider randomizing around that point? Would you consider doing no dissection? And Keith said, yeah, I could do that. And so we reached out to Matt. He said, do that. And Doug Barnhart was at Alabama at the time. And I was on my way to the college to take the clinical trials course where Doug Barnhart was in my working group. So we started putting pen to paper and put together the randomized trial to compare complete dissection. And we just pulled that complete dissection as you see there on the left where you get the crews separated away from the esophagus and you create that space to no dissection. We leave the front of the esophageal membrane intact. Because we had been placing four esophageal crudal sutures, we couldn't change two variables at the same time. So both groups got the four stitches. The primary outcome was a high alert hernia based on an upper GI at one year. And for the surgeons that were involved, we had three surgeons from each hospital. I'm not going to say we didn't trust them, but we didn't trust them. So you had to take a picture after dissection and before stitch placement to see if your dissection was in agreement with your allotment. And we reviewed those photographs every year at absent when we reviewed our annual data. And the Hawthorne effect worked because there were no patients removed because the photograph didn't confirm the randomization alignment. We had to stop the study early. The intent was to go for 360 patients, but at one of our annual meetings, we saw a massive effect size emerge where there was a five to six-fold increased risk of herniation and re-operation and the people that got the traditional dissection. That was based on one year follow-up. So we continued to follow our cohort for a minimum five years. We were able to get a hold of two thirds of them, which we were surprised by, at a median follow-up of six and a half years. And what we found is that there was a few more herniations in each group, but the effect size remained similar. So to complete the sequence of experiments, since both groups got the four stitches, we had to do the study where you got the stitches where you didn't get the stitches. And of course, we didn't see any difference, but was remarkable is that there was no hernias in either group, which taught us two things. One, the dissection really was the culprit, because since neither group got the dissection, neither group had a hernia, that was really the problem. The stitches were never a factor. The second thing is that clearly we were getting better at the no dissection, because we were having that three to six percent rate of herniation in the previous study, and the herniations have functionally gone away. So now we've got a safer, quicker, simpler operation that's more reproducible. So I've often joked that if you're in a practice of more than one, you have a single partner, then you probably have a practice discrepancy that's worthy of investigation. And every surgical group has this debate. So this is a fun example. This is cats and dogs, this is Yankees red socks. Everybody's got an opinion, everybody's right, but there are very many people who just don't care. And that is, with perforated appendicitis, should we irrigate? It comes up at every M&M, people blame abscesses because you did it or because you did it either way. That's what people believe. So one of the oldest dictums in surgery is that delusion is the solution to pollution. But starting about mid-century 1950s or so, there was a move where people started to show physiology that maybe that wasn't the case. Macrophages can't swim. And maybe the the paradigm is a clever space that that walls off contamination so that irrigation could spread contamination to unprepared portions of the perennial cavity. Solid theory, either way, so let's just put them head to head. So we included patients that had perforated appendicitis, which wasn't defined as suggestions of perforation but clear proof of contamination, either a whole in the appendix or stool in the abdomen. In the irrigation group, you got a bag of saline hooked up to your suction irrigator and the suction group you didn't. In order to control the quality of suction, they all had the same battery powered suction irrigator, not the exact same, that would be gross. But we did use the same device. They had the same post-operative management. And what we found was that there was no difference in the demographics. Interestingly, in the study design, the way that we handled this was we had the irrigation bags moved out of the OR because I was concerned there was some people would just get frustrated and irrigate. So that way, if they were randomized to no irrigation, no bag was brought to the room. And as they're doing the operation, if they asked for irrigation, the nurse would have to leave the room. And I was pretty confident that they would lose patients and wrap up the case before the bag came and got hooked up. And it worked. So there's only one patient in the suction only group that got irrigation, which of course was analyzed with intention to treat analysis. And they got a pretty decent eloquot on average, almost a liter. Most people just used the whole bag. And so after a century of debate, what we found was not much. The abscess rate was the same. There was no difference in time to initial POs, regular diet and narcotic doses, days of stay, hospital charges, temperature curves were the same. How about the people who got an abscess? No difference in days of drainage, days of stay, time to regular diet, days of home health or total charges. What about the distribution of abscesses? Did the irrigation have any impact there? Nope. The same distribution. So I'm not sure I've seen a comparative study that had more equivalent results or there's truly no difference. So it turns out there was a population who doesn't care about irrigation and that's the patients. And so I don't think we answered the question because most people are going to continue to be convinced that their way is right. And neither of them are actually wrong. So moving into the concept of innovation, that's a different beast altogether because how can we do a study or how can we establish equi-poise if we don't have enough experience to even be able to establish that. In an ideal world it would look something like this. Somebody gets an idea, maybe they try a case, now they've got a case report. They do a handful of those cases, now they've got a series of them. They do enough that they start to have case control studies, they develop a cohort that they can compare to historicals or against their contemporary compadres and then ultimately prove it with randomization. But what if the difference is really, really big? And that can be a problem. That's a disruptive innovation. You don't need to randomize trial to tell us that that thing in your pocket is better than the 23 devices it replaces. And that's what happened with laparoscopy. So all of these things came out. The colitis was active, the malnectomy, the fractomy, a adrenalectomy, a fund application and many others without a randomized trial. No randomized studies were done. The comparative studies, the delta was so big that it was quickly unethical to even consider randomizing because you could accomplish the same operation without a laparotomy. And so how could you compare those two? That was a disruptive innovation. In that case, what happened was somebody had an idea, they did a case, they reported it, they had a case series after they developed a couple of case series, bam. We went straight to standard of care. What's wrong with that? Is there anything wrong with that? No, not if you're right. The problem is what's wrong. Then that's where you have a problem. So what is subconscious malificence? Is it that? No. It's more like that. The frontal lobotomy impossible today to imagine treating mental illness with penetrating brain trauma. But yet this was a common therapy in the 40s and 50s. The man who brought this to popularity is a noble area. This is practicing an accepted manner to the harm of patients. It's subconscious because nobody would go have malintent on patients intentionally. No one would do that. But view through historical lens, it's happened many times. The arterial bypass for a stroke. After over a thousand cases were done, then a randomized trial showed no benefit and it went away. Arthoscropic debris, a embossed arthritis became one of those common operations performed in the United States. That was more common than a lab coli. That was as common as an appendectomy. And then a trial in 2002 showed no benefit over placebo after literally millions of patients had had the operation. Other examples, the collectamies for epilepsy, the angle check ring, re implants for low grade reflux, the organectomies, all the unnecessary organectomies for trauma, the metabolic catastrophes, the degenial iliobyepass. When I was a junior resident, you couldn't round in the ICU without having your swan numbers in your progress. No, because every patient had one. Today, we ask any of these juniors what a swan-gain catheter is. They're going to go straight to Dr. Google because they're probably not going to know. And sadly, it was only after about a hundred thousand patients had been treated before database data started to emerge that their mortality was worse. And then people realized that blowing up a balloon in the pulmonary artery just for the sake of monitoring wasn't such a great idea. But it was a fantastic device while we had it, so everybody got it. So how do we prevent from doing the same thing ourselves? How can we prevent having the historical lens look back at us and say some other thing? And there's two ways to approach that. A prospective observational study if the leap is disruptive. And a comparator said, we're going to postpone again. Maybe we'll make it longer. And a comparator study if you might have some echo points. So here's an example where we had a disruptive leap where we have done a randomized trial, 110 patients comparing epidural to PCA or patients who got a practice bar. That's the most painful thing that we do to patients is because of the pressure, not because of the cuts. So when you put an effect as far as like putting on braces and tightening to goal the day you put them on instead of doing it slowly over the course of years, really painful operation. So we had an average of about four or five days in the hospital. So we randomized epidural of PCA. And what we found was that looking at the pain scores, what we found was that there was no difference in length of state. There was no difference in any of the recovery parameters during the hospital stay, but looking at their pain scores. If you looked at the peak pain scores with the epidural, they weren't trending down. If you looked at the average pain scores, and this is the peak scores for the PCA, if you look at the average pain scores, the PCAs were trending down. Looking at this data, the surgeon said, well, we don't really need an epidural that the anesthesiologist said you need a better epidural. That's a totally fair interpretation, but if we can get a better transition plan in place that maybe the data could shift and we could have a truly superior regimen. So we started conducting a second randomized trial where we added a bunch of bells and whistles to both arms with the intention of trying to provide better outcomes for both groups. Halfway through that second study, we had randomized 33 and 32 patients in the PCA and epidural respectively. We started doing cryoablation. In that case, using the same side incisions that we do for the bar, we freeze the nerves with this cryoprobe, it goes to negative 60 for two minutes, and we freeze T-45 and 6, I'm sorry, T-45, 6 and 7. And then we place the bar. It was so dramatic. The first patient went home next morning. It was such a dramatic lead that we felt it was completely unethical to randomize between a potential five-day hospital stay and this potential. So we started a prospective observational study, but the criticism in some of the early cryo papers is that, well, now you've got a protocol. You've got a cryo and a protocol before you didn't have a protocol. So how do we know it's not the effect of the protocol? In this case, we had a really strict protocol in a second randomized trial that was mature. And so we could compare the cryo to these two other mature protocols. And what we found was that it is entirely dramatic that we couldn't move the length of stay beyond 4.3, 4.2 days. That was as good as we had done in any of our previous cohorts. And now they're routinely going home on post-up day one. So in that circumstance, we set up a prospective observational study like we probably could have been should have done with the lab. Colies setting up good databases to understand what's the rate of injury. How do we get injuries? How do we prevent injuries? And it was only 30 years later that people started to put that kind of data together. In this case, we're continuing on with the cryo, but we're following these patients very carefully. Where we're going to follow their neurologic recovery and then we're also going to follow how they do during the three years that we have them mainly looking for any persistent pain or neurologists. So in an example where it's a new therapy, but there could be the potential for equipoids. In our hospital, our radiologists had found a couple of retrospective papers that suggested there was an advantage to putting TPA into an intra-abdominal abscess. And so they started doing it. And so we said, well, we should compare that to our saline flushes that we've been doing all along. Our IRB said, now you're too early in your experience with the TPA. So you should do more. The counter argument was that if we do that, then it's not that the TPA is going to go away. The saline is going to go away. And the new expensive therapy is going to be the one that we use. And that's a pervasive problem in our field. That's a pervasive problem in medicine. You don't have to look any further than wound care with basic care wounds heal. If they didn't, none of us would be here. Large animals wouldn't exist. Yet the toolbox just keeps getting deeper and more expensive. And every time you get a wound care consult, they come up with this new triple and pregnant mesh that's got hybrid cells infused with vibranium. And they put that in the wounds until it becomes a scab. And at no point has that stuff been tested against standard therapy. So it's hard to just say, let's just jump in because we have it because we do that frequently. And so I was trying to minimize the slides here, but the IRB bought that argument and said, okay, let's carry on with the trial. But because the data is so clear that the TPA is better based on these retrospective studies, you can conduct this as a one tailed study design. And I didn't really buy that. But the difference was pretty big in the retrospective studies. It was a one day difference in length of stay. And so if you're calculating your sample size around a continuous variable and that variable is a one day difference, that's a pretty big difference. It's going to drive your sample size down to a pretty small number. We still went ahead with the two tailed test. And what we found was exactly a one day difference as the trial designed and predicted, but in the opposite direction. So it turns out that the TPA did improve drainage, but that meant it increased drainage, which meant the drain stayed longer, which meant the patient stayed longer. And so did that drain and chelt the patient. And the answer was no. It didn't improve the number of health care businesses, the days of IB antibiotics. Potentially worse than the risk of recurrent abscesses. And of course, the medication charges were incomparable. So in that case, we had a new therapy instead of accepting it as the new in the modern, we compared it to a standard. And we had a worse outcome with more expense. So now our standard remains. So in our center for prospective trials, we've managed to get a little work done over the past few years, but I'm just going to briefly touch on some of the lessons that I think we've taken away from that experience. And the first is limit the impact of the nasors. So when I used to say that limit the impact of the nasors, meaning there's never going to be a perfect study. We can't order our patients from a catalog so that they all have the exact same phenotype. But we can at least accept the conditions that we treat the patients in and do the best we can to control what we can, and then have as fair of a comparison as possible. And while there may never be the perfect study, it's usually worth venturing into the field and trying to ask the question. But then on a bigger scale, when you're talking about the nasors, the most important nasaer of all, how many people here have been in a situation that you had a grandiose idea, you had a really good idea for a project, a study, going and getting an advanced degree, learning another language, getting to a new level of fitness. And then you thought about it at some point later and you realized, had I started at the time I had that thought, I would already have done it by now. I'm going to raise two hands because I don't have 12. Because we've all been there dozens of times. In that line, what's the path to commitment? And it's when you accept the alternative outcome. And that isn't the end of the world. So we've had four randomized trials that some took a long time to get up, get running, took a long time to educate the NICU, took a long time to educate everybody to get them up and running, that we ended up stopping with no meaningful data. So we failed. So the nasaers were right in that circumstance. But there's not a great discredit in going down that path and discovering that for yourself. And sometimes, not sometimes. Every time you'll learn something from that process too. And probably the most important thing that I think that I've learned in this process is that I'm just beginning to learn. And it has to stay that way. Just like every day is the first day of the rest of your life. The learning venture has to continue to reset because if you don't remain skeptical of your own ideas, then you run that risk of where conviction replaces curiosity. And that's where growth stops. So we can't let that happen. So I'm going to quote Tommy Lee Jones from the movie Men in Black. And this is when he has just given Will Smith a glimpse into the alien life that's on planet Earth. He's having a hard time getting a hold of it, right? So he says, 1500 years ago, everybody knew the Earth was the center of the universe. 500 years ago, everybody knew the Earth was flat. 15 minutes ago, you knew that humans are alone on this planet. Just imagine what you'll know tomorrow. So I'll leave you with that. I've always finished these talks with this quote, which I think is generally true. As long as you're collecting the data. Thank you. Happy to take any questions. Great. Thank you, Sean. It was a tremendous talk and a critical body of work that's really changed a lot of what we do in pediatric surgery. And it's very rare. We also hear about trials actually changed the way we think and actually show that our biases were incorrect. And so to publish that and get a sense of actually that maybe perhaps the medical treatments are the most appropriate. I think are not something we typically think about in surgery. And I want to ask you about the vats because that's when I see a lot of our critical care folks in the audience and that tends to be an ongoing issue of discussion. But before I ask a few questions, I do have any questions from the audience. So the vats question is interesting. Did we have a question? Steve. First of all, when I congratulate you for a terrific talk and, you know, which way you started by saying that randomized trials and sort of impossible. In fact, before your work, most of us believe that you are one of a very, very small number of people in our community who have changed that perception and it is possible to do randomized clinical trials and so that's a massive contribution that you made. Thank you. In all of the trials that you presented, you showed the outcomes and you had heterogeneous patients that actually when they were randomized, you had similar samples. I want to go to the example of the fund, though, where you did sort of two stages. You stopped the curl dissection and then you stopped the forced mixes to the cura and you showed the difference outcomes. What you didn't show is if the operation worked because the reason the work was being done was for reflocks. So by having less hernias to just still fix the reflocks. That's an excellent question. For the sake of trying to maintain an overview, each one of these topics would become an entire balance in and of itself if we dove into it. So that's why I didn't go into all the secondary outcome variables. But yeah, so we took a look at the alty spells, the readmissions for pneumonia and the failure to thrive and the percentage shift between the dissection and the noticection and it was identical between the two groups. And that was as objective as we could get because we didn't have a pH studies or some of that. Well, those aren't useful in children anyway, but independently we just were looking at the clinical outcomes and those were the big three that we measured and we didn't see a difference. And then likewise with the stitch in those stitch, those numbers held up. And so interesting on that topic is that before our problem was a re-operation where you had to fix the hernium. That then sometimes putting in mesh and then it happens again and then that means going back and then putting in more mesh. And now you've got vagal nerve injury and delay gastric emptying and now you've got a GJ tube and the G tube is just pouring out green stuff all the time. We've all been there. Now when you have a failed fundo, we don't see herniations, but we do see an occasional re-operation. And instead of it going like this, it goes like this. And that is much less of a problem to deal with where you have a baby that was having the problems with the emphasis. And now starting to have the problems again and you show that the wraps on done, you go back and just restitch it. You don't have to do the big dissection and the esophageal dissection again. But the other thing is that we do this so frequently in babies because they're having such a hard time with their substrate. They're aspirating on it, they're failure to thrive. That if that starts to open up over the next couple of years, I'm not sure that's the worst thing in the world because then they're lower esophageals, fengar mechanisms, they're maturing. And so I think that we're just getting them through that phase and it becomes a more functional operation than going down the J feeding pathway, which is how a lot of centers try to handle that problem. Other questions for Dr. C. Feeder, Dr. Shamberger? I think in one of your final slides, you said the message was be skeptical of yourself. The corollary to that is be skeptical of standard practice. And I've always said that a surgeon's strength of his belief about a fact is usually inversely related to the strength of the evidence to support it. In the long the line of doing randomized trials, how are we going to get pediatric surgeons as a group to participate in multiple institutions, not single institutions with property group trials and and address some of the questions that we can't at single institutions for the number of patients who are too small. That is outstanding point. And the answer is like anything else that's that difficult, very slowly, how do you eat the whale? This one bite at a time. That would have been totally impossible at the time we were starting to do these trials. I was reaching out to centers and people that were in my cohort that I had finished fellowship with or gone through fellowship with and could get no traction at other places, couldn't get by and couldn't get other people to be a part of these protocols. Just the two center study was an achievement. Even then the data coordination was a nightmare. So it was a different environment because the attitude was that we couldn't do trials. And so the junior people who I was working with to try to get things to drive through couldn't get it going. Now that attitude has changed and now there's a different group of people that are in these places. And the whole idea of cooperation is changing. And so now these consortiums are developing and we have the Midwest Research Consortium. They've developed one in the West. And now the attitude is much more along the line of what can we do together. The PSRC, there's 30 centers in that. Still conducting a randomized trial through those groups is going to be really difficult. But right now we're leading a multi center international trial with 11 centers comparing antibiotics to operation for appendectomy. And I just think that would have been impossible 10 years ago. But the attitude shifting. So I think we're getting there. As a follow up Sean, how do we fund these types of multidisciplinary multi center trials in the future? That's always been the big challenge is you know what how do we sustain that in terms of kind of changing the culture that folks need to fund this as well as participate? That's also a culture change. So early in my experience, I had submitted several grants that I got the response we don't fund these kind of trials. You know the attitude was you're going to get paid for it anyway. So why do you need us? You're not you don't need rats. Don't need lab space. You're not using any beakers. No substrate. So I don't understand why you need any money for this. That attitude is changing. And PCORI I think helped shift that. But the attitude is certainly different now. And now it's the opposite where if you have a multi center study, that's going to give you the backbone of the strength to be able to get the funding. And so what what Pete and Kate did at Nationwide with with their non operative management study. Being in the consortium gave them the backbone to get the two and a half million dollars. They got the pilot funding from from PCORI. But to take it to the next level required that multi center ever in the consortium allowed that. Incredible work. I mean this is something that again we really didn't see ten or fifteen years ago. Other questions for Dr. C. Peter? So I had one about the VATS data. So I think we've bought in as an organization and institution that fiberland therapy is going to be our first attempt at therapy. And we have a clinical pathway that we've developed with our ICU colleagues as well as our IR colleagues. Where the big question comes in is well what happens after the TPA quote fails and how do we define failure? Many times into patient who's still fairly asymptomatic but has a pretty ugly chest X ray. And there's always a lot of consternation about the do well. Do we give additional fiberland therapy for how long and is there a bleeding risk? And I know there's probably a disconnect between us as pediatric surgeons and IR. When I was in Cincinnati as a fellow we did all the TPA. And so we controlled that here they do the TPA. And so I wanted to ask you in your experience since it seems like you've gone away from considering bats in most cases that your TPA regiment must be fairly successful. And so what do you do in terms of your TPA? What do you do in terms of retething and how you define patients who would be beneficial to undergo another course of TPA? So a second course of TPA would depend on identifying persistent, lock-related disease, particularly not in continuity with the drain that you have in place. And so it's the patient that is continuing to be ill, they're four or five days out, and then you get this imaging and find that. And they'll take out the first tube and put a pig tail in that new area and repeat the TPA. The majority of the time you find that the dominant part of the disease is in the perinkuma and they've got pulmonary necrosis, abscess or persistent pneumonia. And so just be patient. So it would be a rare circumstance that you would have somebody that has dominantly plural disease. The perinkuma looks okay. They're not eating, they're not off oxygen. You're plenty far out. You've done a couple rounds of TPA. That to me is the person who would qualify for bats now because you're going to be mechanically breeding that plural space and opening up a lung that can accept the new air and do something with it. But we just don't see that. And so that's why we've gone through years without a bats because when they're still sick, it's almost always perinkable disease. We're about to concern that bleeding risk and possible necrotizing pneumonia. And that's why we don't want to give that second course. Have you had any complications in terms of bleeding with patients who may have necrotizing pneumonia? No, we haven't. We hear that I see you folks. So this is something we got to push back. I heard that's their class. We haven't. But I would say being on the end of the instruments when we go and sometimes get into necrotic lung. I would take my chances with TPA hydrochloric acid, fluoric acid, you name it against that, that, that necrotic lung, then a mechanical debris, and of that necrotic lung because that's typically our staff will love to give your protocols so I can share it with our, our folks here. Any other questions for Dr. St. Peter? Dr. Z. Sean really outstanding talk. Thank you. Just wondering, we all believe in randomization as a gold standard, good study design, very often in pediatric surgery, even with aquapoys and designing a clinical trial, a randomized clinical trial or control trial. The trials are small. And as most of us know, the likelihood of some imbalances, even with randomization, are very likely that there could be some covariate imbalances. How do you sort of deal with that dilemma in still preserve high quality decision making based on the evidence? That's an outstanding question, and I'm glad you brought it up. I almost put in an additional slide because I thought that question might come up because we've taken a look at that. And we have traditionally had this study design where you have an anticipated delta and anticipated difference going in. So you're going to estimate in the example I gave with the TPA and saline one day difference. You have a known standard deviation based on the retrospective data. So you can use that to calculate how many patients would you need to establish a p-value of .05 if that difference really does exist. Now this is all guesswork, right? You're going into it. You haven't done that yet. If it all works out, you find that number. If it doesn't, that doesn't mean the difference doesn't exist. It just means you didn't find it. So you didn't get to enough patients to get to that .05. I became less concerned with that when I noticed a phenomenon over our first several studies, which was, and I think this could be a different trial design in the future, which is following what your delta is until it stabilizes and establishing some statistical parameters around that. Because in the case, let's say we randomized 200 patients and we didn't find a difference. How many patients would we need to find a difference? And if the difference in that irrigation versus no irrigation was 5%, that answer would be maybe a couple thousand. Not unrealistic then, but that's a meaningful difference because more than a couple thousand people get treated for that problem. It's a common problem. If it's several hundred thousand, then you start to get into the point where if you're going to, you can't prove equivalence, but the concept of equivalence to me is that if you're going to require more patients and an individual practitioner is going to see in their career to see a difference, then that's not a difference that's worth investigating any further or trying to prove that difference. But with the maturation of the effect size, what we saw in our studies was that with the exception of the VATS, which was our smallest study, every other study met its final effect size well before we terminated the enrollment. So in the example of single incision versus three-port laparoscopic epinectomy, the difference in OR time was five minutes. It was 4.8 minutes on average. The difference at 150 patients was five minutes. P value was insignificant. At 360 patients, the difference was five minutes. P value was 0.008. So it became highly significant, but the effect size didn't change. And so I think we can reframe the way we look at these in terms of, did you have a mature effect size as importantly as what was the ultimate difference in the end? Because frankly, statistically statistical significance at the end doesn't mean anything. It's the clinical relevance of the effect size at the end. So that five minutes became highly statistically significant, but what does five minutes mean? To me, that might be important to somebody else. That could be background noise because it takes a longer than that to respond to the page to get to the OR on the first place. So is that five minutes important? That's up to you. But it's real. That might be 750 dollars of OR time. That's what anesthesia folks are thinking right. Russy. I'm Russ, everyone, the softenerway treatment center here. And the truth is, the truth is I don't really care about a day of hospitalization or saving 750 bucks or, I mean, that's where the insurance company is worried about. What I care about is fixing kids. And so I've invented and our team has developed a dozen or more new procedures, which we routinely do and are quite good at. We think have awesome outcomes, but it's not for every pediatric surgeon. That's not for every anesthesiologist. These are highly specialized evaluations, treatments, and follow-throughs in the ICU. Disrupts the entire chain of medical care. And we're struggling with it. We struggle every day, but we're really good at it. How do we prove to you that we're doing a good job? How do we randomize or how do I develop Equipoi when I see a kid who's, you know, doesn't have an esophagus and then he has an esophagus or he's dying of a recurrent fish, and now he doesn't have a fish on his breathing or he's got trachea bronchomalacea, which doesn't exist in most people's minds. But we see these kids every single day, and they're literally dying with getting bronchectasis and we fix them and they go unhealthy. How do I convince somebody like you and I want to convince myself that we're doing the right thing, you know, some randomized prospective, caulkern approved strategy? One of the big parameters in randomized studies is generalizability. And a lot of the examples that I was talking about are generalizable because they're common operations that are being done in hospitals all over the world. And those high fidelity outcomes are important because that's what's going to separate the direction that we take with these common things. And we trust that you're going to be able to do the appendectomy, the question, and you're going to be able to save the patient from their appendicitis. The question is whether the irrigation helps. What you're talking about is no longer generalizable because there's just a handful of centers or people in the world that are doing these things. So the position that you're in, the position that Mark Levitts in, the position that some of the fetal surgeons are in, are in the same vote. And you can't possibly randomized. You can't take these complex patients. Each one is its own unique story in and of itself. But what I would encourage is developing prospective observational studies with protocols that have multiple endpoints of follow-ups so that you can track yourself. So that at each point, when you make a change and you start doing tracheopexia and you start to posterior aortic pexia like you're doing, then you can see if there's a difference. And then you may be able to even pick patterns to where, okay, here's a patient that was on this pathway. And we're going to do something different this time, but we do have these endpoints that we follow every time. So we're going to be able to tell if maybe this kid gets off the trache a little faster. Yeah, the difference between careful clinical observation, which was advocated by Kaumann way back in the day, you know, in terms of internal benchmarking versus something that it's not going to be possible to benchmark externally. And that's always the challenge. So, that's a great question. So, I'm going to ask you a question. I'm going to ask you a question. I'm going to ask you a question. I'm going to ask you a question. So, Sean, what you just said about generalized ability is really interesting. And whether it's a small RCT or a larger or possibly a multi-center randomized controlled trial, the notion of being able to provide high quality evidence and being able to validate. That information moving forward into surgical decision making, possibly generalize ability. What are your recommendations regarding validation of your research study? The only thing you can do, which is other centers try to. And so, that's what's happened with the fibronolysis and. Dozens of centers have moved to fibronolysis and they've had the same results. If we start to hear that people are seeing hemorrhage from the TPA, then there could potentially be a problem. And then we have to figure out why does it work if these other hospitals and it doesn't. But that validation, the thing about a lot of these studies is that as soon as it finished, I was actually less concerned about pushing this out to the community. And more concerned about answering the question for ourselves. And so, in every example, whichever pathway one, we were done. We were going down pathway that one and we were doing the next trial. And then going down that pathway. If that can be helpful to other people, then that's great. But now I'm starting to realize that that's. That was probably a little too inclusive of a view. And I do need to be more concerned about external validation as we roll these things out. Because it's going to be a lot more important to help 100,000 and to help the next 100. We do a time for one final question. Okay, looks like no other question. So again, Sean, thank you for making the trek to Boston and the amount of work you've done to really kind of reshape the way we think about high quality events in our field has really been unparalleled. So it's really been a pleasure to have you as our second annual lights in the business professor. Thank you. Thank you. Thank you.