Dr. Douglas C. Barnhart - Improving Quality in Children's Surgery Through Voluntary Verification: The American College of Surgeons Children's Surgery Verification Quality Improvement Project Experience

Yeah, hey, good morning, everyone. Uh, so why don't we go ahead and get started since we're at the top of the hour. Um, so, again, welcome, uh, to now what is gonna be our 6th, uh, annual Weitzman, a visiting professor lecture. And, uh, traditionally, this has actually been a lecture that has highlighted individuals who have made a, a huge impact in pediatric surgical care, uh, specifically through their health and research or health services research efforts. And, uh, this year, it's actually a little bit of a twist on that theme and that we're celebrating. The leadership of an individual who has made a huge impact in pediatric surgical care, but through a different mechanism, and that is really in the development and implementation of practice standards to optimize the care that we give our surgical patients. And so perhaps even more important and more impactful than any of our previous. Uh, um, lectures that we've had. And so a little bit of background, uh, so Doug is a general pediatric surgeon at Primary Children's Hospital. He also serves as the surgeon in chief of Intermountain Children's Health Care, which is actually a, a pretty substantial role. He's not only the surgeon in chief for Primary Children's Hospital. Uh, which many of you know is a very large, uh, freestanding children's hospital, uh, but also he oversees the surgical services and infrastructure of the entire Intermountain healthcare system, you know, which is fairly, fairly extensive. Um, and he's also a professor, uh, at the University of Utah School of Medicine. Um, as far as other background, as you might imagine, Doug's CV, uh, is quite thick, uh, and it took me about a half hour to go through it, and it's full of, uh, as you might expect, accolades and accomplishments and, uh, pretty substantial leadership roles. Uh, but what I want to focus on today, uh, for the purpose of brevity and for this, uh, lecture is, is really 15+ years of incredible leadership in the American College of Surgeons. Um, and this is both, uh, with the Children's surgery, uh, programs in terms of NEQWIP, which is our benchmarking. Uh, tool and also the Children's Surgery Verification Program, uh, which verifies centers such as ours as level one status, and Doug is actually the national head honcho of this program and so it has arguably the best 30,000 ft view of the evolution of this very important program over time as well as the huge impact it's made and more importantly, uh, what, what we're going to look at in the future in terms of systems of care verification and other things such as that. Um, and so we're very eager, um, to hear what Doug has to say. I'm also gonna say from a personal note that Doug has been a tremendous friend of mine over the past 15+ years and also an incredible role model, and I seem to follow in his footsteps everywhere within the college. I'm not sure, uh, how that's happened, but, uh, he's been really an incredible mentor in that regard. Uh, before I turn it over to Doug, I do want to acknowledge the incredible generosity of, uh, Stuart and Jane Weitzman. Um, as many of you know, they are Uh, just, uh, fabulous benefactors to, uh, our institution and our department. They've supported a lot of my research efforts over the past several years, and without their support, we wouldn't have this lecture. Doug wouldn't be here today, and we certainly wouldn't have had that great Capitol Grill steak last night in Doug's honor. So, we do really appreciate, um, their ongoing support. And so with that, I'll, uh, turn it over to Doug to give us a little background. Good morning. Uh, thank you for that overly generous introduction. I, I would reframe it and say that I, I've been fortunate to be, uh, inherit a vision from, from other leaders and have, uh, been able to serve as the custodian of that, uh, vision for the, for the last, uh, decade, kind of serving as the bridge between the, the people that Formed the vision who I'll, I'll talk about uh during this, this time together. And, and the, the thought leaders that created the evidence for that, which is largely based, based here at Boston Children's. So, um, so I'm very, very grateful to be here. Um, it's a real deep pleasure to talk about the, the programs of the American College of Surgeons and what we're trying to do to improve the quality of children's surgical care. It's special because uh to be here because the program wouldn't exist if it wasn't for the evidence that came out of uh Sean's work, and he's really been instrumental in, in leading the intellectual and research efforts that are underpinning of this program. And I also just want to acknowledge Connie Hoke, who has been really our anesthesia champion nationally. And that, again, the program wouldn't exist without Connie and And Dave Mooney served as the, one of our early site visitors when we were trying to learn to do site visits and reached out to the uh pediatric trauma people to help us figure out how to do this. David leaned in and helped us get started with that. So we're very grateful for your institution for all that you've done to help with this program. I, I have nothing to disclose except that I'm incredibly biased about the fact that voluntary participation in quality programs, uh, does, does change children's care. I'd like to set the context for a time this morning with a, with a clinical example that is um the sort of the case that served as the motivation for all of us that are a part of this program. I learned about this case when, uh, one of the young surgeons I helped train called uh seeking advice that the surgeon had been asked to see a baby that had been born in a referring hospital with esophageal re and trachechoesophageal fistula. I think this is Probably for, uh, most pediatric surgeons, one of their favorite cases, partly because it's a lethal anomaly that we can repair reliably with good outcomes. And because it exemplifies really what we do as a, a pediatric surgery team with the anesthesiologist. As we all know, you can't do one of these cases if the anesthesiologist can't tolerate having the SATs below 90, and the surgeon has to be willing to stop the moment an experienced anesthesiologist tells you to stop. So, So I think it really does exemplify the balancing act we try to do in children's surgery. Um, unfortunately, this baby was born in a referring hospital with, without a pediatric anesthesiologist, and the instruction given to that young pediatric surgeon was that they were to repair this child's esophageal atresia at that hospital, and he called me to ask me what he should do. Um, and, uh, my advice to him was that he should refer the baby to the children's hospital and start looking for another job. And, uh, and that's, that's what he did. Um, and now practice in a different city. And I, and although this case is extreme, I think we can all relate to cases where there are kids that have received care and settings that are suboptimal for their needs. Well, we all had experienced those cases, um, Keith Oldham, uh, was the one that formulated a vision that we should do something different about that. Um, and for those of you that know Keith very well, you know that he is principled and systematic in how he approaches things and does things in a stepwise fashion. Um, and just parenthetically, he's, he's also been a wonderful mentor to, to many of us who haven't ever been on his faculty, never been as trainees, but that we've just met professionally, and he's adopted us as his mentees. So, I think there are many of us that owe our careers to, to Keith and his principal divisions, so. Having had a long interest in the quality of surgical care for children, Keith served as the editor for the seminars in pediatric Surgery edition in 2008 that was focused on surgical quality. And in that introduction, he, he wrote an editorial that uh pointed out that as a discipline, we lacked reliable data to be able to identify a quality of care that we provide children. He also pointed out That there was apparent significant variation in the types of quality of care that were provided and that, that this served as a, a call for us to have a better way to measure the quality of care we provide for children with common surgical problems. As I mentioned, if you know Keith, he doesn't propose a problem without having some ten-step solution that he's already got written down. Um, and, and this is an example of that, and, and that, that vision would ultimately mature into the American College of Surgeons Children's Children's Surgery Programs. These programs are the substance of what we're gonna talk about this morning. And that, there are two parts to it. The Children's Surgery Verification Quality Improvement Program verifies that hospitals have adequate resources, which includes both personnel and robust performance improvement programs to optimize the care of children. The National Surgery Quality Improvement Program Pediatric provides the risk-adjusted data to guide the hospitals in these quality improvement efforts. These programs have evolved and developed over the last 17 years since that editorial was written. This uh timeline uh outlines uh some of the landmarks in that. And, um, although that feels like a long time when you hear it, it expresses 17 years to, uh, move the agenda nationally and how we view, how we assess children's hospitals and what, what the expectations are, and even the idea. That an organization can come in and create expectations on what care should be in a hospital. I think it actually is an incredibly quick process for that change to take place. That same edition of seminars in pediatric surgery also included the descriptive, description of the pediatric version of the National Surgery Quality Improvement Program. I, I suspect most of you are likely familiar with this quote, but for those of you that aren't, the program began actually in the Veterans Administration Hospital based here out of Boston. Uh, it was based on a congressional mandate to have risk-adjusted outcome data so they could assess the quality of surgical care that the veterans were receiving. Um, and that demonstrated that they're able to, uh, identify, uh, uh, risk-adjusted outcomes and characterize them so that they could identify differential care being performed at centers. And from the VA it was then extended into adult hospitals under the uh tutelage of the American College of Surgeons. In, in, in that context, in both academic and uh community hospitals, it was demonstrated to be successful in differentiating outcomes in, in adult patients. And a group of, given that, a group of surgeons from the American Pediatric Surgery Association proposed creating a similar problem program for children's surgery. Uh, in this editorial that was written in 2008, those authors were prescient in, uh, recognizing some of the challenges that we would deal with as the, as the program came into existence. And some of those were the heterogeneity cases and the low frequency of children's operation, which was very different than the adult program. They also Identify that we'd have a challenge with uh with identifying uh variable performance in hospitals because the, fortunately, the overall low mortality rate in children surgery outside of cardiac surgery. And they also recognize that important outcomes extended beyond 30 days in children. And in fact, those were the issues that we have been dealing with in the program over the last 15 years. Initial description of the program included in that uh had the fundamentals which have really remained unchanged to date. The Nisquip sample is a multidisciplinary uh sample of surgical cases. It's derived using an eight-day sampling cycle that uses specific CPTs as the sampling guidelines. The data is abstracted by a clinical reviewer with well-defined risk and outcome, uh, variables defined. And every case has a mandatory 30-day close follow-up so that we're confident that we know what the outcomes are. And the American College of Surgeons, since it's time that it began with the adult program has really provided data quality assurance and uh advanced statistical methods to help us uh risk adjust and understand what's going on in these, in each hospital. And these basic concepts have remained unchanged for the life of the program. The program that they proposed was conducted in 2008 to 2009 at For Children's Hospital, and this was really a proof of concept in which they uh tracked 7200 patients. Um, that initial pilot program can confirm several important assumptions. One, it demonstrated that occurrence rates were variable between the specialties and concentrated into certain inpatient procedures. Additionally, it confirmed the ability to identify patient-specific risk factors that would predict complications. And moreover, the type, the type of complications were relatively specially specific. And that would serve as the basis for the further development of the program, including the concentration of our cases and the, those that are highest risk of complications. The next year, 29 centers were enrolled in the beta phase, and this is actually uh where I first encountered the program. They had a meeting at A, so this is probably the most helpful thing for the residents. If there's a meeting proposed on Sunday morning at 6 o'clock, go to it because there'll be plenty of opportunities for you to volunteer for things because not many people will, will show up. And that, that was really how I got involved with NISWIP cause I'm 11 of the people that showed up at the 6 o'clock meeting and put, put my name on the sign-up sheet. Um. But, um, that initial, uh, um, that, that, with those 29, we then uh began a, a beta phase, um, which included 14 free-standing children's hospitals and 15, children's hospitals within hospitals. The, the beta phase was conducted over one year and, and we enrolled over 37,000 patients in that uh one-year period. And as anticipated, the overall mortality rate was quite low. It's 0.25%, but there was an 8% morbidity rate, which allowed us to distinguish differences between hospitals. Um This is a figure from that initial report. And uh for people that are interested in uh surgical quality improvement, this was a really exciting graph for us to see the first time it rolled out. Because uh what it demonstrated was that We could demonstrate uh differential outcomes between children's hospitals in a risk-adjusted fashion. And even in that initial sample of early adopters, um, we were able to see that there were high and low outliers, uh, for, for morbidity. Um, and that served as an inspiration that, that the program was worth continuing and that we'd be actually to demonstrate to hospitals that they had opportunities to improve. Subsequently, the program was open for widespread enrollment, uh, the next year and there was, as this graph shows, there was rapid adoption by, by hospitals, both free-standing children's hospitals and children's hospitals within hospitals, and even some general hospitals that provided care for children. This graph shows both the number of centers and the cases enrolled with each semiannual report. And there are now over 150 hospitals that participate in the program with more than 150,000 cases enrolled per year. Uh, Nisquip pediatrics, therefore represents the largest clinical data source for the assessment of quality of children's surgical care. And given the large number of clinical risk factors that are collected in the, in the, uh, uh, multitude of outcomes we follow, the data set provides an opportunity to identify meaningful outcome differences between hospitals as well as serving as a research database that allows us to identify important risk factors for, for children's surgery. Unfortunately, we did, we did encounter problems early on. Despite the large number of cases and complex risk adjustment, there were significant challenges. First, the mortality rates were thankfully very low. And therefore, we're often unable to distinguish any difference between hospitals in terms of mortality. Additionally, when you dug into the, the complications, the majority of them were infectious. And given those two facts, the program was at risk of being the most expensive and most detailed uh registry of surgical site infections in the world. Um, and we recognized that this was an existential threat to the program. And that this is when Sean and I were on the measurement and evaluation committee together trying to figure out what we should do with this program. Um, and at that point, we, uh, had the insight that, that there was probably very significant differences in resource utilization between, given the variable practice patterns. And that if we looked at those resource utilization differences and could demonstrate that there wasn't a different difference in outcome, you could make the argument that uh more economical resource utilization should be a measure of quality of care. This was new to the American College of Surgeons. They have never used resource utilization as an outcome variable back in, uh, back in 2012. And it, it took quite a bit of a sales pitch to get that included cause it was really a, a shift in mindset for what the program would be like. Um, fortunately, they, they did adopt that. And, uh, beginning initially with complicated appendicitis, we're able to demonstrate that there was in fact significant, uh, variation. Uh, in resource utilization, particularly around PICC lines, CT scans, and parental nutrition. And these, in some centers, these were used routinely being uh started on children as soon as they were diagnosed with perforated appendicitis. And in other centers, there was virtually no use of PICC lines or, or TPN. And when you look for other outcomes such as abscess or uh other complications, there was no difference between them, which made a compelling argument that the centers that routinely use these expensive technologies that they should eliminate them. And simply by feeding that back to the centers within a relatively short amount of time, the routine use of PPICC lines at TPN disappeared. Um, and essentially was eliminated from all the participating children's hospitals. And that ability to rapidly change the national practice. Uh, for a common problem, the problem really demonstrate the value of the program and, and, and honestly, I think saved pediatric Nisquip. Um, and that model will then served as a model for resource utilization and, and all of our other procedures and really served as the way going forward with Nisquip. As the program continued to mature, we sought other ways to increase the value of the data provided to the institutions, and an important part of this was continuing to limit the procedures. sample to concentrate on those that had the highest risk complications or that had significant practice variation between centers. This is also a bit of controversy because it meant that you weren't uh sampling broadly from all the specialties and, and the question of whether you could generally assess the overall quality in the, in the hospital was uh uh put into question. Um, and Sean really led us in efforts to try to understand this concept of, uh, overall complication burden, which included both the risk of complication incidents for each of the procedures as well as the number of procedures being done in the country. So that we could affect the largest number of children, um, uh, most quickly. And that really served as a fundamental principle of how we went about case selection for the sampling. Additionally, at that phase, we began to more actively engage experts from each of the surgical specialties to identify risk factors that were specific to their surgical diseases or procedures. And these were added as modules to the procedures and, as well as adding procedure-specific outcomes. In this movement from a generalized approach to more specific disease approach, it continued to offer more actionable data for centers. And finally, as a data element for the verification program, we added stewardship elements for both opioids and antibiotics into our, into the registry. And finally, we added the timeliness of diagnosis and treatment for urgent conditions as an assessment of quality with the intention that that would provide centers into a window about how their center may not be as robust as they had hoped in the areas of vulnerability for, for children that have emergent conditions. And we're particularly focused on topics like button batteries, mid-gut volvulus, testicular torsion, ovarian torsion, and post-tonsillectomy hemorrhage because they required meaningful interaction between The emergency department physicians, radiologists, and surgeons, and anesthesia to make, to get these kids cared for in a timely fashion. We felt that they were an important way to get a quick biopsy of actually how well your center functions. With these changes and the emphasis of the program, we've been really able to give centers back significantly more actionable data. Uh, semiannual reports return the center's risk-adjusted outcomes covering multiple areas of the program. These include quality reports of overall morbidity and mortality. And, but also, we're able to split it into uh patient groups, including both neonates and pediatrics as separate groups, and then also break it down by specialty and anatomic procedure areas. Well, we're also able to provide, um, procedure-specific, uh, targeted, uh, outcomes to the surgical specialties that have helped identify those. Uh, and with those, we've added the concept of balance measures so that we can, uh, make sure that we're not having an untoward outcome, uh, as a result of, uh, some effort to reduce the, the primary outcome that we're looking at. And I'll, I'll show you one of those in just a moment. And this uh demons this slide shows the things that we're currently uh have as procedure targets, um, uh, appendectomy, spinal fusion, reflux, uh, procedures in neurology, cleft palate and lip, uh, esophageal cases, and stoma cases and general surgery. This, the procedure-specific data, I think are particularly useful for centers as they really provide clearly actionable data. This, this is an example of the uh appendicitis report card that my hospital uh just received. Um, as you can see by this report, we're pretty efficient in, in, uh, providing care for appendicitis and that we have short lengths of stay, uh, for children with complicated appendicitis. And with that, we're able to maintain a Low complication rate, including with post-operative abscesses. Uh, unfortunately, despite that, we have a higher than expected rate of return visits and readmissions. Um, and this is despite the fact that they, these kids don't have abscesses. And this has been a problem for us in the past. Um, that we'd address through improved parent education and, and instructing parents to call our clinic rather than, rather than come to the emergency department. Um, this is common in quality improvement projects. Unfortunately, once we took our eyes off of that, we backslid into our old patterns and we've, uh, really got to revamp those educational efforts, um, to, to stop those readmissions. And I, I think this is one of those examples of how this, uh, procedure-specific focus can really Uh, help a hospital, uh, identify areas to work on. Um, additionally, there are reports bring back raw data that can help you understand your practice pattern compared to others. This one, is, uh, uh, about resource utilization and appendicitis and, uh, provides information about imaging stewardship in terms of the use of CT and MRI as well as, uh, operating room efficiency, both in terms of, from, uh, access to the operating room from the time of presentation to the time of operation as well as the operative time. And then some of those things that were historically important, which were PICC lines and uh parental nutrition. As you, and you can see on this, nationally, the use of those is, is really very, very small, which is a dramatic change from when we first began the program. I've spoken before about balance measures. This is uh one of the, uh, key concepts we try to give in these procedure-specific ones. This, this is just an example of the, the, uh, balance measures and vesicular urethral reflux, really looking at the medium length of stay versus the bounce back to the emergency department. Um, and kind of addressing that question of do you get your length of stay by sending it down, by sending kids home and having them have problems and come back. And as you can see, um, the Our, our center, our urologists do most of these as outpatients and still have a low rate of return to the emergency department. And it's typical with these balance measure plots, all of them look like a shotgun, the, the blast, and that there's really very little correlation to people's uh Early discharges and ER bounce backs and, and other things like that. So it really, I think has more to do with the other factors like preparation and how you manage problems rather than this, this single outcome, which we hope then motivates the centers that have these longer, medium length of stays to feel safe to moving towards uh the shorter length of stays, which I, I think has really been One of our real leverage points with Nisquip to affect change is that these sorts of data where you can show it to clinicians, we all come at these with these biases of why we're afraid to make that change and where you can understand that there are other children's hospitals that are doing this, and it's not necessarily uh meaning that all their kids are coming back to the ER. It, it makes you wanna understand how they're able to do these, these outpatient procedures and move towards that. And we really try to provide these as the Uh, argument to support change for, for all of the centers that are, uh, involved in the program. Uh, in an effort to improve stewardship, centers have provided data about appropriateness of both perioperative antibiotic and, um, uh, which include use, which includes both the spectrum and the duration of therapy. Additionally, opioid prescription at the time of discharge is recorded. Um, and this is provided back both at the hospital level and at the specialty specific level. As, as you can see, this, if you look at these numbers closely, this is an area our hospital's got, got some work to do. We, we send more kids home with opioids than, than our peer groups. Um, and this is currently one of the focuses in the verification program that we're asking participating centers to work on. Uh, which speaks to one of the basic concepts that we've developed as, uh, as a children's surgery program at the college is that we basically view that NSW pediatrics should be the data support for the verification program and provide the data that centers need to, to affect change. And uh when we take those two together, that allows us as a children's surgery program really to determine priorities for uh children's surgical care nationally. Uh, that by adding elements into the data set so that centers know where they are. And then on the flip side of that on the verification program requiring centers to look at it and use their data and make programs about it, we're actually able to affect change about things that are, are important elements for care. So it begs the question, how do we know any of this is working? Um, and I, I think the indirect ways we know that are that we see a high level of engagement by the, by the hospitals that are, are participating. Um, and this is seen both in participation in national meetings, but more importantly, what we see is centers that are outliers in a report will move down towards being either as expected or low outliers, uh, through their, through their specific performance improvement efforts. And we don't see centers, uh, Stay high outliers for long periods of time. It usually takes them several cycles to get out. So we don't, don't think that it's just statistical error, but with, with uh time, we see them move out of that status. And oftentimes, at our annual conferences, they'll come and uh present their performance improvement projects that help them change that status. Uh, as I mentioned in the, with the example of, uh, the PICC lines at TPN and things like that, we also see the significant change in the patterns of care in the overall cohort, and we actually see, uh, some of the outcomes shifting, uh, favorably. So it gets, it gets harder and harder to be a, a high performer because what your previous performance no longer makes you as a high performer because the other centers have improved as well. Um, and, and finally, uh, we think the program works because we have the centers really engaged. When we roll out, uh, pilot programs of variables we wanna test, we always have centers that are eager to take those on and, and help us develop new modules. So, all, all that sounds well and good on the data side, but um I think it's fair to ask, well, how did, how did that really helped this baby in this referring hospital with the esophageal atresia. Um, and, and that does bring us to the The, the second half of the American College of Surgeons program, which is the verification program. The vision for that program is, is really quite simple. Um, it's that every child in need of surgical care in North America today will receive that care in an environment with resources optimal for the children's, for that child's individual needs. Well, it seems like that vision is pretty high-minded and should be non-controversial. Um, well, we rolled this out, there, there was significant controversy around it, uh, both in the surgical world and in, uh, some of, some of the specialties that help us care for, uh, surgical patients. And Um, those, those, uh, questions centered around one, whether this was simply a land grab by the big institutions to, to force the regionalization of care. Uh, there's also the questions of whether or not we actually had any evidence that Being in a pediatric center actually improved care. Um, and, and finally, there was the, I think, the legitimate concern of whether this would result in decreased access, uh, for some children to appropriate care. Um, as, as we formulated the program, we had several foundational, uh, principles that we kept coming back to. Um, the first is that the program is intended to be patient-centered. Um, the most important question was always what is, what is best for the child and the family. And the impact on the caregivers of the hospital economics was considered to be secondary. Um, the, the second strategic decision we made was that we wanted the program to be inclusive. Um, Strategically, we felt that the way to improve care was to actually engage as many hospitals as possible. And the way to do that is to actually be in their hospital with the program. And therefore, we took an iterative approach with beginning to set a standard with the intention to uh giving them achievable standards and then gradually raising the bar. Um, The third part was that we had a deliberate focus on quality improvement as an emphasis. Um, we recognized that every center is gonna have adverse outcomes and, uh, serious safety events. Um, and really, the emphasis needed to be on whether the center could identify those events and whether they could create action plans with loop closure to prevent this from occurring again. And the final uh value we recognized was the importance of this being multidisciplinary and not just being surgical. Um, and so this all began with, uh, Doctor Oldham, uh, convening a task force in 2012. Um, and that ad hoc task force, um, included leaders from children's surgery, pediatric anesthesia, medical specialties, as well as the being supported by organizations including the American College of Surgeons and the Children's Hospital Association. And the, the work product of the, that task force is uh shown here was a publication that proposed a definition specifically around what would be optimal resources for caring for children with surgical problems. Anna proposed a uh a verification program like, like was already being done for trauma centers. I think the most important fundamental change in how we view the quality of care um is that we shifted what we meant by surgical quality. I think historically, we as surgeons felt that the way to improve the quality of surgical care was just to have more surgeons stare at things with more magnification and be more and more intense about it. And then we uh suddenly backed up our vision and realized there was actually this person peering over the ether screen. That had the same goal that we did. Um, and surprisingly, there were really not many forums where we actually talked together. There were parallel quality efforts in anesthesia running and, and to the ones in surgery, but there were not many forums where we actually interacted over the, uh, uh, over the perioperative care of children. And, uh, and that has actually been, I think, the real strength of this program is that it's actually A joint program, uh, that's, uh, done with pediatric surgeons, pediatric anesthesiologists, and, and pediatric nurses. So it's, I think it's really all of those in the, uh, uh, perioperative space have been involved. And that, that I think has really been the, the force to, to make care better for children. Uh, I think, and that really comes to one of the things I learned as a, as, as a surgeon in this from our anesthesia colleagues, which um that, and this kinda is obvious, but safe surgery is dependent upon safe anesthesia. And children's anesthesia for small, sick children is dependent upon pediatric anesthesia expertise. And there's, there, there's a literature out there that, that demonstrates this. Um, and, and applying that literature as one of our standards was one of the most, one of the most controversial things we did in, in creating the standards, uh, was requiring that children under 2 have a pediatric anesthesiologist as their, as their anesthesiologist, and that kids, uh, uh, under 5 with an ASA greater than 3 were supposed to have a, a pediatric anesthesiologist as well. Um, and I'm convinced that, that applying that standards actually saved kids, kids' lives in the United States. That was actually one of our, uh, uh, not only one of our greatest areas of impact, but it's also one of the heaviest lifts for, for a lot of hospitals. Um, and that, for those of us who live in a children's hospital, this comes as, uh, a potential surprise. But there are a lot of hospitals and hospitals where there will be pediatric anesthesiologists, but they won't be directly giving the care for the children. Just because of scheduling reasons, it works out better to put adults in some rooms and And, and mix them with kids and whoever's the anesthesiologist does it, even though you have pediatric anesthesiologists on staff. So, so many of the centers we visited had to go through this whole restructuring of how they schedule to actually let people care for the uh patients that they've been most trained to care for. Um, As the standards were being socialized back in 2012 and 2013, there was also data coming out uh that showed that a large number of children's operations were being performed outside of children's hospitals. This is a paper by Doctor Ziegler that showed that 40% of inpatient operations were being done in general hospitals. And, uh, given this, we, we argued in that editorial that um it was critical to assure that those hospitals were also able to meet the needs of children. Uh, because it simply was not practical, uh, to shift all those operations to children's hospitals even if people wish to. Um, so it seemed like that decision to make this an inclusive program was the correct one. The recommendations from the task force were adopted by the American College of Surgeons to establish the verification Program in 2015. Uh, the program identified three levels of children's surgery centers based upon resources available in the scope of care provided. An underlying principle was that the quality of care at all the centers should be equal. So specifically, a level 3 center was expected to provide. Excellent care, just like a level one. It's just that they were managing children that were physiologically healthier and had uh less complicated problems. Each level is defined by specific requirements of the team, including the surgeons, anesthesiologists, medical specialists, nurses, and pharmacists, and other uh care members. And the foundation of the program was creating a multidisciplinary uh surgical performance improvement and patient safety program that really serves as the backbone of what we're trying to do. And for almost all centers, this required a fundamental change in how we approach surgical performance improvement. I recall that when uh Bob Salwy wrote the who's a pediatric surgeon from Seattle, wrote the, wrote the chapter on that and brought it forward as, as his proposal for what it should look like. He said, he's like, I, I'm, I'm bringing this forward knowing that our hospital won't meet this at the moment. And so I think that was the reality that all of us had to go back and change how we were doing things based on the standards we're writing. Um, a little more about that PIPPS team. Um, as I mentioned, it is, uh, the thing that's really unique about it, again, just like we finally talked to our anesthesiologist, the, the PIPS program is, is in many places, the first time that we regularly convene all the people that are involved in care of a, a surgical patient. And it's not just all of the surgical specialists and anesthesiologists, but also The critical care physicians, the neonatologists, emergency medicine physicians, and radiology as well as OR leadership. And The, the, the thing for us that was great about this, and we hear this in many, many centers, it suddenly gives you a, a venue in which you can solve problems. Um, where before, it was, uh, it was so siloed that it, it was, it really took a major event to, to move people out of the silos to try to create a solution. Um, the, the, the PIPS team is charged with a whole series of surveillance of outcomes and processes. They also are charged with reviewing every uh perioperative mortality for children. So again, in many hospitals, this was the first time that there was a mortality review of, of every child that died, uh, after procedure, after medical therapy. And in many hospitals, it's then generalized from the surgical patients to the medical patients. Um, it really would challenge them to come up with loop closure for, for all the serious events and with a real emphasis on shared learning, not only within their hospital, but with other centers. Just briefly, for those of you that aren't uh uh familiar with the verification program, I just wanted to give you a little bit of the basics between the levels. And this shows the scope of services at different levels. So level one is, uh, centers are able to provide the most comprehensive care in a multidisciplinary fashion for the most complex patients. Level two centers are capable of carrying of children with common problems. Uh, they're typically managed by a single specialty. And these centers always have a general, uh, pediatric general thoracic surgeon and pediatric anesthesiologist, but they have variable, uh, other pediatric specialists based upon their scope of practice. And level 3 centers provide care for children with straightforward problems. The level of uh physician expertise varies between the levels of centers and, uh, the, this just outlines this again, and the level one centers typically have all of the specialists. Level two is variable, and level 3s are typically staffed by a generalists with uh some pediatric expertise and ongoing education and practice. Similarly, the ICU requirements uh vary between, between the centers. Um, with, with this structure defined, we released the standards in 2015, anticipating the centers would need the time to close the gaps. And we recognized that every center was going to need to create this PIPPS program that we'd outlined. Five centers were identified for pilot site visits in 2016, and each of the centers uh had specific areas they needed to improve on to be verified. Ultimately, all of those were verified as level one centers. We did try to engage one level two center, uh, but they were not able to work out their anesthesia staffing model to, to be verified. And in 2017, we opened the program for, uh, regular enrollment. And, uh, Joe Cravara and Stella Harrington, who was your surgical services manager, and I did the initial site visit in the, in the regular program at Penn State. Um, Just wanted to give you a little bit about what happens in a verification visit and how this process works. It's really intended to be a quality improvement consultation by outside clinicians rather than an inspection. Um, I always tell people our intention is to help the centers improve. And if we fail to identify opportunities for improvement, we've failed to add value. Um, the visit occurs over 2 days with a 3-person team, including a surgeon, an anesthesiologist, and a nurse. And during that time, we really try to get to know the center by reviewing medical records, interviewing leaders. Uh, reviewing performance improvement projects and meeting frontline staff. And ultimately, we determine whether or not the standards have been met. Uh, based on those initial site visits and remediation plans, we identified some major areas of impact. Um, the development and the maturation of the PIPS team was a change in every center we visited. Um, there was a shift in many centers and who provided anesthesia for children under 2. Um, similarly, the availability of pediatric emergency medicine providers changed in many centers. And in many centers, there actually was a change in how they privilege surgeons. Uh, for the children's hospitals that are part of big systems, oftentimes, they didn't have age-specific surgical privileging. And, um, and the, the specifics of, of privileging for children changed. Uh, just to show you the value of a multidisciplinary surgery in Pittsburgh, I just want to give a brief example from our center. This is a non-English speaking family that brought their child to the emergency department after the child reported swallowing a, a foreign body. Child looked well and the history is confusing to our triage nurse, and the child was triaged with medium priority. The child was evaluated by the uh Ped's emergency medicine fellow obtained a chest X-ray. He appropriately recognized it was a button battery and consulted Pete's GI um, who, after evaluating the kid, determined that surgery or ENT should be called because of the button battery. Surgery was then called and removed the battery, but there was a significant burn. The child never received sucralfate in the emergency department because, uh, the, the EM physician was uncertain whether that would, uh, mess up the NPO guidelines for anesthesia. Um, and so the time from presentation to removal was 4 hours and that child ultimately developed a stricture and needed multiple dilation. So there were lots of opportunities for improvement. Um, with our surgical PIPS program, this was a pretty easy, uh, problem to solve. Um, we have everyone at the table and the result is this algorithm that's now the standard throughout Intermountain Children's Health, um, where the triage nurse, uh, by protocol obtains the chest X-ray and notifies the emergency medicine attending. Um, if a button battery is identified, the child's given Cafate by protocol, and we activated trauma one. and all of those, since we've done this, every patient's been in the operating room within 60 minutes from presentation. And that was possible because we're all at the table and there's no question about, uh, uh, how we're gonna respond to if the child's given Carafate or, or how the surgeon is gonna respond if it's, uh, activated as a trauma. Um, shortly after we launched the program, we're contacted by hospitals that specialize in musculoskeletal and oncologic diseases that they want to be included. So we, uh, developed a separate set of standards and have subsequently gone on to verify, uh, both musculoskeletal hospitals and oncology hospitals. Um, as a quality improvement program, we wanna be improving. And so we, uh, um, We've revised our standards, uh, uh, 5 years into it. Our goal was to make sure that the standards are meaningful, uh, and as minimally burdensome as possible. Um, that's always a balance. 4 years, um, Uh, into, we've learned a number of lessons. Um, one was that although we thought we were being inclusive, we'd actually made a mistake on the academic standards by requiring that every children's hospital that's level one have residents. And we learned that there are hospitals that don't have residents. We're able to provide comprehensive care, so we changed that. Um, we also worked to more precisely define what nursing care standards are. Uh, but most importantly, what we did was we introduced patient care expectations and protocols and said that we expected hospitals to have a systematic approach to caring for patients to improve outcomes. The most important of these uh uh are out uh outlined here, and that really what it did was uh moved us towards the idea that a maturing protocol, a maturing program should have protocolized ways of caring for kids so that it's done in a standardized fashion. The single most important standard, it was the addition of the perioperative anesthesia risks assessment Program. Um, I'm convinced that this is making care safer for children. The goal of this is to make sure the centers reliably identify and mitigate risk factors prior to anesthesia for surgery. The basic requirement is that there's a systemic, systematic review of all children with comorbidities with guidelines under the supervision of an anesthesiologist as a medical director. And while this may sound overly simplistic, I can assure you from our site visits. Uh, that most centers lack this structure, uh including that most of them don't have specific screening guidelines to guide the nurses that make these calls. So it's really been an area of change. And we've really seen a wide variety of practices from small hospitals having it where CRNA can review the records two weeks out for every child to large centers that are developing EMR risk-based calculators. So it's really, I think, an area where we're going to see a lot of shared learning. Um, These are just some of the other areas where we set patient care expectations uh that we're asking centers to, to demonstrate that they're engaging these in a meaningful way. And we've specifically asked them to uh demonstrate perioperative antibiotic stewardship programs and opioid stewardship programs as well as assessing for the timeliness of repair, of response to these emergent surgical problems that we've identified. I think these are all things that we agree should happen and is on everyone's agenda, but there's something about having a deadline and knowing that we're coming next year that actually gets, gets the reports written and looked at, so. Um, we're now in the phase of really trying to demonstrate that our, uh, program makes a difference. Um, and we're very fortunate that we actually have Anusha Matura, who's a, a resident from Stanford as our, uh, research scholar at the American College of Surgeons for the last year and a half. And she's engaged in a mixed method study to assess the value of the program. And this has really included both quantitative and qualitative studies. Most of her efforts aren't, uh, published yet, so I will, I will not steal too much of her thunder, but I did wanna share a little bit of it with you. Um, one thing she's already, uh, been able to demonstrate is the, uh, use of opioids and procedures where children should not be discharged with opioids at all by consensus is different between, uh, participating in, uh, non-participating centers, but in terms of verified centers. Um, she's also, uh, has preliminary data that is going to show that, uh, that the complications in duoden and atresia is different between verified centers and non-verified centers. Uh, including all centers that participate in this quip. Um, she has conducted qualitative research where we've looked at, pulled together focus groups, um, to understand what the benefits and the challenges of the program are. And the themes from, from these qualitative uh focus groups have really converged on several important ideas. Many of these focus on the broad issue of culture change that happens during comm increased communication and collaboration. They also talk about the value of external validation of priorities that the clinicians are trying to advance in their institution. And that, that prompts the commitment of resources and engagement uh to priorities that they recognize that we're unable to effect change for. Uh, since its inception, the program's consistently grown in, in spite of COVID and the financial challenges that came with that. We now have 61 verified centers. The verified centers are spread across the US and NW's in 6 countries. Um, this is just again, the, the timeline, which, uh, uh, just to demonstrate what's really happened really over the last, uh, uh, about 8 years. That's all well and good, but it, does it helped that baby that was in that referring hospital. Um, well, that, that hospital is not verified and, uh, given their volume, it's likely that it never will be, um, because we're a voluntary program. Um, but I, I do believe we're partway there in helping this baby because I think what we've done is we have defined national standards of what care should look like. So for that young pediatric surgeon that's, that's confronted with that, They can now confidently say that that's not how children should be cared for and that there's a national organization that says so. Um, we hope that that will prompt centers like that hospital to either obtain the appropriate resources or decide to, uh, prospectively limit their scope of practice. I think the ultimate way that we're going to affect change and what, what are the, the next horizon for us is, is to begin system verifications. Um, so, my health system has uh 35 hospitals. We have Uh, 2 children's hospitals are building a 3rd. And I think really the way to impact the children that are cared for in the general hospitals is to make the children's leaders in those systems responsible for what happens in their, in their children's hospitals. And we're, uh, working towards trying to create a, uh, a system by which we'd, uh, verify an entire health system that it's meeting the requirements for all their hospitals. Um, and finally, I'd just like to wrap up by thanking you for your generosity and, uh, having me here today. Um, I specifically, uh, wanna thank, uh, Connie and Sean. Uh, you, you, uh, really are the critical thought leaders for, for this, and the program wouldn't exist without you. Uh, also incredible worker bees. Um, but most importantly, I just wanted to thank your institution because it it's a voluntary program, the validity of this for the ACS comes from the fact that hospitals are respected, engage it. Um, the, the fact that the, the well-known children's hospitals are respected, that are respected by other hospitals do this is what makes the small hospitals wanna be a part of it because they, they wanna, they wanna earn that merit badge that you have. And the, this, this program would not be successful if we didn't have engagement of hospital like yours. So, again, thank, thank you for all that you do. Yeah, thank you, Doug. That was uh an exceptional overview, uh, absolutely outstanding. And every time I hear the talk, I actually learned things that I can't believe I didn't know. Um, questions. Comments. Mm Connie asked a question I won't be able to answer because she's been, no, no, actually I just wanna hopefully it's a comment. No, actually it's a, it's a comment, um, and it's really, um, meant to, I wanna thank you guys all for, for giving me this opportunity to do this, and I, and I agree Keith Oldham was, has really been a tremendous mentor for me, but I want to make one point that you said, but I just want to emphasize it. The, one of the things that I learned, and actually, I am a member of the American College of Surgeons, but not an official member. I'm an affiliate member. But what I learned is that when you learn about all of the quality improvement programs, we are the only program that has pediatric anesthesiologists or has anesthesiologists as part of their program. All the other programs site reviewers are surgeons and maybe some nurses. I don't, I don't know about some of the other ones. But none of the other ones really recognize the importance of anesthesiologists except the pediatric except the Children's Surgery verification Program, and I think people really need to realize that we are the only group within this large expanding group of um Quality improvement programs with the American College of Surgeons that actually involves anesthesiologists because you guys were insightful enough to recognize that we were equally as important in whether on the outcomes as, as who the surgeon is. Thanks, Connie. Uh, 11 thing I'll disagree with you on, I, I don't think we're equally important. You, you guys are actually more important. I think that, that was one of the things we learned out of the data, right, is that the, the risk of cardiac arrest in small children and even simple things like tonsillectomy. is, is predicted by whether that child has a pediatric anesthesiologist or not. And there was, there, there's good literature about that. So the surgeons don't know that, you actually ought to have an anesthesia talk about uh the risks of uh child anesthesia. And, and now I tell, like, if I have friends who are getting an operation somewhere, I'm like, look, you need to have a pediatric anesthesiologist. Like, we can fix the bad surgical problem. The, the bad things that happen to anesthesia may not be able to fix. So I think that having a Uh, pediatric anesthesiologist for, uh, small children is absolutely essential. It, it, that was not an easy battle for us to win. And, uh, but, yeah, we, we had great support from our, our pedia anesthesiologist and, uh, eventually got to where we've reached the point that the ASA wouldn't openly oppose us was what the, the peace treaty we negotiated, so. Yeah, I was gonna make a comment before Steve, and that is, is that we are very spoiled here at this hospital. I think a lot of children's hospitals folks are very spoiled, but there are a lot of hospitals where, you know, the adults are providing the care and when you review those charts, I mean, it's really incredible, um, the amount of risk that's accepted at many hospitals simply because it's kind of standard of, of care there and so, uh, as Doug mentioned that was probably been the biggest impact nationally in terms of making change, but. Uh, another question that pertains to that and so we have, um, you know, 60, almost 70 hospitals now that are verified, and most are level one, small handful of level two, and when we take our hospital and we make it a little bit better, we clearly are better because we've seen the evolution, but probably the greatest impact, of course, is getting those community hospitals to become level 3s and, and that's a that's a struggle because collectively that's probably where we're gonna make the biggest impact for that safety aspect of things and. Kind of what are your thoughts on, on where the college should go to address that unanswered gap? Yeah, I, I think obviously one of the challenges, the, the financial resources to do these things. I, I, I, I, I think they're probably two groups of those general, smaller general hospitals, right? And uh, The ones that are freestanding, not in a system, maybe critical access, I think those are really gonna be, be hard to penetrate. Um, they, they just really don't have much program resource. I think probably that's Through education and trying to help the providers there be, be the, as well prepared as they can be. I, I do feel for hospitals like, like the ones that I'm a part of, we, we do actually have a leverage arm, right? That if, um, that if you're in a system, the, the children's health leaders should lead what happens to children in all of those hospitals. And that, that's the transition we're making now at Intermountain uh Health, which is That, and we're still defined actually what it looks like, but that That the children's providers define what quality looks like and starts to define the scope of practice for these other hospitals. We've done that with anesthesia standards. Um, we drew the cut mark at 6 months rather than 2 years. The, the literature that's kind of, some papers at 2 years, some are at, uh, 6 months. And just at a practical level, we, we had to split it at, at 6 months so that we could provide care rather than at 2 years. We're doing that. We have centralized review of all pediatric events that occur. Um, that involves children provider. So I think that's Probably our way that we're gonna, I think that's the next place that we can make uh effective in, in these ones where there's general hospitals with, um, they're tied to a children's hospital. I think the other area that's probably an area of risk that I, I, I think is a real challenge is ambulatory surgery centers because they are um unregulated. Many of them aren't in systems. And, and they have a stronger profit motivation than, than, than general hospitals do cause they're often provider-owned or partially provider-owned. So that, that's a part we haven't even tried to wade into. But, and, and there are lots of kids who get tonsillectomies and airway things done in those, in those settings. So, um, but yeah, I think the next hurdle is, uh, we'll, we'll work on the, the general hospitals that are part of systems and see if we can identify those. We've got several systems that are interested in that and we'll Try to push that forward next. Oh, we're running short on time. I, I wanna, um, add to Sean's thanks to you for serving as our, our, our Weitzman lecturer this year and, and, um, highlighting something. We have a lot of people here virtually, uh, a lot of them are also from our anesthesia department in, in addition to many of our anesthesia, uh, team here, and you've, uh, made their heads swell appropriately because, uh, we all know in, in, in, in our sort of, we have less of a system than you have, but we're often in discussions about where and how we provide surgical care and I'm frequently, uh, quoted as saying that the fastest way to have an anxious surgeon is to have an anxious anesthesiologist, uh, because we, exactly with the reasons you, you state, um. The type of work that you are doing. That Sean is doing that you guys are highlighting is something that happens a lot in the background and many people in academic medicine. Make a difference one patient at a time. And we're very proud of that, but this type of work makes an impact throughout systems throughout countries beyond our country. Uh, it's, it's magnified. It's like trauma prevention. People don't get a lot of credit for that, but it actually has more impact on children's health than any one of us can make individually, so. Um, Keith Oldham, uh, uh, I, I know is proud of, of your, uh, succession. Uh, people like, like, uh, Connie, Joe, uh, Sean, a lot of the people are doing things in, in, in the background, even within the institution. I would say most of us don't know what other people are doing, um, during the day, I think this afternoon if we have pips maybe, uh, um, and, uh, I was completely naive as to what was going on in the backgrounds institution. Uh, and all the work that goes on and, and people don't realize we literally look at every single testicular torsion, how long it takes to get to the operating room, how long it takes to get imaging. Uh, these are things that these guys have championed in the background, uh, and it has made such an important difference. Um, um, I wanna thank, uh, Sean and, and Bob Shammerer for, um, creating, uh, his, his, um, endowed initiatives in, in, in multiple, including this lectureship of which you're now in a. A chain of, of, of really distinguished lecturers and, and, um, uh, Mr. Weitzman was here a year or two ago in town and we had the opportunity to introduce him to uh Sean and some of the other faculty, and, and trainees who have benefited from his investments, uh, and, and, uh, highlighted this, um, work done by yourself and others, um, and all the things that have been stewarded and, and he said, he's a pretty successful businessman. He said, This is the best investment I ever made, and that's clear. Um, so I will, um, we will pass on to him and, and his family, uh, the work you've done and, and I wanna thank you, uh, so much for it, and we look forward to, um, making you work for, for the dinner last night and lunch today because our, our, uh, faculty and our trainees and, uh, and our, uh, broader team are gonna really enjoy, uh, learning from you, uh, throughout the day. So thanks so much for, for joining us. Thank you very much.