Uh, we're gonna talk about artificial intelligence and we have, uh, Carlos Colonga from where again? Hospital Reginal Matern Infantile Spees de Monterrey. He's from Mexico and, uh. We have M Gatee who uh came as my uh medical media fellow from Turkey. She, after graduating medical school, and I was fortunate to recruit her to stay on as, uh, one of my colleagues now at Cincinnati Children's helping us in innovation and Ramisha Aban, who I was introduced to years ago, who is, uh. Uh, ENT surgeon who is now a, uh, PhD and a professor in, uh, instructional technology at Utah State University. Rock on. All right, this is the team they're gonna talk, update us on artificial intelligence. Take it away. So, uh, we are going to talk about, um, uh, AI, specifically regenerative AI, uh, use cases in, uh, uh, medicine and medical education, and we have three, categories of, uh, use cases, research, productivity, and medical education. Um, so let's start with, with that, uh, poll question here. So, uh, which of the following is false, um, is false, OK, so that's tricky a little bit. Uh, chat GBT can help to organize literature review. Chat GBT can analyze data within an Excel sheet. Chat GBT can run statistical analysis. Chat GT can help in the research design, or none of the above. So what do you think? Any thoughts here in the room? Do you think any of these are false or all of them are true? What about the unpaid version? It still doesn't. Yeah, Mike, Mike. Yeah, oh, great, yeah. We're talking about both of them, yes. OK, let's keep going. Yeah, so it's none of those, all of them are true because, so we um are researching how Chativity uh can help us in uh the research process, um, and I personally use Chativity in, uh, systematic review and meta-analysis and, uh, um, uh, you know, when you do systematic review, you skim through, you have, uh, you screen. Thousands of articles and then you have a title and abstract screening and then the full text screening and uh this uh process is intimidating. You have to have uh many people uh involved in that process so we tried Chat GBT to do the title and abstract screening as well as the full text screening and it does a great job. Do you wanna explain how to do it? Yeah, sure. I think we have the next slide here, um. Talking about that, so we have a bunch of articles to go into title and abstract screen and then we ask Chat GBT this prompt. Uh, I'm conducting a systematic review of the use of AI in organ transplant plantation. I need, uh, you to act as a data analyst. Uh, please analyze the attached Excel sheet and screen. They included articles for title abstract. By the way, that, uh, Um, the AI in organ transplantation paper was published already, so, uh, we use ChaGBT in that, use the following inclusion criteria. We have, uh, some inclusion criteria here and then after screening the title and abstract. So now we want Chat GBT also to manipulate the document itself. So it's not only just. Giving me texts uh and uh answers through Chat GBT but also go inside the Excel sheet and then, uh, read the data from Excel sheet, add columns to the Excel sheet and say whether this article is relevant or not relevant or may be relevant and because we don't trust Chaivity a lot. I asked Chat GBT also to add a column to explain why the uh Chat GBT decided that decision. So you have a better understanding about this data and uh if you don't agree with Chat GBT you can manipulate the prompt, engineer the prompt to make it more accurate. OK, and we get an excellent result with that. We also, we, we had to do the human, uh, title abstract screening also because we want, uh, to have, uh, a reliable answer, but we have, we did enter rate reliability between Chativity. And the human uh um uh reliability and uh the interre reliability was excellent so chargeBT could be used in that uh part, yeah. So what is, what is the first cycle and second cycle so yeah, OK, so, uh, first we wanted to use chargeBT uh to give us yes or no. Question, uh, yes or no, relevant or not relevant, and then we just uh redesigned the prompt a little bit to make chat GBT say whether it's relevant, not relevant, and maybe be relevant. So we decided to consider all of these versions of the prompt and then make the interreliability between all of that, uh, columns. Question, I, I've, I've done this not for the exact same purpose but I don't get the same answer each time so my question is about reliability. I know humans are not reliable either, but you know it sounds like every time you offer this question to at least to Chad GPD, I haven't tried other ones, er, you get a, a slightly different answer. It it approaches the problem like for the first time every time I think so. Yeah, so if you use chat GBT in the general search, you get different answers, but we created custom GBT for that. So you have multi, you, you can walk chat GBT on serial of prompts until you get the most accurate results and which is just not that difference between if you run it multiple times you get a slight difference and we had to have human. reliability here also to just make sure that it is reliable. So, but uh that's a good potential for, for uh the future iterations for JGT for example. I think also it might be one of the questions that what is the AI limitations when you use it for an actual paper and go for publishing. What is the uh disclosures that you do for the, uh, journals? Do they have any limitations? Do you have any extra information on that? Yeah, so we use so we use Chat GBT here. It is as a part of the methodology and part of the reliability process. No, we don't use it to, uh, for, uh, for example, as an act of plagiarism or or writing, yes, um, so yeah, uh, our journals are very, yeah, of course, yeah, and, and we frankly say that this, uh, cycle of screening is done by tragivT. The second cycle of screening done by tragiT. 3rd cycle of screening is done by humans. So yeah, yeah, you always have to have a, a human in the equation. So we have to take in, in mind that. These AI models, they help us increase our efficiency and sometimes we equate wrongly efficiency to productivity, but it sometimes you get productivity out of the efficiency so it helps us be more efficient by accelerating how we do the work but we always, at least in, in, in the medical and the science part, we have to have a a human in the equation because we have to remember that even if we use the prompts that help you get the same results. Because chat GPT is a black box model that we don't know what it is trained about and they're always changing the weights it'll change, so it needs us to tell it what to do and to understand and and read the the answers of chat GPT. Yeah, you wanna move on to the next one? Yeah, so let's go to talk about Jenny AI whose motto is use AI to supercharge your research paper. So what does it do? You open the GE AI, which is one of different academic, uh, models that has been finally fine-tuned with research papers, and you literally put your prompt I wanna write about, uh, this, this and that and this example let's talk about pediatric interception. You put the prompt and the first thing that you'll get is a suggestion. You can start writing this. You, because you're the surgeon that's that has the hand on the wheel can say yeah I like it I want to modify it. I want to change it in this and and that way and then uh the next slide please. You can also talk with it it's in-house chatbot and say, hey, I really want to give it this spin. I want, I wanted to to research specifically in this, and the chatbot will give you the answers that you're that you're searching for. It will cite. Papers to you and it can help you cite on your preferred type of citation um specifications into your paper. Next, please, so here you've decided that yes I, I like what it is saying. I also want you to add this, this and that on my academic paper and it gives you a little suggestion of the summary of the papers that you want to introduce so we can see how this accelerates your paper writing but you're still at the command of it you're still in the home. Next, please, so let's do an audience poll to see how you're feeling about this. So how recent are the articles suggested from the use of these AI models? So they have a cut-off date of 2023, they're older than 5 years, older than 10 years, and they're on par with the latest published research. What do you think? Any ideas? She keeps writing Can we see the the answers from the international audience please? I don't know if the poll is up. Oh, can we see the poll results? OK. Half the people know what's up. Yeah, so mostly nowadays they're, they're mostly on par. We just have to research into what model we're using and what library they're using, but it is on par. We just have to remember that they're searching for those latest articles in this specific type of library, which we'll talk about it in, in later in at a later moment. Well, that was the moment, yeah, so, so also there's another tool, uh, with the Jenny AI is called open evidence and. Um, I think you said this is the difference that this is, uh, not necessarily helping you write it, but it helps you do literature search exactly, which is amazing. Uh, this has been, this was part of the program accelerator over in in by a Harvard team and the Magic Clinic, and they specially focused on the Elsevier Elsevier library, so you can ask any type of question. It will reply in a conversational manner. Trying to answer your question while summarizing the papers, the, the latest papers that it's citing, so it gives you a conversational answer and then at the end of it it gives you the papers that you can personally check out to confirm what it's saying. If you want to press the details, it gives you a bigger summary, and then it suggests, hey, so you're talking about these things just cited Dr. Russell's paper in the second part, yeah, yeah, and, and if you wanted to go accurate, yeah, and, and it also gives you suggestions. So I know you're talking about the appendectomy. Hey, you could also talk about this and that which is up in the next, uh, slide. And you can ask follow up questions. Yep, yep, yeah, so it's, it's pretty great. So what would you say the personal practice changes for your part? For my part, OK, that's. So, uh, so we are going to, uh, talk about the, um, the medical education part, is that right? Medical education is, is coming next. OK, we, we, we're gonna first talk about the productivity part, but I think, uh, it kind of summarizes that large language models can, uh, enhance the efficiency and productivity, uh, when you do research by, uh, analyzing the large data sets for you, or you can use. Those tools to help you write this is especially very important for the international audience that have, uh, once you have something, um, on hand uh and work on it is way easier than, uh, writing something by yourself by yourself from scratch so these uh large language models also can be an editor and help you write these papers. And we're gonna move on, yeah, Jose. Doctor Holcomb, what's the JPS policy about this AI tools? I think we wanted to submit something. They asked if we used it. Yeah, so, uh, you can use it for, um, you know, background information, but you can't use generative AI, that is, you know, to, to have AI write you the paper. And are you making sure it's not being used or you have to disclose it at some point? Well, first of all, if, if you used it and you disclosed it, it's probably not gonna get accepted. So, um, but, uh, we have, we being Elsevier has tools to, to figure out if, if it's been used or, or if it's being used. So So that's sort of the policy is you can't use generative AI to create your paper, to write your paper. Uh, I think it's, uh, OK to use it. Uh, you could, for, uh, for, um, uh, language, uh, it's probably appropriate, uh, and for, uh, literature, you know, background information, it's certainly appropriate. I think it all started, uh, a couple years ago when chat GPT started getting used in, uh, different fields. Uh, the first couple papers were written and chat GPT was among the authors, and I think it's not been permit and permitted anymore. You cannot do that, but I don't know, um, probably it's the same for the older journals. You cannot ask Cha GP to write a section of it and showed it in the authors. Yeah, most of the journals I see is just uh asking you uh to, if you want to use chat GBT you use it to just enhance the writing style, not create the ideas. So, yeah. And our, uh, next question because we're gonna talk about chat GPT and, and other AI tools and how do you use to enhance your daily productivity, not necessarily for research, and we asked the audience how often do you use chat GPT or similar AI tools it could be cloud, Gemini, anything. Um, it's a, it's a spectrum from ghost to never to always, and one third is never. Uh, whenever Todd sees those numbers, he kind of have it like a heart attack every time he sees people thus never use chat, GPT, or AI. The What is it? So, uh, if you first start with productivity part with MeatGeek, uh, this is one of the tools that helps you that you basically synchronize this tool with your calendar. It jumps in any meeting that you attach to your calendar and it takes notes for you and at the end of it, it sends you a mail of the transcription, notes, and the summary. And it basically tells like who was in the who were in the meeting, what was the summary, uh, and what you need to do, uh, it goes with the recording and it uh assigns action items per person who were in the meeting. And Fireflies is a similar one with the MeatGeek. If you go look, uh, in the Internet to see, um, meeting summarizer AI tools, they're like a bunch of them. Most of them does the same job. They're on different price points. It's, um. Most of the time it, it just personal preference. They mostly do the same thing. Uh, I know, uh, you start using MeatGeek and then you transfer to Fireflies because we influenced you. Yeah, I think Fire, yeah, I, I, so I use MeatGeek. I use the paid version of MeatGeek and then Firefly is the free version and then the paid version, and I think that when you invite Rami to a meeting shows up as 4 people. Rami AI note taker one Rami AI the human is not there, is that right? So I, I think Fireflies, the Fireflies, uh, my personal preference is because it has also action items that, uh, so after the meeting it, uh, it gives you the action items for what's next, so it's just keep you, uh, um, uh, informed. It is great. I use Fireflies because of you guys. I started using it. I, I saw you were using it and I yeah, Carlos also comes as two people every meeting. Um, the issue that we've run into because I was addicted to it, I needed it because it gives me takeaways and all the key points. Here's what you need to do. Well, but then our hospital got people got worried, especially with some of the meetings I was having, that it was auto recording everything. So you just have to figure out what's the safe way. Our at Cincinnati Children's, they now have. Co-pilot's approved by co-pilot that's approved by the hospital, but even then our lawyers asked me to stop using it on half of my meetings because they're legally sensitive. Well, you talk about confidential things, yeah, confidential things. So, um, it's, it's, I don't, we're gonna make progress with that because this is such a valuable tool and we have to get past the security part of it, but it is, especially in a. Yeah, especially if you're using an institution level, it needs to be a lot of, uh, preventative, um, like preventions be taking place for security purposes, especially like you and Doctor Wen Alman's level executive meetings, but like you and me in a like meeting fireflies can record that there's not gonna be that one. Added it to our platform so we use a co-pilot and that's the only thing that's basically approved you know for Carl's stores to use um in regards to note taking or getting information recording stuff and then, you know, analyzing it afterwards. I think it's the same as Cincinnati Children's and uh they keep the they they generally send us like weekly AI guidelines of how should we use it and what kind of information should not be used when you use copilot or ask for meeting summaries and everything and I think that's a good step to um. Endorse people to use with not giving away the secrets of the company basically yeah we have to have these talks we have to learn how to use it because it, it's, it's here to stay. We, you, we can't close our eyes and don't admit that everyone's using it to increase their efficiency and improve the way they manage the the day to day lives. Yeah, Zoom AI basically does the same thing because some companies have, uh, Zoom instead of. Microsoft, but the plus I saw in the Zoom, uh, if you're late to a meeting, uh, and you have Zoom AI companion, you can ask if your name is mentioned what people ask you to do, and recap the meeting until, uh, like just before, uh, you join what happened so I think that that's a very cool thing and um if you're generally late to the meetings I would definitely recommend. One minute. One minute. OK, sorry. OK, this is Todd's favorite read aloud GPT we created for it. You can use like you can use this QR code and you can use by yourself. It's basically helps, uh, you listen to your long emails or any text that you need to listen to instead of you read. Uh, we know Todd talks, uh, they have to not talk sorry, he has to read a lot of things with the emails and everything and sometimes, uh, what do you call like walk and talk talk talk. So it's, it's hard to, it's hard to read when you walk, so you listen to it and you use a chat GPT to have conversations instead of an audiobook. I will drive. I'm about to drive to Cincinnati after this, and I will talk to chat GPT for an hour about a book. So I'll say summarize this book, and I will go back like a book club. I'll say, tell me now what does it say about this? Tell me, let's go into chapter two. I wanna know this. I don't get this, so I will have a conversation while I work out, while I am driving, yeah. And I think this is one of the uh most important ones that uh for this audience here um Todd said he's getting a lot of requests for a letter of recommendations and we created this GPT for him and um you can use the QR code if you can't put the slides up here. But that's, but they keep saying we created GPTs. It's very easy to do so if you're using the same prompts over and over and it's a series of prompts you can create it and just keep reusing it. So when I have a letter. That I have to write it goes through. So yeah, so when you use this QR code you're gonna go to this, uh, letter of recommendation, uh, GPT. All you need to do is upload the CV cover letter. What are you, why are you writing this letter of recommendation? It's gonna give you a draft. You can start working on it and personalize it for the candidate. Um, we do not recommend the first thing you get from Chachipi to use it, but it's a great start for it. It will sometimes be over. Uh, yeah, this person is from amazing and so you, you have to, I mean, I actually almost always come back and say I really have don't know this person. I'm giving a, uh, a reference, so please tone it down. And then it's like this person's very good, blah blah blah. So you have to just tell it what you want. All right, like with Halpern, I'm gonna say to please really tone this down. He's not that good. Do you wanna, do you wanna do the medical education real fast? Yeah. Sure, so just one minute, yeah, so we just, uh, talk, uh, quickly that. So we, and the medical education part if you want to, to have a personal trainer, we use also chat GBT and other, uh, generative AI in that. So you can, for example, ask Chat GBT to upload a guideline to Chat GBT and then ask Chat GBT to create a training session, an interactive training session between you and Chat GBT. Can we put the slides up? Thank you. Yeah, so, uh, you can, um, ask JGBT, uh, to create a training session out of a guideline so it doesn't have to go into hallucinations or fabricating information. It takes the guideline and then it creates a series of interactive questions where you can interact. Chat GBT and get trained on uh specific things. So for example, that's an appendicitis case that we uh ask Chat GBT to create problem solving activities and also we ask Chat GBT to score these activities and give us feedback at the end. So that's the feedback that uh. Uh, it gets at the end, on a similar note, uh, you could also make it do any type of case, step one, step two type of case, any type of, of case you can ask for it. You, you do the, the correct prompt, and with that it'll help, it'll help you do the medical case in a step by step manner. And at the end it, it will give you feedback. All right, this was awesome. I let us go way over. That's my fault. I'm sorry. Yeah, it's cool. Keep it there this, this is phenomenal. These guys are great. Um, we're gonna keep doing series of courses throughout the year, um, for these kind of questions. But guys, thank you so much. That was awesome.
Click "Show Transcript" to view the full transcription (21337 characters)
Comments