{"id":15614,"date":"2026-09-21T04:54:37","date_gmt":"2026-09-21T04:54:37","guid":{"rendered":"https:\/\/medical-article.com\/?p=15614"},"modified":"2026-09-21T04:54:37","modified_gmt":"2026-09-21T04:54:37","slug":"owen-tripp-included-health-how-to-fix-ai","status":"publish","type":"post","link":"https:\/\/medical-article.com\/?p=15614","title":{"rendered":"Owen Tripp, Included Health\u2013How to Fix AI"},"content":{"rendered":"<p class=\"wp-block-paragraph\"><em>It\u2019s been a while since I talked with Owen Tripp, CEO of Included Health. They\u2019ve now introduced Dot their AI companion which <a href=\"https:\/\/includedhealth.com\/announcements\/included-health-evolves-dot-from-answers-to-action-with-human-in-the-loop-support-always\/\">had a big upgrade last week<\/a>. We talked a little about that and I snuck in their video comparing the Dot Experience with a standard LLM. But the conversation really got into how do we make AI safe and trustworthy\u2013which is definitely the hot topic these days. Owen is putting together a coalition of the willing to work on that exact topic. I\u2019ll be watching closely\u2013<strong>Matthew Holt<\/strong><\/em><\/p>\n<div class=\"wp-block-embed__wrapper\">\n<\/div>\n<p class=\"wp-block-paragraph\"><em>This was such a great discussion I wanted to publish the transcript. The way I do that is to copy the YouTube-generated transcript and drop it into Claude to smooth it over. I then read it, and if I think it\u2019s made an error, I dip back into the video and listen to what actually happened and make a correction. This is all to say: I think this transcript is pretty accurate, but it might have a bunch of AI- and human-generated mistakes.<\/em><\/p>\n<p><span><\/span><\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Matthew Holt with The Health Care Blog, back with Owen Tripp, the CEO of Included Health. Owen, we haven\u2019t chatted in a while, and the good news is absolutely nothing has happened with Included Health, or about this topic you may have heard of called AI, in the last six months \u2014 but apparently we\u2019re all stopping now, or something. Anyway, just to bring everyone up to speed: Included Health, for those who haven\u2019t seen it before, is a company that now encompasses a lot of different healthcare services, going all the way from second opinions \u2014 which I\u2019ve used \u2014 to primary care, all the way to navigation, working with many big employers, including the biggest public-sector employer in the world, in the union, I guess \u2014 CalPERS \u2014 and many others. You\u2019ve been talking a lot about a number of things, like health plans, but probably the most significant thing Included Health has done in the last year is come out with its own AI platform, called Dot. So let\u2019s start here \u2014 we\u2019re going to go bigger into AI, but let\u2019s start with: what is Included Health doing for its customers with AI right now?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, good to see you. One of the things I think we can celebrate at the start of our conversation, especially as two old dogs in healthcare \u2014 I\u2019m not sure, actually \u2014 I think on this podcast we\u2019ve talked about the story of us first meeting, in which you told me this would never work.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">That would never work \u2014 I said that second opinions alone would never work.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Anyway, one of the things I think we can celebrate is that this topic, which hopefully you and I are going to spend some time on today, is really one where I don\u2019t think we\u2019re starting from behind. Those of us who are technologists working in healthcare really feel like we\u2019ve always been trying to drag healthcare into modern frameworks \u2014 data compatibility, member-facing and member-usable experiences, provider tooling \u2014 all that stuff has always felt like we were operating years, if not decades, behind. This is one where I feel like the best applications of AI today are already in healthcare, or at least some of them are, and Included Health is very proud to lead the way. It\u2019s been the most fun I\u2019ve had on product, product design, and technology since probably when we launched into navigation almost a decade ago.<\/p>\n<p class=\"wp-block-paragraph\">So I\u2019m excited to talk to you about it today. Really, what\u2019s on my mind \u2014 I can set up what Dot\u2019s been doing, but I have to tell you that I come into this conversation today, on Tuesday, September 15th, with my mind and heart fully on: how do we make sure that all of this innovation, all of this incredible power we\u2019re putting directly into people\u2019s hands to access better healthcare, is safe, is private, is free of bias? These are topics that are just screaming loud in my head, and I want to tell you about what we\u2019re doing with Dot, but specifically what we\u2019re leading with on those key domains of safety and privacy. Matthew, if I may, I\u2019m just going to set up what Dot does for your viewers, because they may not be super familiar with it. If you\u2019re not one of the many, many million Included Health members today, you might not have had a chance to play with this. So Dot is a member-facing assistant that can straddle mind, body, and wallet across the healthcare domain, and indeed tries to make those differences actually disappear, because what we know about member and patient need is that when you\u2019re going through something, you\u2019re likely going to have medical questions, you\u2019re going to have financial questions, you\u2019re going to have administrative questions about what\u2019s covered and how you get access to it. You\u2019re going to want to merge physical and mental health in a way that feels natural, and doesn\u2019t require you to take some off-ramp into another agent, another experience.<\/p>\n<p class=\"wp-block-paragraph\">So Dot\u2019s starting strategy was really to be your front-end, superpowered medical member of the family who could take you across all of those domains. And to give you a sense of history here \u2014 while we\u2019re talking about it today with a freshness, as if we\u2019re just launching \u2014 this is really about two years of experience we\u2019re going to talk about today, in playing around with, and then ultimately productizing, what that member-facing AI can do. We\u2019ve launched this to millions of members. This isn\u2019t some speculative set of information I\u2019m going to share with you today \u2014 this is born specifically out of our direct experience, watching people have very long conversations with Dot. In some cases these go 20 or 30 minutes of engagement, and people are having conversations where they disclose, on a comparative basis, more information than we sometimes see in our primary care practice, where a human is acquiring the same level of history, concern, need, and chief complaint, and so on. We see that Dot is able to handle seamlessly questions about what\u2019s covered by the plan design, who\u2019s in network, what they\u2019re specialized for, and then \u2014 today, and this is the big update we\u2019re sharing with the world \u2014 in this most current version of Dot, we can actually convert that advice, recommendation, and understanding into action. We think that\u2019s going to be a big part of the future of healthcare AI \u2014 that it all has to convert to action. So you can schedule appointments, you can follow up on medical records, you can explore and pre-select different members of your care team to make sure they\u2019re part of your ongoing care. So that\u2019s a little bit of foundation and background on Dot, and I\u2019m happy to start our conversation there.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, let\u2019s dig into a bit of what\u2019s going on here, because you\u2019re seeing the big LLMs \u2014 and some of the smaller ones, I\u2019ve been playing with one called Inciteful Med, which is very interesting \u2014 as well as Anthropic, ChatGPT, starting to advertise how they\u2019re linking back to find your medical record at whatever place and bring it in \u2014 sometimes using partners, sometimes doing it directly, who knows \u2014 but they\u2019re clearly starting to go down the path of realizing that just answering your question about the generic issue you have isn\u2019t enough. So you have the advantage of not only knowing a lot about your members and customers \u2014 what plan they\u2019re in, probably a lot about their health, and so on \u2014 you also have on your team, primary care, mental health, specialty referrals, and a bunch of other stuff. So now, moving from questions to action \u2014 yes, I get the admin, my deductible is whatever, if I want this drug it\u2019s going to cost me that much, which is very valuable \u2014 but the bit people are getting most interested in, and we\u2019ll lead into our later conversation about privacy, security, safety, and where this all goes with AI, is: okay, how much can the AI do, and at what point does it need to bring in a human? And how much time, effort, energy does the human \u2014 the clinical provider, or whatever flavor \u2014 save because the patient has had that conversation with the AI, with Dot, first? So give me a flavor of where that line is today \u2014 what people are saying, when does it go to a human, how does it organize that interaction \u2014 and then where do you think it\u2019s going?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, let me give you some good use cases of what it can do today, and then I\u2019m going to back up and go to the part of what we bring to the equation that makes all this possible, because you\u2019ve covered a couple of pieces of that, but I want to highlight a few others. So \u2014 what can a super-intelligent automation do when it has agentic capabilities to reach across the rest of the system and conduct activities on your behalf? I highlighted the ability to schedule an appointment \u2014 that in and of itself is actually quite a complex experience. You have to understand what health plan and insurance coverage that member has, understand their medical history and specific needs, be able to select a provider or providers well-suited and high-quality for the underlying member need, then find a way to manage the schedule on behalf of both the member and the provider, and then confirm that appointment and make sure all the necessary information is transmitted to both the provider and the member. So that\u2019s actually, inside of it, quite a complex set of activities. All of those are covered, and can cover outpatient activities of really any variety today.<\/p>\n<p class=\"wp-block-paragraph\">One of the cool things we\u2019ve worked on, which might seem small, but I can assure you anybody who\u2019s recently tried to book an appointment has experienced this: you go in thinking you want to book a specific provider, only to discover that provider isn\u2019t available, but the office, in a friendly manner, suggests five other people who could take you faster. Well, to handle that situation, we have to have switching logic built in that helps us understand, in advance, whether any of those other providers would actually be suitable. I think we\u2019ve talked in the past about our approach to algorithmic assessment of quality, so we need to know immediately that, of those five other potential providers, one or two are actually okay, and three others are on the no-fly list, and be able to seamlessly switch that logic in the moment. So just unpacking that one transaction in healthcare helps you understand the complexity there. But that\u2019s not Dot\u2019s only trick. Dot can fetch and review your plan documents, your explanation of benefits \u2014 we can and will alert you to charges that are out of line, and then automatically suggest that we should go fight on your behalf. That\u2019s why we\u2019ve returned millions of dollars of patient-responsible payments to the healthcare system, because they were overbilled by the provider systems themselves. I\u2019ll give you a third example \u2014 we all know that many of us who enjoy commercially covered health plans, the kind our employers provide, with all their benefits \u2014 of course, Included Health offers one to all of our employees and their families \u2014 well, those benefits aren\u2019t always easily recognized or memorized. So another trick Dot has \u2014 one we\u2019ve had from the beginning, we\u2019re just sharpening the capability \u2014 is to say, \u2018Hey, we know you\u2019re coming out of this knee episode, we\u2019ve talked about your bill, we\u2019ve talked about the best place to get your pharmacy, but now we really need to talk about physical therapy \u2014 and you do have Hinge as a benefit, which is going to be less expensive for your specific need than other providers. We think that\u2019s a great place to start \u2014 can we go ahead and enroll you directly?\u2019 That\u2019s the way we want to build across the rest of the ecosystem \u2014 instead of paper partnerships with other benefits providers, the intent is to build freeways that let members connect directly to those benefits. Those are just examples of how we move across mind, body, and wallet \u2014 of course all of our virtual care services too, whether it\u2019s behavioral health, from coaching to psychotherapy to acute psychiatric care, primary care, urgent care, specialty care, all the second opinions \u2014 all of these are integrated through this singular AI platform, and the idea is that the member shouldn\u2019t have to figure out where they need to go, which is, of course, what we\u2019ve all had to do up until this point.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, I\u2019ve actually recently been enrolled in a new plan, and it\u2019s quite something to see they have a list of, I don\u2019t know, 20-odd point solutions, including maybe seven or eight in mental health and three or four in MSK, and you think, how on earth would anybody figure out the difference between these? So I think if you have an AI tool \u2014 whether it\u2019s Dot or anything else \u2014 especially one that\u2019s linked into those and knows, okay, I\u2019m an employee at Walmart and I have these six things available to me, there\u2019s probably one, or one or two, that are actually better for me \u2014 and if you\u2019ve got the data behind that, that\u2019s a very powerful thing.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, and this is \u2014 you and I have been not only students, but I think boosters, of this ecosystem for a long time, and one of the things I\u2019m excited about is that point solutions, which have always had clinical validity but just haven\u2019t had much engagement, might see that they\u2019re used more, and used more appropriately. I\u2019m really excited about that. You can go through the list \u2014 fertility benefits, neurodiversity benefits and support systems, weight loss, other cardiometabolic care \u2014 on down the list, you can find the most appropriate solution. Those should have a deflationary effect on healthcare, because in almost all cases the digital equivalent of that service or solution is less expensive and easier to use. The problem has just been that it\u2019s never been easy to find on the benefits menu. Now, there will probably be some pricing effects, as those same vendors figure out that the PEPM contracts they\u2019ve been on don\u2019t make sense anymore, but I think all of that would be a welcome change.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, and how their stuff gets paid for, and who recommends it, and all the rest, is a separate bucket we can get into. The Peterson folks are running around now saying the kidney disease programs don\u2019t work \u2014 I saw that from them yesterday.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, we see that.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">All right, let me push you a bit more on this. So I\u2019ve been telling Dot something about my health, and all the rest of it \u2014 how much can Dot tell me? And at what point do you decide, okay, I\u2019ve got to stop and get you an appointment? And does it have to be an appointment, or is there a live-human option? What kind of humans do you have available right then and there, versus who I have to wait 20 minutes for, versus who requires an appointment next week?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">This is a great question, and I\u2019m going to answer it on two levels. First, let me start with, actually, the provocative wrapper of this whole thing, just to make sure we\u2019re in agreement: AI today \u2014 even consumer-grade AI, which isn\u2019t what we\u2019re talking about here, we\u2019re talking about enterprise-grade, medical-grade AI \u2014 but even the consumer-grade AI available to most people already can do more than its owners are letting it do. In other words, we could push these things all the way through to diagnostic care. It just turns out that most owners of those AI systems don\u2019t want to inherit the liability and the ambiguous law associated with that. But I think that\u2019s coming. So, with that as a baseline for where we\u2019re starting, the problem actually becomes: how do you safely deploy and manage those agents, or assistants, to help people get care that\u2019s actually safe and contextually relevant? I\u2019ll get to the human-in-the-loop part in a second, but a lot of our early effort was really about making sure we established and trained for guardrails that helped Dot specifically alert and pull people into conversations as needed, either to advance care because it needed human review and approval, or to schedule a necessary follow-up. And critically \u2014 although thankfully these incidents are fewer \u2014 to make sure we\u2019re escalating immediately on signs of suicidality, signs of an acute event requiring intervention. We\u2019ve had this built into even our original virtual care practices, by the way \u2014 a clinician could push a button on the back end and dispatch EMS, because we know where people are calling from, and unfortunately we have to do that hundreds of times a year. So now Dot can do the same flagging from a safety-incident perspective, but also has a lot of guardrails so it doesn\u2019t wander off beyond the scope of prescribing, recommending, and so on. I think we\u2019ve shown you some side-by-sides of consumer-grade AI versus Dot on these topics, where Dot is operating more in the context of what is safe, efficient, evidence- and law-based care in each of the states in which we operate.<\/p>\n<p class=\"wp-block-paragraph\"><em>At this point there\u2019s a 2 minute demo of how Dot works<\/em><\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I get that, and \u2014 my personal experience, which hasn\u2019t included using Dot, because I\u2019m still certainly not as good a health member \u2014 one day I\u2019ll get a job with the State of California and be able to use it.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">You should be a fireman in our home county (<em>note: Both Owen and Matthew live in Marin County, CA)<\/em><\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">\u00a0I should become a volunteer fireman or something useful, once I get my shoulder and knee fixed, that\u2019ll all happen. Anyway \u2014 but feeding my now rather extensive imaging and diagnostic history into both Inciteful Med and Claude, they are coming up with quite a lot. You\u2019ll say, \u2018what about this,\u2019 and they\u2019ll say, \u2018yeah, in many cases here\u2019s what we\u2019d recommend for this to happen \u2014 here\u2019s a course of action, you should probably get this procedure, you might want to get that procedure. If you\u2019ve got this injury, you may be able to wait and do some PT; with this other injury, you may not be able to wait, because it\u2019s going to cause muscle loss, bone loss, whatever.\u2019 So I\u2019ve got all this running around \u2014 I\u2019m getting a lot of advice to go talk to the medical system with, from these already. So is that diagnostics?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">It\u2019s getting there, right \u2014 it\u2019s certainly part of what you\u2019d normally get when you\u2019re talking to a friendly cardiologist or orthopedic surgeon across the fence, or at a cocktail party \u2014 that\u2019s the kind of thing you\u2019d get towards.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So I guess, how far, in Dot\u2019s case, are you going down the path toward \u2018here are the types of things that could happen\u2019 before you actually introduce someone to a specialist or primary care, versus, \u201cit sounds like you\u2019ve got enough going on here, based on what we know from your medical record, that we want to get you in front of those people\u2019?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, let me answer that on two levels. Right up front \u2014 today, in its current release, it will hand you off, when that clinical interaction needs to happen, either to a member of our own practice, or a recommended member who\u2019s high-quality, taking new patients, accepts your insurance, and is in the local community \u2014 and I think we do that really quite well, and continue to work as hard as we can to reduce every piece of friction. So I\u2019d say the commercial consumer LLMs are running ahead of that.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">They\u2019re not running ahead of \u2014 I mean, I\u2019ve had ChatGPT and Claude tell me specifically, \u2018okay, you\u2019ve had this condition, you need to get this fixed using this kind of\u2014\u2019<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Oh, no, no, no, I agree with you \u2014 I haven\u2019t tested all of those things within the last 24 hours, and they\u2019re moving so fast. No, this is why I said that provocative foundational statement \u2014 I think what\u2019s actually possible today, if we removed the governors and guardrails from even our own AI, built here in San Francisco, we\u2019d find that we could let it loose on any number of problems, and it could \u2014 if not specifically diagnosing and ordering \u2014 take you 97% of the way there. That is not, at least today, the safe, appropriate, and most effective way to think about healthcare delivery, and we\u2019re going to get to that in our conversation. But the other level I wanted to answer on your prior question, which I think is useful for you to understand, and a way to better know our company \u2014 we really think of our corporate citizenry on two levels. There\u2019s what we\u2019ve put into Dot, which I\u2019m telling you about today and we\u2019ll continuously update, and then we think of ourselves as a massive stakeholder and influencer on what future possibilities should and could look like. There, we\u2019ve been public about this \u2014 we\u2019ve partnered, over the last year, with one of the leading frontier labs on how to do safe AI treatment of care, and there\u2019s a study we\u2019re working on with them, actually several studies now, some already published, some still to be published, that measure the efficacy of these tools in doing exactly what you\u2019re describing.<\/p>\n<p class=\"wp-block-paragraph\">However, getting to a place where an AI can appropriately diagnose or differentially diagnose your sinusitis is going to be a win, but not sufficient for where healthcare delivery needs to go. It doesn\u2019t actually lead to activation against that problem \u2014 meaning, okay, now let\u2019s go through these therapies or recommended lifestyle changes. It also isn\u2019t particularly accountable or responsible to your overall healthcare picture \u2014 thinking about how much it\u2019s going to cost, what\u2019s going to be covered, what the next steps are. And it doesn\u2019t incorporate the whole-person-care element of this.<\/p>\n<p class=\"wp-block-paragraph\">If I can, I want to pause, because I skipped over this, but I think your audience tends to be pretty sophisticated on this stuff, so I want them to build the whole picture in their mind. We have the ability to train and deliver and be intelligent on all the topics you\u2019d expect of a modern AI company on the delivery of care, and that\u2019s great \u2014 but to actually solve the healthcare conundrum in the United States, you\u2019re going to need a lot more than that. So the other things we bring to the party: a connected EMR that connects to over 70% of practicing physicians in the country, where we can do push-pull on records \u2014 it\u2019s not just about what you supply to us, or your Apple Health record, which at best is a very thin slice of your overall health experience. We also merge in the entire financial experience \u2014 what you\u2019ve spent in claims, what your claims history tells us about where you\u2019ve been, what your pharmacy benefit manager data tells us about where you\u2019ve been and what you\u2019ve experienced \u2014 and then, critically, the actual overall wrapper of what your plan design allows you to do and not do. Unfortunately, on the cost side, this is where people find themselves hitting the rocks \u2014 they think they\u2019ve found the perfect treatment or medication, only to discover it\u2019s not covered by their plan, or that it\u2019s covered but at enormous personal expense. So to really address the whole problem, you have to merge all of that information together, and that\u2019s something we\u2019re already doing, and we think it will be possible for other AI companies to work on, but not necessarily out of the box the way it is for Included.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Okay, I get that, and I think that in itself is a couple of tremendous leaps. The first is \u2014 setting Included aside for a moment \u2014 a random person can take their diagnostics, scans, and labs, get them online, and feed them either via the LLM doing it themselves, or just by copying and pasting, and get back a lot of information that really could only have come from a clinical professional or doctor, you know, two or three years ago.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Totally.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">And you get much smarter \u2014 as I said, I\u2019ve been going through a bunch of health stuff, and I\u2019ve gotten incredibly smart about some of it very quickly. And then you do worry that sometimes you\u2019re the roadrunner No, I mean the coyote running off the cliff chasing the roadrunner \u2014 you think you\u2019re doing really well, until you realize you don\u2019t have years of medical practice to fall back on.<\/p>\n<p class=\"wp-block-paragraph\">But no, there\u2019s no question that many patients are getting very smart about this, and really diving in \u2014 you hear these stories about people whose kids have some weird condition, couldn\u2019t find out what it was, they put it into ChatGPT, and it spits out the answer that 27 doctors didn\u2019t give them.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yep, you have that whole movement, and that\u2019s a huge win.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">It\u2019s a huge win. The second thing is what you\u2019re doing, that most people don\u2019t have yet \u2014 and I think we can both continue to poke the rest of the healthcare system, plans, and others to get people there \u2014 which is, yes, it\u2019s all this stuff you know about people from the admin side, the financial side, the clinical side, and you haven\u2019t even started chucking in things like continuous monitoring, what\u2019s in your Apple Health record, or your MyChart, or whatever it is \u2014 where there\u2019s a ton of things going on that probably haven\u2019t been picked up, or if they have been picked up, haven\u2019t been explained clearly and properly, or for whatever reason, people are left in kind of a mixed state. There should be much more clarity coming out of these AI tools \u2014 and a tool like Dot, I\u2019m sure, does that: helping get somebody to the right place, but also helping prep them to get there, and helping the person on the other side \u2014 the provider they\u2019re seeing \u2014 know what\u2019s coming. Can you talk a bit about how you see that: the more informed AI, the more informed member, and the more informed clinician working together in the future?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, it\u2019s a good push, and, to make sure we\u2019re all on the same wavelength, these things, for as magical and powerful as they already feel, are still only training on largely the open internet, plus a few proprietary data sources they\u2019ve purchased. A lot of what needs to happen when you go through the rotating door at the front of a hospital or clinic, enter the exam room, and then have a bunch of decisions to make and steps to take afterward \u2014 that workflow is actually hidden from, or opaque to, the AI. And why is that important? If you\u2019re trying to advise on treatment decision support and build consensus among a provider, a broad care team, the member, and perhaps the member\u2019s payer, you actually need the ability to understand what\u2019s going to happen next \u2014 that domain knowledge about where the patient is likely to need to go. If people don\u2019t understand what I\u2019m talking about, go test the AI on exactly the experience of a patient post-operatively, for your favorite condition \u2014 what\u2019s the recovery room going to look like, what\u2019s going to be billed, what\u2019s the anesthesia contract at this hospital, what\u2019s going to be on the formulary at your pharmacy. It\u2019ll quickly tell you those things are important, but it\u2019ll have no ability to actually parse them and make them available to you. That\u2019s just going to take a domain-specific company to really push through those and build those workflows, and make sure we\u2019re all operating on the same page, with a shared understanding of what needs to happen. I\u2019m very hopeful about that \u2014 not because I see it working already, there\u2019s a lot of room to grow and do more work.<\/p>\n<p class=\"wp-block-paragraph\">But ultimately, and this gets into the big part of the conversation I want to make sure we cover, we have to do that in a way that addresses the fact that all this information members are interacting with through these AI systems has to be built around a privacy notion that wasn\u2019t considered in HIPAA. It has to be built around a set of safety standards that no malpractice laws really consider, that no compliance laws today really consider. You have to think about these AI models like a new version of a health system, one with really different standards around personalization, privacy, and safety \u2014 because, while I think we\u2019ll all be delighted by the magic of the access, and frankly the low marginal cost of that access \u2014 and these are good things, like the examples you\u2019ve given, the ones I\u2019m giving, these are wins for society and humanity \u2014 they come with massive risks. They come with risks to safety, risks to privacy, risks of bias, risks of malicious intent and manipulation. I\u2019d love to tick through a few of those examples today, if we have time, because I think this is where the industry \u2014 and here I mean the people building these things, me and my colleagues certainly, but also every other company out there \u2014 are real, big stakeholders in where this goes from here.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So before we get there \u2014 let\u2019s say I\u2019m a clinician working with Ami Parekh (<em>Included\u2019s Chief Clinical Officer<\/em>), or one of your team, and a member comes to me who\u2019s been using the AI. How much of that conversation gets shuffled through to me, so I can see what\u2019s happened to them and where they\u2019ve been, before they arrive in my telehealth visit?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Oh, you get all of that. Thanks for asking. Matthew, the fundamental architecture of our company is that \u2014 in the past I\u2019ve talked about how there are these two places in healthcare, and we need a third place. You\u2019ve got the health insurer, who pays for the bills, exists in the suburbs of town in a windowless building, and is in the business of paying claims. And you\u2019ve got the health system, which is often in the middle of town, and has no idea how much things are going to cost. They have their big system \u2014 they call it the EMR \u2014 the payers have their big system, they call it the TPA, administration, claims processing, whatever you want to call it. We need to merge those two things together, because, to your setup, the provider needs to understand, when they see Matthew, that all the things Matthew has already said that are relevant to his history get incorporated into the discussion you\u2019re going to have. I don\u2019t know if you remember, but at the outset I mentioned people are having these extraordinarily long and detailed conversations with Dot. Part of what\u2019s powerful about that is that we\u2019re able to collect history that, to the member, may have seemed inconsequential \u2014 they don\u2019t connect it to their symptoms, they don\u2019t connect it to anything that matters in their medical life, but in fact it\u2019s hugely consequential. This is a shared conversation \u2014 if you think of Dot as another member of the care team, alongside your human provider, with all the support services we already offer today, then you\u2019re actually developing that whole picture, and everybody else is contextually aware. It\u2019s not sitting in these two isolated systems, with the poor member trying to merge it together themselves.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So you\u2019ve got, I assume, a summary of that chat, or that conversation, in front of the clinician? We can start here talking about safety and liability and privacy, right \u2014 you\u2019ve got a self-contained universe where it goes from the member\u2019s chat with Dot through to some summary that helps advance, and inform, the clinician of what\u2019s going on when Matthew, or whomever, shows up in their virtual exam room?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">They\u2019re using that.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I assume \u2014 I don\u2019t know, but I assume, given that you\u2019re also using some kind of scribing and summarization to develop the record \u2014 you\u2019re building the record as you go, but as you said, you\u2019re also surfacing things that presumably are important to the clinician. So that all sounds great and wonderful. Are you concerned about how Included Health is managing that process, or are you concerned about how everybody else is managing that process? Or are you concerned about both?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I\u2019m concerned about both. And I\u2019m ready to update you today on things I think we\u2019ve started to figure out and taken appropriate steps on, as well as some areas we know are important that we still need to figure out ourselves.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So let\u2019s hold for one second \u2014 this week, or the last six or seven days, I can\u2019t even keep count anymore \u2014 we\u2019ve had a lot of back-and-forth. The fellow who left OpenAI because he thought it would be safer, and then decided it wasn\u2019t. Dario Amodei has written a 4,000-word piece about how we need to get external people into Anthropic and others, how everyone should work together \u2014 not only nationally, among the frontier labs, but also, let\u2019s get the Chinese on board, because this thing could kill us all \u2014 that back-and-forth. And Trump says it\u2019s all fine, which almost certainly means it\u2019s not.<\/p>\n<p class=\"wp-block-paragraph\"><em>[laughter]<\/em><\/p>\n<p class=\"wp-block-paragraph\">So, in the context of that \u2014 and I know people are worried about AI systems giving bad actors the ability to do both, say, a Hugging Face-type attack, but also to create pathogens, and who knows what \u2014 in the context of all that, let\u2019s stop and ask: what are the concerns that you have? And then we\u2019ll talk about the steps you think you and others can take. But what are the main concerns?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, so, I want to say this sentence first, before the next 20 sentences follow: I am an AI optimist, in general. I see these technologies as having transformatively positive properties for humanity, and specifically within the healthcare domain \u2014 I think the chances and likelihood of tremendous good outweigh the opposite. And yet the risks are asymmetric in their potential damage, and that\u2019s why I think all of us who are building these things have to be incredibly responsible about trying to examine, as best we can, and safeguard against current and future questions as we design these things. I\u2019d say I do worry, when I\u2019m in conversations with peer companies today, and even more so with the buyers of these solutions, about how few questions people are actually asking \u2014 if they understood the risks, and I\u2019m going to get to a few of them, they\u2019d probably think differently.<\/p>\n<p class=\"wp-block-paragraph\">So, you ask what I think the problems are that need to be solved \u2014 in no particular order \u2014 I\u2019d say the most pervasive one, based on how these things have broadly been designed \u2014 and this one is not true at Included Health \u2014 is that people, usually for reasons of speed or cost or both, have strapped themselves onto one of the foundation models without negotiating who owns the data and how it will actually be managed. Because the least expensive way to obtain a foundation model to power whatever you\u2019re working on is to allow that foundation model to train on everything you\u2019re telling it. That\u2019s a huge problem for healthcare \u2014 it probably violates federal law, or it will.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So let\u2019s just be clear \u2014 you\u2019re saying, I\u2019m a healthcare company of some flavor, I want to use AI, and whatever I\u2019m doing, I\u2019ll happily feed my data and my patients\u2019 data and my processes into the foundation model, and then, basically, the foundation model has it.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">You got it \u2014 it trains on it. And, by the way, it\u2019s training on everybody else\u2019s data as well.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">That\u2019s right.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">This is bad, because when you put personally identifiable human information into these training models, those models are absolutely open, and will use that to train on any number of other things and characteristics it would like to know about and associate with those human beings. And even if you\u2019re doing that on an anonymized basis, you\u2019re running the risk of training that data set in a way that can be undone \u2014 there are already public examples of this happening. People have deanonymized data \u2014 or, when you\u2019re uploading unnamed, but one-to-one, human-based data, it\u2019s easy to lock that into that human\u2019s profile forever.<\/p>\n<p class=\"wp-block-paragraph\">Because once it\u2019s inside the training data set, it\u2019s very hard, if not perfectly impossible, to pull it back out. I\u2019ll give you the most obvious example \u2014 I tell people I\u2019m a lot of fun at cocktail parties, but this is the one thing I tell people never to do: people are uploading their personal genome sequencing data directly into these consumer-grade AIs. You can\u2019t change your DNA, at least not today \u2014 so what you\u2019re doing, even if there\u2019s no name on the report, is telling that model everything about yourself that is uniquely you. That\u2019s a problem for how that AI could, in the future, price you for life insurance, or your likelihood of being a good mate or parent, or whatever. So I worry about that, because there\u2019s been this one-way flow, because it\u2019s been the economically easiest path to put data directly into these frontier models. Now, at Included \u2014 and this is the model I\u2019m about to publish an open letter on, because we want to make our approach publicly visible, hopefully as a template for others to use, but also to comment on and help us improve too, since this needs to be a multi-company approach \u2014 we\u2019ve built it so that not only do we anonymize data, none of it can be retained where we use external models. All of Dot\u2019s prompt engineering stays on our side of the house. All of our client data exists in a private cloud, controllable exclusively by those clients and the members who use them. These are really important controls for how data can flow. So the first one, broadly, is the topic of privacy \u2014 and where you\u2019ll see us go on the whole privacy-by-design journey is that we believe the member \u2014 this isn\u2019t yet live in the product \u2014 but we believe the member should be able to control all of that data down to the individual level, meaning, ultimately, we\u2019d like the member to be able to control what persists in their personal record. I think that\u2019s going to be a really big topic.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Before you leave that one, I think, from an optics standpoint, and probably from a user-control standpoint, what you\u2019ve just laid out is the most efficient and clean way of doing it \u2014 but there are a couple of things going on. One is, if the answer is, \u2018well, I\u2019m getting this now\u2019 \u2014 when I ask, say, how long Claude can look at my UCSF record, and is it a day, a week, a year \u2014 the most it gives me is a year \u2014 I\u2019m not sure I\u2019m that worried about what Claude is reading from my UCSF record. It\u2019s going to know a lot about me, for sure, but that\u2019s pretty helpful \u2014 it\u2019s giving me back very useful stuff, as we discussed earlier \u2014 and I\u2019m not absolutely certain I\u2019d want to switch that off. So if you\u2014<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, I don\u2019t think you want to \u2014 I get that example, and, if we could, I\u2019m going to abstract away from Anthropic and Claude specifically, because I don\u2019t actually know everything they do \u2014<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I did pick the model you\u2019re not in a relationship with, by the way.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, no, no, we work with them too, on certain stuff \u2014 but I want to say the thing I\u2019m pointing to is probably not the use case you have in mind. Now, if, in coupling your electronic medical record with Claude, that had express uses of that data \u2014 not only could it not retain data private to you, it couldn\u2019t even train and abstract certain concepts from your medical record \u2014 which I doubt they gave you. My guess is they are doing that, because that\u2019s a big part of\u2014<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I\u2019m sure they are, and I\u2019m actually not sure that\u2019s a bad thing<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, listen, I think you can have your cake and eat it too, is my point. You can have all of that insight, surveillance, and the ability to take next steps on your health \u2014 which is presumably what you cared about when you coupled with it \u2014 combined with the ability to retain sovereignty over your own data. I started a reputation and privacy company before this, and we spent a lot of time on this: all of us, as humans, are willing to give up privacy in exchange for certain things \u2014 we\u2019re willing to have data used to advertise to us in order to get free services, that was the starting concept behind Google as a search engine. But this is really different, because your health data is immutable \u2014 it\u2019s unchangeable, and it can be used to do a lot of things in the future. Frankly, it could be used to do a lot of things today that you might not like, or be fully aware of. And it\u2019s not clear that law and technology security have caught up with that use case. So my cake-and-eat-it-too moment is: why couldn\u2019t you glean all of that information, have the benefit of that self-understanding and discovery and access, and have the ability to say, \u2018okay, great, when I leave this session, or terminate my membership with this company, I want that data out of here.\u2019 I think we should build that \u2014 I think that\u2019s quite exciting.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I\u2019m just wondering if everybody did that, would the model not \u2014 would it become stupider, because it lost that data and didn\u2019t know things anymore? Because, remember, what we were saying 10 minutes ago, is that right now, a lot of stuff goes on in the world of healthcare that\u2019s germane to the patient experience and the clinician experience, but doesn\u2019t get captured anywhere, and someone\u2019s got to put that into the flow to make the medical AI better \u2014 like Dot, better than the generic ones. So I don\u2019t know enough to understand whether it needs to know what my imaging studies were from 2014 to be more intelligent about this.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I suspect it needs to. I think there\u2019s a bunch of data it\u2019s already getting that has certain permissions, and it can train off of what the provider system itself is using to ensure privacy, compliance, and safety. And I guess what I\u2019m saying is, all of us who are these next-generation health systems, if you will, have an obligation to think on those terms, and think about the power of when that data is beyond our reach. But listen, we\u2019ve only gotten through one of these \u2014 and I want to make sure I call out a couple of others, because there has to be a unified framework. So, another one \u2014 we know, because these things train on the open internet, and because they\u2019re highly biased by the data sets they\u2019re training on, that there\u2019s actually bias against populations that are served. I\u2019ll give you a couple of examples that have already been discussed, but you can imagine these getting deeply amplified. We know, for example, when we\u2019re looking at the prevalence of disease and using population-health algorithms to say those who are the highest spenders need the most intensive healthcare services, that there\u2019s a problem \u2014 whole populations are underrepresented in those data sets because they\u2019re not spending as much, because they don\u2019t have as much money to spend. That\u2019s a problem. A different example \u2014 this is all published, this isn\u2019t me pulling a case study from within our company \u2014 is that when you look at automated and semi-automated dermatology scans, they\u2019re largely trained on lighter-skin models, and miss things in darker-skin models. This is all changeable, but you have to be able to look at that and make sure that, when you\u2019re building your agents and your AIs, you\u2019re expressly trying to lean against any bias that comes into it.<\/p>\n<p class=\"wp-block-paragraph\">Of course, the scariest version of this, because it trains on the open internet, is that it will pull in unproven and questionable medical approaches, but deliver them in a plausible narrative that makes you feel good as the patient reading it \u2014 like, \u2018oh, I should inject myself with ivermectin, because it\u2019s going to work,\u2019 even though it doesn\u2019t, and you could be led to believe, by the AI, that it\u2019s true. So that\u2019s a huge one. And then the last thing I\u2019ll say, broadly, on the topic of safety \u2014 and I\u2019d point people who have time to read it to the piece we published with the New England Journal of Medicine Catalyst, with some other researchers \u2014 is that our approach to safety is human-in-the-loop: over a certain level of severity, in any interaction with Dot, there\u2019s a human reading it.<\/p>\n<p class=\"wp-block-paragraph\">That\u2019s more expensive for us, but we think it\u2019s critically important \u2014 that\u2019s also part of how we train Dot. We built a clever way to do switching on the back end, so people available on shift can actually take a look at these as they\u2019re coming through, making sure we\u2019re covering 100% of it. But that\u2019s how we\u2019ve trained our human-in-the-loop systems, because there\u2019s a real risk of hallucination, a real risk of injury. Again, if people want to read it, there was a pretty major finding \u2014 I don\u2019t know if you saw this \u2014 from the auditor general in Ontario, Canada, that a shockingly high percentage of the cases they reviewed, where AI was taking ambient scribing data from physicians and building care plans, had medications swapped, incorrect medication amounts, services recommended that weren\u2019t actually recommended, hallucinated recommendations for blood tests when people didn\u2019t need them. These are real concerns. The problem is that the AI\u2019s conversational style is so seductive that we can easily believe these things are happening correctly, and so we, as companies and stakeholders in this early movement, really have to commit to addressing it.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yeah, I think the question of how to deal with hallucinations \u2014 and hallucinations being generated not only from text, but also from audio, from voice, and in ambient scribing \u2014 continues to be something people are struggling with. I\u2019d say this is another case where, look, we had tons of mistakes before \u2014 it\u2019s not like the baseline was great.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">No, for sure \u2014 tons. The baseline was not great.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">But so I\u2019m optimistic about that. Having said that, we\u2019re also getting a bunch of case law starting up \u2014 like, can you record someone without their permission, who\u2019s entitled to the document. There was a case today in Washington state where someone was trying to get an ambient scribe transcript out, but the law said it belonged to the provider \u2014 the patient couldn\u2019t access it. Is a patient allowed to record their own visit? What happens if the patient\u2019s recording and the provider\u2019s recording don\u2019t agree? You already have \u2014I\u2019ve experienced this with my own medical records \u2014 stuff in the record that\u2019s wrong, and no easy way to correct it. This has been a problem; I think it gets amplified now, because we\u2019re recording directly into the AI a lot of things that just didn\u2019t get captured at all before.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Yep.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So I\u2019m with you on that \u2014 clearly we know there are mistakes, and we know AIs don\u2019t agree with each other. I had a ridiculous case today where I was trying to figure out why the PIN code on my computer had changed, and whether the AI had hacked it, or my daughter had deliberately hacked it, or \u2014 probably that one. Anyway, ignoring the answer \u2014 I did it on Gemini, I did it on Claude, and they completely disagreed with each other about what was going on. I don\u2019t know the back end of either,<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">But they were both equally declarative and authoritative! So I think the issue I\u2019m pointing to isn\u2019t so much that issues, problems, or mistakes didn\u2019t exist before \u2014 obviously you and I know very well that they did. What I worry about most is the complacency that will settle over people, just assuming that what they\u2019re getting is superior to what a physician was giving them historically, because I think we all have \u2014 perhaps insufficient, but some amount of \u2014 awareness and self-advocacy when we go into the health system, to make sure things are working out the right way. When we\u2019re told certain things by our friendly AI, we have to be careful. I had an experience trying to dose Tylenol appropriately for a kid who can\u2019t yet take the pill form, and the AI massively hallucinated on the math. It would have been a stupid dose \u2014 he\u2019d have been drinking two bottles of it \u2014 obviously I wasn\u2019t going to do that, but had I, it probably would have killed him. That\u2019s an issue we can laugh about, but there\u2019ll also be a sense of security, peace, ease of use, and efficiency, when you don\u2019t have that kind of medical-grade AI guardrail saying, \u2018okay, this is dosage, we need a human to review this, and we can get somebody on the line within seconds.\u2019<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So the drug-overdose one is a super interesting one \u2014 don\u2019t forget, this is the centerpiece of Bob Wachter\u2019s earlier book \u2014 the drug overdose at UCSF, which came out of Epic and a combination of the pharmacy, and it was a teenager, and there was a calculation that didn\u2019t get caught, very similar to what you just described. In that case they did give the equivalent of two bottles, and it killed the patient, and everybody was at fault \u2014 from the person who designed the system, to the pharmacist, the nurse, and arguably the patient himself should have known better \u2014 but in the end, it went all the way to that massive overdose.<\/p>\n<p class=\"wp-block-paragraph\">My sense is that we\u2019re going to get more and more comfortable with AI telling us the answer. The AI is probably going to hallucinate less and less, but it\u2019s still going to depend on guardrails.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Depends on how it\u2019s trained.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, I guess there\u2019s two things \u2014 there\u2019s training, and there\u2019s final oversight, and then, as you mentioned, the guardrails.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Exactly.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">How do you think that will play out for \u2014 including \u2014 how do you think it\u2019s going to play out in an actual case, like the Tylenol dosing example, or whatever drug it is?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Well, on this point \u2014 like most topics in business, and let\u2019s not forget these are businesses we\u2019re talking about \u2014 they\u2019re going to operate against their ultimate incentive set. We know, for example, thinking pre-transformer, about the original search and social media sites we gathered information from \u2014 those companies performed better on revenue the more they reinforced what we already wanted to believe and do. That was true because advertising worked, and we\u2019d spend more time on the websites. I\u2019m afraid the same will be true with these consumer-grade models, because people will be looking for confirmation of things they already believe, and that\u2019s just human nature.<\/p>\n<p class=\"wp-block-paragraph\">So, without picking on any of them individually, I just don\u2019t think it will be in their natural incentive set to say, \u2018no, actually, we\u2019re going to inject expertise and evidence and truth\u2019 \u2014 setting aside the whole political part of that conversation, which has been nasty the last few years around the role of science \u2014 I just don\u2019t think it\u2019s in their business incentive set to pause a conversation like that. However, companies like ours at Included Health, and I imagine many of our fellow travelers on this road, are going to say our incentive set is to deliver high-quality care \u2014 we\u2019re measured against those outcomes, we\u2019re also evaluated on whether members like and trust us. It\u2019s the conjoining of those two things \u2014 we have to actually be able to deliver the outcomes too. So health systems, both current and future-state, are going to be evaluated more on safety and quality.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I\u2019d say, to wrap this point, there seems to be a bit of back-and-forth about how you deal with this \u2014 your neighbor and fellow CEO, Jeff Tangney at Doximity, has put together an advisory board of the wise and so on \u2014 and I\u2019m a little confused and concerned about how you put guardrails in when you\u2019ve got \u2014 you were talking about human-in-the-loop, for you at Included \u2014 but there\u2019s only so much people can read, and sure, obviously the AI can spit things out at scale, you can put guardrails in, and hope, but then every session is individual. I\u2019ve had this conversation before too \u2014 back in the day, Mayo Clinic would write a learned tome, for both the clinical view and the consumer view, about diabetes, whatever it was, by their committee, and that was it \u2014 it got stamped, and now the AI is spitting a new one out, more and more individually, every time. So, can this work? Can we eliminate the hallucinations, the dangerous stuff, the telling-you-what-you-want-to-hear from AI? Can we put those guardrails in, the way you\u2019re talking about?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I know we can absolutely put safety guardrails in, because we\u2019ve already done it \u2014 we\u2019ve already flagged and bounced a whole bunch of safety incidents. Now, our approach so far is quite human-intensive, and that\u2019s true, and we\u2019re okay with that \u2014 that\u2019s our legacy, and we\u2019re proud of the track record we have on safety. Some of the other topics I raised, I think, are going to take even more work. But what\u2019s so critical \u2014 my call to action here \u2014 is that those of us who are building these things, already operating at the edge of the envelope, need to get together to work on the principles of what design looks like, because the existing law and regulation don\u2019t really contemplate a lot of these use cases, and what we will have to do by law, and what we should do, are different \u2014 and law ultimately catches up with ethics, but not as fast as any of us would like. So this is my plea \u2014 that we\u2019re going to start to organize a bunch of like-minded people to agree on principles, and hopefully show a technical blueprint, so people don\u2019t have to invent these things on their own, on how you actually do it.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">No, that sounds very sensible. Obviously the current administration has a very different view than where the previous administration was going, and who knows what it\u2019ll be like in a few weeks, after the midterms, or a few years after, if there\u2019s a change in administration \u2014 but that\u2019s a long time to wait for that to catch up. Right now, we have this early intention to cooperate among the big frontier models \u2014 I don\u2019t think we call it that \u2014 how would you say what you\u2019re talking about your call to action, is reverberating among your fellow leading AI companies?<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">So, the people I talk to \u2014 and I\u2019m lucky enough to be here in San Francisco, close to a bunch of the companies doing this work \u2014 I think they\u2019re not only open to it, they also sense it, and some of them are at different levels of their journey on the ability to deliver against it, but there\u2019s an openness to the conversation and a collaboration that\u2019s really exciting. We want to put together a group with a clearer point of view that can help educate and articulate that point of view to other people, and we\u2019re just starting the work to put that together. But this is the thing I wanted to bring to you today, because I think, in this week, where we\u2019re having this massive national and global questioning around the safety of AI, we have to look directly at the lives of the people around us that we\u2019re charged to take care of, and make sure we\u2019re doing all we can, in our specific corner of the neighborhood, to think through the risks of these things. And again, the risks pale in comparison to the benefits \u2014 we\u2019ve already seen it, it\u2019s going to be huge \u2014 and I\u2019m personally a user, I\u2019d recommend everybody use it \u2014 but I think doing this the right way is going to be meaningful to the future of our country.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">Fantastic. Well, I look forward to hearing, relatively soon I hope, what\u2019s going to happen out of the groups you\u2019re initiating. I think there\u2019s going to be more national conversation about this \u2014 clearly we need to be paying very close attention, both in healthcare and outside of it. We\u2019ve heard a lot, obviously, over the last few months about whether \u2014 I don\u2019t know, whether we can slow down. I do worry \u2014 and I\u2019m not the biggest fan of Scott Bessent and that crowd \u2014 but when you think about the Chinese \u2014 \u00a0this is also not just a healthcare or consumer issue, it\u2019s a national security issue.<\/p>\n<p class=\"wp-block-paragraph\">\u00a0Clearly we\u2019ve all seen the movies about the bad things that can come out of this \u2014 but there\u2019s a lot of great stuff that can come out of AI too, some really optimistic books about abundance, and clearly a lot that can be done in healthcare, not only in drug discovery but in fixing the way healthcare is delivered, which you\u2019re working very hard on. So we have to get it right, and it\u2019s obviously something we all have to focus on. I look forward to hearing back from you, Owen, soon, about how you\u2019re doing that.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Owen Tripp<\/strong><\/p>\n<p class=\"wp-block-paragraph\">We\u2019ll do it.<\/p>\n<p class=\"wp-block-paragraph\"><strong>Matthew Holt<\/strong><\/p>\n<p class=\"wp-block-paragraph\">I\u2019ve been talking with Owen Tripp, CEO of Included Health, and we\u2019ve gone deep into the world of AI \u2014 we\u2019ll talk more about this soon.<\/p>","protected":false},"excerpt":{"rendered":"<p>It\u2019s been a while since I talked with Owen Tripp, CEO of Included Health. They\u2019ve now introduced Dot their AI companion which had a big upgrade last week. We talked a little about that and I snuck in their video comparing the Dot Experience with a standard LLM. But the conversation really got into how&#8230;<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-15614","post","type-post","status-publish","format-standard","hentry","category-articles"],"_links":{"self":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts\/15614"}],"collection":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15614"}],"version-history":[{"count":0,"href":"https:\/\/medical-article.com\/index.php?rest_route=\/wp\/v2\/posts\/15614\/revisions"}],"wp:attachment":[{"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15614"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15614"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/medical-article.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15614"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}