My wife is a doctor and she already has an AI scribe that records visits, transcribes them, summarizes, and prepares her orders.
It seems like having that scribe also review lab results wouldn’t be far off, but we are still not ready to let the AI system interpret and inform the patient without human intervention.
The problem here wasn’t that a human missed the low platelet count, it’s just that it wasn’t reviewed yet. Both a doctor and an AI system would haven spotted the issue immediately.
I am not saying this isn’t a great result, but the AI intervention must come from the patient side, not the care providers side. The providers have too much at risk, while the patient is able to pick and choose their own AI and decide what levels of risk they are able to accept.
It doesn't need "AI" or a "Scribe" to spot a low platelet count in lab results either, computer systems have been able to do this for decades.
There's a systemic failure that a computer wasn't already flagging up the low count for immediate follow-up.
What I find interesting is that AI (or more specifically LLMs / chat-GPT ) has had such an impact it's broken down barriers from people who previously would be against any computer involvement or automation. People have gone from, "You must not automate anything" to "We should feed everything into chat GPT"..
And from someone who has in the past been interested in statistical modelling, it's dizzying. It's as if statistics were seen as boring and fallible, but "AI" is exciting and wonderful.
The main issue is that some developer has to be told to make it so that if plateletCount < 25.000 plateletCountColour = "bright red". That is, a big issue in software development is managing requirements.
AI is more flexible and can make up requirements on the go, that is, based on information it has it "knows" that a low platelet count is bad and what the recommended course of action is.
Of course, search engines and public websites do the same thing - "what should I do if I have a low platelet count" or "low platelet count and red skin dots" are valid search queries.
Most modern EMRs will flag results like this. If it happened to arrive in the middle of the night like the author said, the PCP would have just been asleep. I don’t think there are systems for waking up PCPs for concerning lab results.
And nor should there be. PCPs are basically by definition for non-emergent situations. If the PCP was at all concerned that the patient had something that was an immediate threat, they would have sent them to the ER in the first place.
Having someone on-call to deal with routine lab results that just happen to catch a possible imminent threat would be an enormous waste of resources.
This would be enormously expensive. The number of things an EMR thinks are an imminent threat will vastly outnumber the things that actually are thanks to liability concerns.
So now you need an extra on-call doctor just to filter out the false positives, or the on-call doctor gets notification fatigue and ignores them.
Or in the best case you send a lot of people to ER in the middle of the night for no reason. Again thanks to liability concerns, but this time on the part of the primary care practice.
I believe it's because statistics can be interpreted in different ways and requires analysis (work). "AI" just gives you an answer, even if it's wrong.
I think there needs to be a third option. Today I can plug a device into just about any vehicle and diagnose / debug tens of thousands of issues. This needs to be a thing for humans.
I should be a able to put a few drops of each of my bodily fluids into a set of disposable receptacles or strips and it should be able to test for hundreds of thousands of issues initially and millions of issues in the future. A pocket fictional Dr. House so to speak. This would distribute the testing workload to the people at home or work and give doctors something better to start with than only perceived symptoms and would allow prioritizing patients during triage. It should find everything from the obvious emergencies to the most obscure anomalies that could be addressed at a later time. It should also be able to tell me what to start or stop eating.
There should also be a high-risk-accepted mode where the device can make guesses about what is occurring based on all the data it gathered even if it does not have 100% of the scientific data to do so. All this in a sub $1000 device I can buy online without a prescription. This device must not have any way to leak patient data. Connect it to a printer via USB and it just makes a hard copy to hand to a doctor or they could just read this device with their eyeballs. No cell phone required. No wifi or bluetooth. No dial home. No uploading to LLM's. No cloud. Patients can download an updated OS and firmware image for updated medical data and update using a thumb-drive or micro-SD.
I said in another thread that I think that the first step can at least be that an LLM helps triage/prioritize the test results so docs can at least know an order in which to review them, especially with radiology/imaging results which are mostly words-based, rather than the numerical results that often come out of blood tests.
I can't tell if you're trying to say that ChatGPT hasn't added anything here?
Yes I'm sure a human doctor would have noticed the low platelet count -- if they bothered to look at it. Which they didn't (or at least hadn't got around to yet).
Sorry, I edited my comment with a final statement. I think chatGPT added a lot and I think the future of AI in medicine is with patients bringing their own AI to help interpret the results.
On the provider side, it might be useful to automate the analysis of various patients, particularly ones who have elusive diagnosis. I hear stories of some patients who sent from specialist to specialist trying to figure out what's wrong with them where the symptom is obvious but the cause is not. I had a colleague who had vertigo and it took 6 months to figure out what was going on. She couldn't work or drive. AI/ML might help in that scenario on the provider side. I understand it's pretty good at analyzing imaging as well.
If the patient agrees to it (and is informed of it), I can definitely see value in automatically recording, transcribing and logging a visit (as long as the doctor does a final approval / editing phase). Part of me wants to enable that by default in some of our Teams meetings too because for some reason they often end up chaotic and emotional (enthousiastic / energetic, I mean, instead of businesslike), and it's difficult to bring it back to some core points.
Her legal team said they are allowed to get verbal consent. She has a sentence she says verbatim something along the lines of “ I am going to use this computer to record this visit and take notes. Is that OK?”
If I heard that, I would think the person asking is going to take the notes, not feed the recording into a LLM.
I do NOT want a transcription ERROR in my medical records.
I bet if she says something like "AI will transcribe" then there will be more hesitation, although maybe not more refusals because most people don't know how it can make mistakes.
Anecdotally, she has reported it’s _much_ better than her human scribes. And she is still ultimately responsible for the content and reads it all before signing the note.
I believe so, yes. This is nothing new however as "tele scribe" companies have existed for a long time. These sort of cases are exactly what HIPAA regulations were designed to enable.
That sounds like the old trick of pulling out a pen and paper and saying "I'm going to record this, okay?" with a tape recorder in your briefcase.
Are they informed that their personal medical information will be shared with an unknown set of third parties / LLMs subject to manual QA / cloud services with telemetry?
Honestly sounds like she's opening herself up to massive lawsuits (depending on State law). I would reach out to the legal department if any doctor pulled this kind of shit on me.
If you think the clinic's lawyers haven't already reviewed this setup with a fine tooth comb then you haven't seen how medical lawyers work. This isn't a rogue PCP surreptitiously recording visits. Its a contractor that was selected and negotiated with from on high that was subsequently deployed to all clinics in her large company.
Also, I guarantee you signed a paper when you first were enrolled with your PCP giving them permission to share your medical records with third parties when needed (subject to HIPAA compliance of course).
Don't see what an LLM really adds here that wouldn't be served by a sensible algorithm which flags up results needing urgent action.
Maybe the end user get more clearly explained and non-bewildering information, which is useful, but also points to a failure of any health system / internet resources to provide this.
The LLM _HAS_ empowered the patient to act on his own health, which is undeniably a good thing.
It seems to me what the LLM adds is that it is the sensible algorithm which flags up results needing urgent action in a growing set of everyday domains.
The human like conversation urging to go to the ER was particularly useful in this instance. I am not saying I don't fear for it's mistakes but clearly it's ability to communciate in natural language is something to not take for granted.
Indeed the victory for the AI here is the ability to convince the author to take immediate action. The second win for the AI is the concise one liner the author tells the triage nurse so that her further diagnosis and treatment may proceed. A regular human being would be likely to start with a long form story of not feeling well for some time.
To be fair, This is what AI is good at. No matter what problem you're trying to solve, AI will provide a solution.
In this case, lab result analysis was entirely based around standard care protocols that LLM is really good at regurgitating back to you.
Yeah sure, anyone could probaly promt feed this into a homeopathy solution which would have killed the patient as per their own predisposed will. You could argue a standard programmatic diagnostic evaluation would be better but its not entirely clear wether human decision making improves based on webMD above an agitated Chatgpt urging you to seek professional help.
And now that I think about it more, no one except doctors and AI could reliably improve diagnosis based on a picture and lab results side by side. So if your healthcare system doesn't provide a response then chatgpt might actually be the closest neighbour in time of need.
It's not an either / or though, this is a generalized LLM (app being a chatbot), there's also specialized LLMs for things like writing code, prose, etc.
They are open source in the way Android is open source. The actual results are open source indeed, but the culture and the chain isn't, so, if the people who run the projects decide that they are not open anymore, they won't be. And so, the projects will stop, as there will be nobody to release the newer versions. Same as how Android could maybe maintained indefinitely, but not improved in a way that it is right now.
All competitive open models today share a common property; someone spent a large amount of money to train them and then released the model for free.
I don't understand why the argument continues to be we will have a rich ecosystem of base open source models; unlike opensource ai which is individuals donating time, opensource ai requires someone to donate very large amounts of capital.
A friend of mine had gotten an imaging test done recently, and had to wait 7 days to have her doctor review it. Meanwhile, she got the report and was paralyzed with worry by some of the words (like high risk for ____, etc).
I feel like this is an area where AI/LLMs/whatever can actually help triage test results to tell doctors "hey you really should look at this test result first, it looks concerning" - this would happen in a way that you can't do with other bloodwork (which returns numbers on a scale).
Most radiology reports are just written words, so LLMs could potentially use that to triage and help alert the care provider to prioritize it, so patients aren't sitting there worried for days on end until a doc analyzes it.
Arguably the issue in your friend’s case is providing the patient with a document that’s not designed for them. Radiology reports are written with a particular audience in mind.
They are also written with a particular medicolegal perspective, which the intended audience will appreciate.
I often give my patients copies of their results/reports, which gives me an opportunity to explain them.
If we are going to send reports to patients directly, maybe they need to be written differently? Who should pay for that different report to be written/checked?
We already have AI tools for prioritising the order some tests are processed in (e.g. which X-Rays the radiologist looks at first). But once the report is written the author has a responsibility to ensure any urgent result is escalated. As a requester (I order tests), if I result needs urgent action within days or less, I expect a phone call to myself or another doctor in the practice.
I totally agree that we’re not the audience. But when a patient gets a result and their doc doesn’t call them for nearly a week, the patient is left to fill in the blanks in the worst way possible. In this scenario, the practice screwed up by taking so long.
Question to which I don't have the answer: does an instant LLM response that says you may have cancer, infection or some other malady based on a report alleviate your suspicions? Or would you rather wait for a measured discussion with a professional? Personally speaking, I might be fooled into contacting my doctor inappropriately citing a false positive from DrGPT which seems inefficient.
I know there are a lot of conversations going on about the dangerous elements of LLMs, but it’s nice to read a story like this alongside them — it’s a reminder of the remarkable potential of this new technology.
Could you have found out somehow else about the severity of the problem? Maybe, sure! But the fact that you had this LLM to ask, and it so easily understood whatever info you gave it, asked clarifying follow-up questions, and gave you info/directions in the way you needed to hear them — it’s such a new way of interacting with computers, almost like an API but made for humans to use, and cases like this is where that value shines most clearly IMO.
Thank you for sharing this feedback! Yes, the big unlock for me was using the LLM to explain things to me on my own terms and that's what ultimately gave it more persuasive power over even a simple (non-personalized) google search
Oof, you don't go down to zero platelets without something gone seriously off the rails.
Seems like it got caught right in time, but wishing for the best in the follow-up.
This being said, a problem with blood tests is that a layperson won't know right off the bat if a value outside of the normal range is ok or a life-threatening issue.
I understand it might introduce a lot of complexity for labs and that there's some level of "big picture analysis" involved that will require some human intervention, but even just a color-coded line for WBC/RBC, hemoglobin and platelets could already go a long way, as OP's case demonstrates.
These have existed for decades, although it might not always be surfaced. My provider gives me the reference range with every blood test and a red line for abnormal so I can know immediately where to start looking.
People will not know to ask about platelet count. She had to tell the AI what she was seeing, and then upload a picture to get a more precise diagnosis and recommendation. You can't upload a photo to google and get a similar result. You will get lots if similar photos maybe. And even if you could, the UX would be terrible. Combine that with having AI on your phone and being able to just take a picture, or talking live to it.
Anyway, this comment reads wildly similar to the dropbox comment.
> People will not know to ask about platelet count.
I would. You would.
> Anyway, this comment reads wildly similar to the dropbox comment.
My point was that this isn't revolutionary/interesting. AI w/ camera can detect apples that have bruises, but AI's cousin Vision/ML has been doing this since at least the early 2000s.
Further to my point was that while yes, using all of the ways the author went about identifying the issue (picture etc) AI makes for a nicer UX, it wasn't required, nor was the invasion of privacy into the author's medical history required.
The author torched a forest to figure out their issue when burning the paper mill would have done the trick.
I would've had no idea prior to reading this article.
There seems to be real value in the LLM being able to probe with relevant diagnostic questions, then picking out "Your platelet count is alarmingly low. You should go to the ER urgently." from the three lab work panels uploaded. Even if I had properly gone through those results myself at that point (not a given, by the timing), the level of danger seems very easy to miss.
> My point was that this isn't revolutionary/interesting [...] it wasn't required [...] The author torched a forest to figure out their issue when burning the paper mill would have done the trick.
And you can do anything a computer can with sticks and sand.
If an LLM is the best tool available (that isn't a given, but it does seem to have done well here) I don't see what makes it the equivalent to torching a forest. Its size? Energy usage? More capable than strictly necessary? Maybe factors I'd consider if designing, say, object detection for a low-powered doorbell camera - but not really things I'd worry about in what's potentially a medical emergency.
I don't know about this specific result but I've uploaded photos to Google Lens with good success. It's not an LLM or generative AI so it can't generate a response, but it's actually really good at identifying things in an image.
I agree with you about the Dropbox comment though. This whole thing is about having a polished user experience rather than bouncing between Google searches, Google Lens, and various other websites.
Yes, that's a serious ux issue in modern test results. Instead of just a range there needs to be some kind of variance score. It can be presented simply like, 10% out of range, or, 20x lower than normal. Hell, even just a second range that says "critical, seek medical attention". Then you can judge how close to that line is worth going for. There just should be more information. It's all computers nowadays anyways, they could print thousands of words, send me a video explanation, they could do anything!
She already knew that something is not normal in the test result, so she could just have typed the text from the result in google but sure sending a picture is faster on a mobile device.
Btw I agree AI can help, my issue is with greedy people salivating for money while attempting to do "AI doctor", "AI teacher" "AI physiatrist" ,
this important AIs should be build by non profits and correctly checked by competent people for safety. Because of $$ reasons this AIs will be marketed with more abilities then they have and the downsides will be hidden, again because of $$
While I agree in principle. She didn't know she had low platelet count. The AI spotted it after it asked for blood works.
This should become standard. When blood works or whatever exam results are available they should go through some kind of automated system that can flag critical situations like this, without waiting for a doctor.
When I go for labs, I always am notified via txt/email that the results are ready. Every lab I've used has a scale system for a given test along with notes from the lab techs about a particular result.
While this might not be standard everywhere in the world, and probably isn't, if one was concerned about their health and had access to their results themselves, wouldn't you look at them?
This reminds me of the redditors who post questions about mundane government services where it would be quicker to simply call the agency in question and ask.
I know what you're talking about, and the last lab results I got contained more data than I've ever seen before on one of those things. There were one or two results outside of the normal range, which my doctor addressed with me directly, and now everything is mostly normal.
Are you saying that the doctor failed to take action on something that was at a concerning level on her labs? Or that she should have scanned her labs for anything that was on the edges of those normal ranges?
I generally agree that more data is a good thing, but simple blood work can be kind of a large amount of data now that's not immediately recognizable to the average person. However, this is exactly the case where I want more assistance with AI... there's simply too much data for me to reliably tell what's meaningful and what's not anymore. AI can process a bunch of data and tell me what I need to know in about a second or two.
Wow, I'm glad you figured it out and I bet those two weeks were hell. With anything health related, I think information found online and information from AI tools should just be considered helpers toward determining something more specific that you can address. I certainly would not vouch for all AI assistance, and you shouldn't just give more weight to an AI system more than an article from a supposed expert necessarily. But, if you know you are working with good tools (like Claude Sonnet, which I would recommend for this particular case we're talking about) they shouldn't steer you too far in the wrong direction. Claude specifically has been the gentlest, most helpful general assistant I've ever had, but I don't ever lose sight of the fact that these are simply tools helping me, a non-perfect human with a lot still to learn, and I wouldn't recommend using them as any kind of "authoritative" solution, just something that's about the best help I've had with this kind of stuff so far. The final decision should be between you and your doctor.
> Are you saying that the doctor failed to take action on something that was at a concerning level on her labs? Or that she should have scanned her labs for anything that was on the edges of those normal ranges?
From the post that her doctor hadn't yet reviewed the labs and the turn around time for the doctor was 2 - 3 days. I'm naturally curious about my own results so I do look at them and search the WWW for the meaning of the results if the lab tech doesn't provide any contextual clues. If WebMD says I have Lupis and I'm going to die tomorrow, then I'll reach out to my doctor in some form or fashion.
> simple blood work can be kind of a large amount of data now that's not immediately recognizable to the average person.
You'll find every [common] blood test has descriptors on the Intertubes from a variety of sources. Insurance, in the US anyway, isn't going to allow niche complex tests without going through weeks of denials. And even when doing standard blood workup, you're not going to need to know the items in the green, just the one or two out of range.
You're of course free to allow AI to invade your medical records as you see fit if that is the way you want to go about it. My point above was that "AI" wasn't necessary nor revolutionary in respect to identifying the author's medical issue.
> This reminds me of the redditors who post questions about mundane government services where it would be quicker to simply call the agency in question and ask.
Wow oh wow does this depend on the agency, and depend on the question, and assumes you even know how to ask the question the right way. And I think considering that, the analogy operates quite well. People often don't know how to ask about medical things, they just describe symptoms ("platelet count" probably isn't a highly googled term for self-diagnosis). People do similar things when they have a problem with government and don't know where to turn, may not know which agency to contact, or how navigate that agency's phone trees, etc.
> it would be quicker to simply call the agency in question and ask.
Depends on the service / agency. Some have long wait times.
Heck ChatGPT might be quicker than both reddit and the agency at this point, and I'm not an "AI bro" by any means... But I gotta accept when its usefulness actually works.
These systems likely vary around the world, this is an Australian perspective.
> When blood works or whatever exam results are available they should go through some kind of automated system that can flag critical situations like this, without waiting for a doctor.
We already have this. Based on either individual results or how results relate to each other.
Labs will detect critical results and then a range of things happen:
- a scientist may repeat or check the result if the value is extreme, to ensure it isn’t an error
- a scientist may alert a pathologist to the result immediately
- a scientist or pathologist may contact the referring doctor/hospital by phone to provide the result immediately
- in some cases, the lab will contact the patient directly (if they need to go directly to hospital for example)
This also gets done with radiology.
This then gets followed by the official report in hours to days depending on how much confirmation is required.
Sometimes what’s “normal” is highly dependant on other factors. In that case you might get a table or other comparative information provided.
That said, some results are still concerning if they’re “high-normal” or “low-normal”, and some results are not that worrying even if they’re slightly abnormal.
So I couldn't find the specific post I was thinking of off hand, and there seems to be about a dozen or so hits on various forum-type sites like Quora/Reddit etc. So... yeah I don't know, maybe it is real? But there has been an instance of someone using an LLM to make a story about an LLM saving their life. So maybe this one is real. Maybe it is not? Who knows. Reality is kind of mushy these days.
Additional notes here, the author of the article in this topic clearly indicates on her page that she writes alongside "Taylor Script", her AI counterpart. So now the line is even more explicitly blurred. An AI definitively was involved in the creation of this content... so now the question is, to what degree does the content relate back to real world events? Only the author knows I guess.
I read this post with my wife (medical doctor) and we agreed that this breathless post missed the mark a bit. ChatGPT gave exactly the advice from a Google search for low platelets and rash. A call to any doctor, nurse practitioner or urgent care would have resulted in the same outcome with arguably more trust. The fact that the author trusted ChatGPT more for some reason (AI hype?) seems immaterial. The hyperbole of statements like a minor cut with low platelets = death or having "no platelets" (both false) solidified it - hype over substance.
Google for "low platelet" sure, if you already know that's important which she did not. The hard part here was entirely noticing that there was an urgent problem.
As multiple people in this thread have commented, most providers give you a result, a red highlight of the abnormal result, and a reference range. We don't know exactly what the post author received of course, but it's pretty likely that low platelets were called out.
You might as well piss on the floor, take a picture of the puddle, upload it to chatgpt and ask it if you'll develop cancer between 2033 and 2037, you'd probably get the same amount of hallucination.
Well known that 101 type rules are learned to be simplified near-lies in 301, and possibly completely untrue in 501. We shouldn't hamstring ourselves on the fear that the idiot will hurt himself with the tool. It's up to you to determine if you're the idiot or not, and then how to delegate to others if so. This type of cooling/censorship is strange to me. These are the exact strange and beautiful usecases we were all excited for aren't they? To expand the capability of the average human, to greatly expand the effectiveness of the intelligent users. Obviously the damn thing can lie! We all know that! The stakes presented by chronic illnesses go far far beyond "scared to hear a lie" and tend to land in "please god save me i'll do anything". People usually have _already tried everything at their disposal_. What's it gonna do, tell them to walk in front of a bus? Every medical conversation I've had, it repeatedly tells me to consult human doctors. And I've had so many useful interactions that it seems to me your philosophy would completely occlude. I'm still figuring out serious problems i have. Things that don't get appointments for months because I've been told I'm fine. Of course a hypochondriac could go insane, just like they might with the DSM-V. It's a lot different when you have to skip work because of constant nausea, or when your productivity tanks because of frequent heart palpitations, or when your child is losing too much weight. I take very harshly to being told i can't use a tool to help myself because you think I'm too stupid to not hurt myself.
This is peak technosolutionism and it sounds like a cult or a religion at this point. It's all about believing, "expanding" the human mind and other woo woo shenanigans.
LLMs aren't an oracle of truth, science and medicine can't solve everything. It's OK to be sick and not have answers, if you've "done everything" and the experts don't have an answer chat gpt won't automagically summon an answer from its statistical soup. It sucks but it's a fact of life, we're not gods, and neither are llms.
If uou are in the USA, you skipped the step where the Genetic Information Nondiscrimination Act of 2008 and (depending on state, but they are common), the state law and/or insurance regulation with similar effect is repealed.
If you create a custom gpt, it’ll will delegate to a custom gpt-4 model, “a specialized GPT-4 variant optimized for advanced health, genetics, and performance optimization.”
In my experience, LLMs have a hard time processing large amounts of raw data and finding patterns (even simple univariate time series) so I would be surprised if a GPT could find _real_, non-hallucinatory patterns in raw data.
From what I recall from a bioinformatics course, I really doubt you can have your DNA fully sequenced for just $200. But, prove me wrong, do you have some resource saying otherwise?
Nebula offered specials for full genome sequencing.
They’re now owned by DNA complete.
DNA Complete prices it at 495. I paid half that for 30x.
“By analyzing your DNA multiple times, we can significantly increase accuracy. 30x Whole Genome Sequencing means that 100% of your DNA is sequenced an average of 30 times, thus providing even higher accuracy. Typical DNA Ancestry tests sequence less than 1% of your DNA.”
Yes, compute is cheap, but it was the actual lab procedure that required expensive equipment while also being time-consuming (although I don't remember the time required, maybe it wasn't that much). I remember the professor mentioning an upcoming machine by Illumina that was soon be able to sequence the DNA without cutting it into pieces, this is probably part of the cost reduction as in this way you wouldn't need to run a multiple alignment algorithm to reconstruct the sequence from the pieces. Thanks for pointing at this resource.
So is dang/HN in general good with the several comments in this thread giving explicit medical advice that is some variation of "don't go to the doctor, use chatGPT"?
I'm not sure the value of the discussion around those comments outweighs risk of spreading/normalizing that idea.
I'm highly interested in your story/diagnosis. Did the dna panel give you crucial information in this journey? Do you have a write up somewhere on your process? I'm currently going through a very complicated and subtle heart issue and I'm looking for any source of data I can get.
Eh, i think it's fine. It isn't like the teeming masses are gonna run out for dna tests that they spend hours formatting and selecting and then fed into a complicated extraction process involving careful prompt engineering. I think the danger in recommendation lies almost entirely within the space of "easier than going to a doctor", which this process certainly is not. You wouldn't do that for a cold. But god damn am i glad that there's people talking about this. I have several chronic and subtly terrifying health issues myself and this information can save lives. What alternative do you propose, that we should not freely share our methodologies for survival?
Unpopular take: ChatGPT is better at diagnosing medical problems than a doctor who does a cursory evaluation.
By cursory, I mean an evaluation that does not take into account a very thorough assessment of all the evidence. P(condition | evidence) is not very accurate when the evidence is vague and sounds like many other conditions. It is a rare doctor who spends more than a few minutes asking about all possible symptoms, even those that may not occur to the patient that they have. Such doctors certainly exist, and I’ve had a few, but for the most part I have to do my own research and present it to a doctor for it to even be considered as a possibility.
As anecdata of one, I’ve been having all sorts of weird rashes that come and go. Could be eczema, lupus, psoriasis, who knows. Googling makes you think it could be anything. So I took my entire medical history, all medications, all symptoms, their frequency, etc. and threw everything into ChatGPT with the prompt “Give me a list of possible diagnoses ranked by likelihood based on the evidence”. At the top of the list “Mononucleosis (EBV induced)”.
Ah, but I forgot to mention that I had mono 12 years ago. I can’t catch that twice. So I told ChatGPT that I had it previously. “Still mono. Possibly recurrent due to some problem of the immune system.” So I went and got an EBV antibody panel. Positive for all three, including the EBNA test which is positive if you ever had mono previously.
It blew my mind that a LLM sussed this out, because mono wasn’t even on my radar.
It must be wonderful to live in a place where that's an option.
Due to my partner's work, I live in a town of about 100,000. There is a doctor shortage - New patient intake can be months. The system hangs on with physicians' assistants. Going to a specialist often means months of waiting or hours of travel.
We are living in a degraded health system between the lack of doctors, burnout of doctors and nurses, and a private equity system buying up everything in sight and sucking out every last dollar it can.
I think there are two ways using LLMs for medical advice are dangerous: 1) Treating oneself after getting bad advice or 2) Not getting treatment at all because an LLM thought a critical condition was normal.
But the article and the comment both describe situations where they talked to a physician first, went to ChatGPT to refine their understanding, and then went back to a physician. Seems like a good use of LLMs.
I agree with this. Any coder can tell you that AI will write some decent code, but it takes (at least at this point in time) an experienced coder to tell "this is crap code" or "this is decent code" – find a better doctor, because it's going to be hard to tell whether the advice it's giving you is the equivalent of crap code or good quality code.
I guess I don’t understand how it’s dangerous. I’m not giving myself any kind of independent treatment based on the output of an LLM. But it provides possibilities that doctors may have overlooked. Those doctors can then empirically assess the possibility in detail to determine whether it is correct.
The danger is on the other side, you give the AI all your symptoms, it tells you you're fine, nothing is wrong, don't do anything. And then you ignore it and don't go to the doctor and fall over dead.
Unlikely, probably. ChatGPT aims to please.
There's also the risk that it recommends something that is in appropriate, but within your ability (like over-the-counter meds that shouldn't be used in your condition).
But - all this stuff happens all the time anyway. Most people don't involve a doctor until they encounter something really wrong that they don't recognize. They're more likely to mention it to someone else offhand and may take whatever recommendation they get.
> The danger is on the other side, you give the AI all your symptoms, it tells you you're fine, nothing is wrong, don't do anything.
It goes both ways. Replace "you give the AI" with "you give the doctor" and the issue "nothing is wrong" does not magically disappear. As a matter of fact, many doctors I have spoken with couldn't diagnose a thing and I wouldn't trust them.
One of them googled about the potential symptoms of the disease in front of me because he had no familiarity with it. And that's fine I guess since humans cannot know everything? But this only reinforces the point that LLMs can definitely have their place in assisting people, and their doctors, with diagnosing their health conditions.
Imagine you have a doctor who has read through literally all the medical books, documentation, research topics, experiments ... whatnot. Human of that kind does not and simply cannot exist. I think not many people understand the value and potential of LLMs, no matter how much this particular topic would be seen as sensitive or controversial.
I think parent comment is suggesting it's dangerous for you to publicly tell others that ChatGPT is better than their doctor at diagnosing them (however little of your doctor's attention you get) regardless of whether there is or isn't truth to it in certain cases.
People can't know when they should/can trust an llm over a doctor. People will also misuse LLMs in the context of medical diagnosis where asking the right questions and examining the right places (and properly) can be critical.
People have biases, people can have psychosomatic symptoms, people can be easily influenced, etc.
> I think parent comment is suggesting it's dangerous for you to publicly tell others that ChatGPT is better than their doctor at diagnosing them (however little of your doctor's attention you get) regardless of whether there is or isn't truth to it in certain cases.
> I’m not giving myself any kind of independent treatment based on the output of an LLM. But it provides possibilities that doctors may have overlooked.
You might not, but others would.
I mean people take random advice from the internet all the time. IIRC There was a whole craze about ingesting or injecting bleach to “cure autism” or something.
Random advice from an LLM is no better.
Even worse, maybe, since there’s not going to be a bunch of negative replies to whatever the LLM outputs.
Depends on how you use it. Just using LLMs is probably not good but LLM + seeing the doctor is quite likely better than just seeing a doctor occasionally.
At least LLM advice is normally quite sensible compared with a lot of the junk advice out on the internet.
This doesn't seem relevant to the post, rather an anecdote where you confirmed that you had EBV in the past with the help(?) of ChatGPT (which you already knew). I don't see where you diagnosed the cause of your rashes.
I didn’t. I followed up with a doctor and mentioned the possibility of mono who then performed the EBV tests and provided the official diagnosis after previously seeing a different doctor who incorrectly diagnosed the problem as eczema.
> New bruises without injury = your blood is not clotting properly
I can't see any mention at all of bruises being a symptom in the article, did I miss that bit?
Also if you're going to fill an article with ai-generated images, can't you at least photoshop in some correct text instead of getting an AI to try and writing the text for you? "Honestly? chatgpt convunced me." isn't awful, but "Ancôoidions, your instructions." is completely incomprehensible to me
The 6th image on the page, of a woman lying in a hospital bed, has a speech bubble with garbled ai-generated text inside of it, and the next image has a classic-style walking robot, half submerged into a rug, saying similarly garbled text
I don't have an expert medical system from the 1960s and an IBM 360 to run it on. Or a checklist [that I know is relevant]. Or a reference book [that I know is relevant, and know where to look in it, and what to look for].
Only as reliable as the information contained in the system. The whole point of an LLM is to agglomerate as much information as possible with little manual intervention which causes them to both be a) far more intelligent and knowledgeable than any old-school symbolic expert system, and b) also more prone to biases and hallucinations.
Lots of gushing about AI but I'm not really too interested in that.... more about what's up with your missing platelets! I'm assuming it's not at all a good thing to suddenly not have any without any prior history of things related to this? Hope everything works out okay in the end :).
If this story narrative is true as described in the article this is a ground for reporting the primary care physician to the state board for a review. I am not saying it is a malpractice per se, but a clear fail of the doctor's practice. All modern EMR systems (most popular in US is Epic) _automatically_ flag all the results that are out of normal range. A physician assistant should be alerted to this fact and take appropriate actions. Sitting on ER grade emergency for two days and then telling a patient it will take another 3 days to review the results is unacceptable.
I disagree. Doctors can't be expected to be available 100% of the time. The only doctor to see this patient's result was the pathologist (or should have been before being sent off to the requesting doctor). It had not yet been handed over to the requesting doctor in a fail-proof manner.
The pathology company I send my samples to contacts my clinic to ensure we received urgent results. This it's important because there's no guarantee the message ever arrived in the doctor's inbox. If you work in tech you should know this to be the case.
The only way to ensure timeliness is for the pathology company to contact the patient directly if they can't verbally hand over to the requesting doctor.
Yes it did. Let's not only praise ChatGPT because it said the right thing at the right time. ChatGPT told you how to stay alive, and the ER team did the work to keep you alive. Both systems were critical in the save.
Unless they are advertised as medical practitioners, I don't see how they could possibly be liable for bad medical advice. You can't sue your friend or dad for bad medical advice. Maybe you can if they claim themselves to be a licensed medical practitioner you'd have a shot at a successful lawsuit.
You don’t have to claim you are a licensed medical professional to be sued for medical advice that harms someone. You just need to present your advice in a way that a reasonable person would assume that you have some kind of medical expertise.
Based on the confident manner with which all LLMs present their answers and based on the fact that people are clearly using LLMs for their medical expertise, I think it would be pretty easy to go after one for giving medical advice that caused actual damages.
Yes, it is beautiful that someone can potentially self-diagnose with ChatGPT* - but if you're worried about your health, you really shouldn't need to use it and just be free to go to the GP or GPC (outside opening hours) without worrying if you are going to have a debt for the foreseeing time.
* with at the same time having this data nicely collected for OpenAI to boost further training
I've been re-watching House lately and I can't help but constantly think that a LLM would be excellent at diagnostics. At least in the sense of, "the patient has displayed XYZ symptoms, the test results have XYZ data, and ergo the following conditions are likely. Especially in the more obscure interactions and second-order effects.
Medical experts have done teardowns of House episodes and most of what happens in the series wouldn't happen in real life, that is, diagnoses are better than how they are depicted there.
Oh yeah I'm sure it's not super accurate, I just mean the specific task of diagnosing a patient would seem to benefit from a LLM-like piece of software.
Possibly but in my experience LLMs are more useful for brainstorming and generating original ideas than “being accurate” all of the time. Hence my idea that they could be useful in diagnosing difficult conditions.
Yeah, it's a trade off between certainty and creativity. Constraint software would be all certainty which, if it contained all the required information, would be the perfect system. But we don't have "all the information", or at least i haven't seen anyone claim so. So some degree of out-of-box creativity is required to catch more obscure correlations and strangeness. I think AI is just about as certain as previous systems, but with a big boost to the creativity aspect. It's also free. I can imagine the omnidiagnoser would be a closely held trade secret worth billions and thus absolutely not free to the public.
What the author should have done, as soon as they saw the physician had not yet looked at the results, is to immediately go to a pharmacy and show them the results. I’m glad AI has been useful this time, but pharmacists are experts too (and would tell you if they are not sure of something, instead of confabulating).
According to the story, the ChatGPT conversation that led to the ER visit happened on a Sunday. In my part of the world, all local pharmacies are closed on Sundays, so going to a pharmacy and showing the results would not have been an option.
That’s interesting, that wouldn’t occur to me at all. Is that a thing where you live? Going to the pharmacy to get diagnosed? (with something more complicated than a runny nose)
> Next, ChatGPT suggested I ask a physician for some bloodwork. That’s when I remembered—I’d already received my lab results (unceremoniously, via email) late last night. One result was abnormal, I mentioned to ChatGPT.
As a non-native Ensligh speaker, I asked ChatGPT what the ER acronym means: Emergency Room (I guessed E was for "Emergency" after reading your story, but I didn't know about the 'R').
AI is also helpful to parse the story about how AI is helpful.
You can save one life for a lot cheaper than the countless billions poured into OpenAI. This might be the least efficient life-saving intervention in history, and that's before calculating for harm caused.
It costs UNICEF about $4 for a single malaria vaccine[1].
"Globally in 2023, there were an estimated 263 million malaria cases and 597 000 malaria deaths in 83 countries." [2]
" In the 20th century alone, malaria claimed between 150 million and 300 million lives, accounting for 2 to 5 percent of all deaths" [3]
There's a widely claimed factoid that Malaria has killed half of all people ever born, and while that might be true, there's plenty of debate you can find about how accurate that claim is. In any case, Malaria is one of, if not the, most deadly infectious disease in human history.
We could have wide-spread elimination of malaria transmission now. We could eradicate malaria with further developments. [4]
> IMO, saving one life justifies every cent invested into OpenAI.
Common reporting says openAI spent $5b last year. That's 1.25b vaccines. The population of Africa is 1.5b, and Africa has the most malaria cases of any continent. So you could make a really, really big impact in that 263m cases/597k deaths per year just by investing in something that already exists.
>We could have wide-spread elimination of malaria transmission now.
The vaccine doesn't work that well. As your article [4] says
>The highest impact will be achieved, however, when the vaccines are introduced alongside a mix of other WHO-recommended malaria interventions such as bed nets and chemoprophylaxis.
> The vaccine doesn't work that well. As your article [4] says
It does not say that.
And in any case, as we should all very well know by now, vaccines don't have to be 100% effective at disease prevention to be effective at disease spread reduction, especially at scale.
But, lets grant the point: Fine. Use some of that $5b annual spend to fund other preventatives.
Oops bad punctuation - I should have had a colon. The text after the > is cut and paste from the article. From wikipedia "fourth dose extends the protection for another 1–2 years. The vaccine reduces hospital admissions from severe malaria by around 30%" So it's a helps a bit situation rather than just vaccinate everyone and malaria is over.
My Mom was in the ICU back in 2024 due to lung and heart issues. I wouldn't say I blindly followed what GPT suggested but it definitely helped to understand lap reports, ECG, and other imageries to understand doctors' lingo
AI is extremely good at finding patterns quickly. Even without AGI, this will be extremely useful. I kinda feel like AGI and replacing developers is a distraction to the power AI/ML already has available.
This is why we need AI to be open. Imagine if ChatGPT had forced her to watch a five-minute advertisement before each query – which is going to happen in the near future.
the really sad part here is that even without any fancy AI a lot of unusual blood results can be reliably detected and flagged as urgent or emergency
i.e. he shouldn't had to have to consult ChatGPT in this case
through then given how good people can be to convince themself that they shouldn't call help even if they really need it (or the opposite) ChatGPT might really help there
TLDR: She was having some health issues, so asked ChatGPT. It recommended she get bloodwork done, but she already had and got the results the night before but her doctor hadn't reviewed the results yet. So she uploaded the lab results to ChatGPT, which responded with "Your platelet count is alarmingly low. You should go to the ER urgently."
> Needless to say, something is deeply broken in our healthcare system.
As someone currently experiencing multiple unexplained systems and getting shuffled around to experts who continue to push appointments further and further back, I can't agree more. I've been using AI a lot lately, and some of these specialists I have actually met with are often surprised at how much knowledge I came in with (granted, I've always been this way, even before AI, it's just now even easier).
But yeah, I had a similar but much less extreme case where I had tests come back, and my primary didn't even look at it for four days. Luckily, I was able to supplement and resolve the (minor) issue myself after talking it over with AI.
This is likely to get downvoted, but in my experience with the health care system in America, I'd prefer consulting ChatGPT over ~50% of the doctors. Too many doctors who I have no idea how they graduated from med school, and even worse, advocate for solutions that maximize profits.
If I can provide one advice is to always question your doctors. Get multiple opinions on serious issues, and educate yourself like this person did. Come prepared to doctor visits and don't be afraid to challenge them if they tell you something that contradicts what you learned.
“Really?” One finally said, with emphasis. Then, a nod. “That’s actually pretty amazing. Good on you. That’s exactly what we want AI to do—help people catch things early.”
“Really?” One finally said, with emphasis. Then, a nod. “That’s actually pretty amazing. Good on you. That’s exactly what we want AI to do—help people catch things early.”
- Fall back to a degraded text mode representation that is still readable
- Redirect to an error display
- Load the site correctly, then a few seconds later redirect to the error display anyways, forcing users to quickly ctrl-A ctrl-C so they can read the article
It seems like having that scribe also review lab results wouldn’t be far off, but we are still not ready to let the AI system interpret and inform the patient without human intervention.
The problem here wasn’t that a human missed the low platelet count, it’s just that it wasn’t reviewed yet. Both a doctor and an AI system would haven spotted the issue immediately.
I am not saying this isn’t a great result, but the AI intervention must come from the patient side, not the care providers side. The providers have too much at risk, while the patient is able to pick and choose their own AI and decide what levels of risk they are able to accept.