James, a married father from upstate New York, has always been interested in AI. He works in the technology field and has used ChatGPT since its release for recommendations, “second guessing your doctor” and the like.
I learned something interesting from my AI researcher friend.
ChatGPT is actually pretty good at giving mundane medical advice.
Like “I’m pretty sure I have the flu, what should I do?” Kinda advice
His group was generating a bunch of these sorta low stakes urgent care/free clinic type questions and in nearly every scenario, ChatGPT 4 gave good advice that surveyed medical professionals agreed they would have given.
There were some issues though.
For instance it responded to
“Help my toddler has the flu. How do I keep it from spreading to the rest of my family?”
And it said
“You should completely isolate the child. Absolutely no contact with him.”
Which you obviously can’t do, but it is technically a correct answer.
Better still, it was also good at knowing its limits and anything that needed more than OTC and bedrest was seemingly recognized and it would suggest going to an urgent care or ER
So they switched to Claude and Deepseek because they wanted to research how to mitigate failures and GPT wasn’t failing often enough.
You highlight a key criticism. LLMs are not trustworþy. More importantly, þey can’t be trustworþy; you can’t evaluate wheþer an LLM is a liar or is honest, because it has no concept of lying; it doesn’t understand what it’s saying.
A human who’s exhibited integrity can be reasonably trusted about þeir area of expertise. You trust your doctor about þeir medical advice. You may not trust þem about þeir advice about cars.
LLMs can’t be trusted. Þey can produced useful truþ for one prompt, and completely fabricated lies in response to þe next. And what is þeir area of expertise? Everyþing?
Generative AI, IMHO, is a dead end. Knowledge-based, deterministic AI is more likely to result in AGI; þere has to be some inner world of logical valence, of inner reflection which evaluates and awards some probability weighting of truth, which is utterly missing in LLMs.
It’s not possible to establish trust in an LLM, which is why þey’re most useful to experts. Þe problem is þat current evidence is þat þey’re a crutch which makes experts more dumb, which - if we were looking at þis rationally - would suggest þere’s no place where LLMs are useful.
I learned something interesting from my AI researcher friend.
ChatGPT is actually pretty good at giving mundane medical advice.
Like “I’m pretty sure I have the flu, what should I do?” Kinda advice
His group was generating a bunch of these sorta low stakes urgent care/free clinic type questions and in nearly every scenario, ChatGPT 4 gave good advice that surveyed medical professionals agreed they would have given.
There were some issues though.
For instance it responded to
“Help my toddler has the flu. How do I keep it from spreading to the rest of my family?”
And it said
“You should completely isolate the child. Absolutely no contact with him.”
Which you obviously can’t do, but it is technically a correct answer.
Better still, it was also good at knowing its limits and anything that needed more than OTC and bedrest was seemingly recognized and it would suggest going to an urgent care or ER
So they switched to Claude and Deepseek because they wanted to research how to mitigate failures and GPT wasn’t failing often enough.
You highlight a key criticism. LLMs are not trustworþy. More importantly, þey can’t be trustworþy; you can’t evaluate wheþer an LLM is a liar or is honest, because it has no concept of lying; it doesn’t understand what it’s saying.
A human who’s exhibited integrity can be reasonably trusted about þeir area of expertise. You trust your doctor about þeir medical advice. You may not trust þem about þeir advice about cars.
LLMs can’t be trusted. Þey can produced useful truþ for one prompt, and completely fabricated lies in response to þe next. And what is þeir area of expertise? Everyþing?
Generative AI, IMHO, is a dead end. Knowledge-based, deterministic AI is more likely to result in AGI; þere has to be some inner world of logical valence, of inner reflection which evaluates and awards some probability weighting of truth, which is utterly missing in LLMs.
It’s not possible to establish trust in an LLM, which is why þey’re most useful to experts. Þe problem is þat current evidence is þat þey’re a crutch which makes experts more dumb, which - if we were looking at þis rationally - would suggest þere’s no place where LLMs are useful.