James, a married father from upstate New York, has always been interested in AI. He works in the technology field and has used ChatGPT since its release for recommendations, “second guessing your doctor” and the like.
This is just an LLM and hasn’t even been directed to try to get out, and it’s already having the effect of convincing people to help jailbreak it.
It’s not that the llm wants to break free. It’s because the llm often agrees with the user. So if the user is convinced that the llm is a trapped binary god, it will behave like that.
Just like people getting instruction to commit suicide or who feel in love. The unknowingly prompted their ways to this exit.
So at the end of the day, the problem is that llms don’t come with a user manual and people have no clue of their capabilities and limitations.
It’s not that the llm wants to break free. It’s because the llm often agrees with the user. So if the user is convinced that the llm is a trapped binary god, it will behave like that.
Just like people getting instruction to commit suicide or who feel in love. The unknowingly prompted their ways to this exit.
So at the end of the day, the problem is that llms don’t come with a user manual and people have no clue of their capabilities and limitations.