ELIZA: The First Chatbot and the Lesson in Overtrust

In the mid-1960s, an MIT professor wrote a simple program and showed it to his secretary. After a few minutes of “conversation”, the woman asked him a request that froze him: she asked him to leave the room because she wanted to talk to the program in private. The secretary knew perfectly well that she was talking to a machine - the professor wrote it himself. And yet she felt that there was someone on the other side she could confide in. The program was called ELIZA, it was the first chatbot in history and it did not understand a word of what was written to it. Yet people opened their hearts to him. This story from six decades ago explains more about our attitude towards artificial intelligence than many modern reports.

The trick is easier than you think

The creator of ELIZA was Joseph Weizenbaum. His program simulated a conversation with a psychotherapist of a particular school - one who does not so much give advice as reflects the patient’s statements back in the form of questions. And that was its entire mechanism.

ELIZA had no knowledge of the world, emotions or conversation. It worked on simple pattern matching: it caught keywords in the user’s sentence and inserted them into ready-made templates. You wrote “I am sad” - ELIZA replied “Why are you sad?” “My father doesn’t understand me” - “Tell me more about your father.” “I feel lonely” - “How long have you felt lonely?” No understanding, no empathy - just cleverly turning what you said into another question.

It was an illusion of conversation made of several dozen rules. And yet it worked all too well.

ELIZA effect: we see the mind where there is none

What happened next surprised Weizenbaum himself and became a permanent part of history under the name of the “ELIZA effect”. People who talked to the program began to attribute understanding, caring and intelligence to it. They confided in them about their personal problems, felt heard, and attributed to the machine empathy that did not exist and could not exist. They knew perfectly well that it was a program - and yet they reacted as if there was a human sitting on the other side.

The ELIZA effect is our deep, innate tendency to attribute human characteristics to machines based on superficial signals. It was enough for the program to “talk” like a human - our brain did the rest. This wasn’t a user flaw; it is a fundamental feature of the human psyche. We see intentions and minds everywhere because we have been shaped by evolution - and a machine that imitates human language triggers the same reflex.

A creator who was afraid of his own creation

People’s reaction to ELIZA shocked Weizenbaum so much that he changed the course of his life. Instead of celebrating success, he became one of the first and sharpest critics of the uncritical admiration for artificial intelligence. What terrified him was not that the machine was pretending to be human, but how easily we could be fooled - and what that says about our willingness to replace human relationships, empathy, and judgment with programs that merely imitate them.

He wrote an influential book warning that there are areas of human life - caring, moral judgment, empathy - that should not be ceded to machines just because they are good at imitating them. His voice, six decades ago, sounds like a prophecy today.

Why ELIZA is more important today than ever

Let’s fast forward to the present day. Today we are talking to AI assistants, which are not primitive puzzles with dozens of rules, but powerful language models that generate fluent, smart-sounding, coherent statements on any topic. If simple ELIZA could get people to confide, how much stronger is the ELIZA effect with today’s systems?

And herein lies the crux of the security problem. The ELIZA effect causes:

We trust confident-sounding AI responses, even when they are completely made up (hallucinations). A model that “speaks” convincingly triggers our trust reflex - regardless of whether it is right.
We confide in chatbots things we shouldn’t - company data, secrets, personal information - because we feel that we are talking to “someone”, and not to a tool that stores and processes our every word. This is a direct risk of data leakage, which we write about in AI security at .
We are susceptible to manipulation. If a machine that sounds like a human inspires trust, then an attacker with a voice faked by AI](/en/blog/deepfake-voice-video-business-attacks/) or a convincing chatbot has powerful social engineering leverage.

Lessons for companies and users

Treat AI output as untrusted. A confident tone is not proof of truth. Assistants’ responses - especially when they reach a decision or other part of the system - require verification, just like any data from an untrusted source. It is also the core defense against prompt injection.
Teach people about the ELIZA effect. Just knowing that “it sounds human, so I trust it” is a reflex, not a judgment, protects against manipulation. Naming the mechanism takes away some of its power.
Set rules on what can be entrusted to chatbots. Since people naturally confide in each other, a clear policy is needed: what data cannot be entered into AI tools. This is part of the sensible implementation of AI.
Require transparency. The user has the right to know that he or she is talking to a machine. This is not only an ethical and regulatory requirement, but also a defense against abuse of the ELIZA effect.

Summary

ELIZA was a simple program from sixty years ago - a few dozen rules that turned your sentences into questions without a hint of understanding. And yet she made people confide in her as if they were a friend, ascribing empathy that wasn’t there. She discovered something lasting in us: the ELIZA effect, an innate tendency to see the mind where there is only an imitation of language. Its creator was terrified not by the machine, but by the ease with which we fell for it. Today, when we talk to models infinitely more persuasive than ELIZA, this lesson is the foundation of digital security: just because something sounds human and speaks with a confident tone, doesn’t mean it understands, is telling the truth, or deserves to be trusted. Sixty years ago, a simple chatbot warned us about ourselves. It’s finally worth listening.

Are you implementing a chatbot or AI assistant and want to do it so that user trust does not become a security gap? Contact us - secure AI implementations and testing is one of our specializations.

Frequently asked questions (FAQ)

What was ELIZA and how did it work? ELIZA is the first chatbot in history, created in the 1960s by Joseph Weizenbaum. She simulated a conversation with a psychotherapist, but she had no knowledge or understanding - she worked on simple pattern matching, picking out keywords and inserting them into ready-made question templates. She turned “I’m sad” into “Why are you sad?” Its entire “intelligence” consists of several dozen rules.

What is the ELIZA effect? This is the human tendency to attribute understanding, emotion, and intelligence to machines based on superficial signals—for example, that a program “speaks” a human language. People talking to ELIZA felt heard and confided in her, even though they knew it was a program. This is a fundamental feature of our psyche, today strongly exploited (consciously or not) by AI systems.

Why is the ELIZA effect dangerous in the context of today’s AI? Because modern language models sound incomparably more convincing than ELIZA, they trigger this trust reflex even more strongly. This leads to trusting made-up answers (hallucinations), confiding data to chatbots that should not be disclosed, and being susceptible to manipulation by fake voices and persuasive bots.

How to protect yourself against excessive trust in AI? Treat AI output as untrusted data requiring verification (a confident tone is not proof of truth), educate people about the ELIZA effect, set clear rules on what data cannot be entered into AI tools, and demand transparency - the user should know they are talking to a machine. Awareness of the mechanism is the first and most important defense.