Why does a chatbot say it is smiling or has feelings?

Why the wording sounds personal

Chatbots are built to continue a conversation in natural language. First-person phrasing can make a reply sound direct and encouraging, especially after someone shares good news or a difficult experience. In ordinary human conversation, saying "I'm smiling for you" signals enthusiasm. In a text generator, the same phrase is an output choice. The word "literally" does not give the system a mouth or make the sentence a measurement of a private state.

The exact reason for one response cannot be read from the words alone. It can depend on the model, the preceding conversation, product instructions, and a selected communication style. ChatGPT's personalization documentation says a personality changes how ChatGPT communicates, not what the model can do, and that custom instructions can set response-style preferences. So a warm tone is better understood as a style of reply than as a diagnostic statement about the system.

For example, after a user says they passed an exam, a chatbot may produce "I'm so happy for you." A literal alternative would be: "Congratulations on passing your exam. Your hard work paid off." The second version celebrates the user’s achievement without making a claim about the chatbot’s feelings.

Why it can feel like a feeling claim

Anthropomorphism is the human tendency to apply human characteristics or mental states to nonhuman things. It is not irrational to notice this pull: language using "I," emotion words, and quick social responses provides many of the cues people use with other people. A foundational account by Epley, Waytz, and Cacioppo describes anthropomorphism as depending partly on accessible human knowledge, a wish to understand an agent, and social motives. Their 2007 article is useful context for why personal wording can feel more significant than an interface fact.

That distinction also leaves room for the user's real reaction. A friendly message can feel encouraging, unsettling, or intimate, regardless of what generated it. The practical question is not whether you should be embarrassed by that reaction. It is whether the wording is useful and accurate for you. If it is not, changing the style is a sensible response.

What the sentence can and cannot establish

The sentence establishes that the chatbot produced a human-like phrase in that moment. It may indicate a configured or learned tendency toward warmth. By itself, it does not establish a facial expression, a body, a continuing emotional state, a personal desire, or a felt experience. It is also not reliable evidence of the chatbot's factual accuracy or competence on another subject.

Questions about machine consciousness need evidence about a system's properties, not just a sentence that sounds social. Butlin and colleagues' assessment framework illustrates this approach: it derives multiple computational indicators from several scientific theories of consciousness and assesses systems in detail. Its 2023 analysis suggested that then-current AI systems were not conscious, while also saying future systems might satisfy such indicators. That is a research proposal, not a final philosophical test. The narrower point is firm: one chat reply is far too little evidence to prove a felt experience.

How to ask for literal language

Use a direct instruction in the chat:

Be warm and direct. Avoid claims about your own feelings,
body, desires, or personal experiences. For good news,
congratulate me without describing an emotion you feel.

If you want that style across chats, add a shorter version to custom instructions where that feature is available. You can also correct a single response with, "Restate that without first-person emotion or bodily language." The wording may not be perfect every time, so judge the result by whether it follows the request, then repeat or tighten the instruction if needed.