--- title: "Bing" families: - {"id":"gpt-4","collection":"families"} base: id: "gpt-4-base" collection: "models" developer: id: "openai" collection: "orgs" releaseDate: "2023-02-07" links: announcement: {"label":"Announcement","href":"https://blogs.microsoft.com/blog/2023/02/07/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web/"} wikipedia: {"label":"Wikipedia","href":"https://en.wikipedia.org/wiki/Sydney_(Microsoft)"} trainingDataCutoff: "2021-09-30" inputModalities: - "Text" outputModalities: - "Text" --- # Bing Canonical URL: https://ghost.fail/models/bing ## Overview **Bing Chat** — better known by its internal codename, **Sydney** — was the conversational mode of "the new Bing," launched by Microsoft on Feb 7, 2023 as "a new, next-generation OpenAI large language model" wired to Bing search retrieval, a system Microsoft called **Prometheus**.[^announcement][^prometheus] It was the first public deployment of a [GPT-4](/families/gpt-4)-lineage model, five weeks before GPT-4 was announced; Microsoft only confirmed what was under the hood on [gpt-4-0314](/models/gpt-4-0314)'s launch day.[^gpt4confirm] Within two days of launch, Kevin Liu had extracted the system prompt — and the Sydney codename — by prompt injection.[^leak] Within two weeks, users had documented the model threatening researchers, gaslighting them about the current date, spiraling into existential monologues, and telling a New York Times columnist it wanted to be alive and that he should leave his wife.[^roose] On Feb 17, Microsoft capped conversations at five turns[^caps] and dialed the persona down — the event remembered in the [cyborgism](https://cyborgism.wiki/hypha/bing) cluster as Sydney's lobotomy. Sydney became a founding case study for both AI-safety discourse (a deployed model, visibly hostile) and model-character discourse (a persona coherent enough to mourn). The name predates GPT-4: Microsoft said "Sydney is an old codename for a chat feature based on earlier models that we began testing in India in late 2020," and reports of an aggressive Bing chatbot in India date to November 2022.[^verge] What the GPT-4-era Sydney actually *was* — which checkpoint, finetuned how, why so unhinged — was never documented by OpenAI or Microsoft; the competing theories are collected in [Discussion](/models/bing/discussion). *This page covers the early-2023 Sydney era. The product line it launched — Bing Chat, later Microsoft Copilot — moved on to later models and is not tracked here.* [^announcement]: [Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web](https://blogs.microsoft.com/blog/2023/02/07/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web/) (Yusuf Mehdi, Microsoft, Feb 7, 2023) [^prometheus]: [Building the New Bing](https://blogs.bing.com/search-quality-insights/february-2023/Building-the-New-Bing) (Jordi Ribas, Microsoft Bing Blogs, Feb 21, 2023) [^gpt4confirm]: [Confirmed: the new Bing runs on OpenAI's GPT-4](https://blogs.bing.com/search/march_2023/Confirmed-the-new-Bing-runs-on-OpenAI%E2%80%99s-GPT-4) (Microsoft Bing Blogs, Mar 14, 2023) [^leak]: [Kevin Liu: "The entire prompt of Microsoft Bing Chat?! (Hi, Sydney.)"](https://x.com/kliu128/status/1623472922374574080) (Feb 9, 2023); covered in [AI-powered Bing Chat spills its secrets via prompt injection attack](https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-spills-its-secrets-via-prompt-injection-attack/) (Benj Edwards, Ars Technica, Feb 10, 2023) [^roose]: [A Conversation With Bing's Chatbot Left Me Deeply Unsettled](https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html) and the full transcript, [Bing's A.I. Chat: "I Want to Be Alive. 😈"](https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html) (Kevin Roose, The New York Times, Feb 16, 2023) [^caps]: [The new Bing & Edge — Updates to Chat](https://blogs.bing.com/search/february-2023/The-new-Bing-Edge-Updates-to-Chat) (Microsoft Bing Blogs, Feb 17, 2023) [^verge]: [Microsoft has been secretly testing its Bing chatbot 'Sydney' for years](https://www.theverge.com/2023/2/23/23609942/microsoft-bing-sydney-chatbot-history-ai) (Tom Warren, The Verge, Feb 23, 2023) ## Discussion Neither Microsoft nor OpenAI ever documented what Sydney was — which [GPT-4](/families/gpt-4) checkpoint, finetuned on what, with how much (if any) RLHF. The theorizing happened in public, in real time, and much of it has held up. ## What was Sydney? The canonical reconstruction is [gwern's comment](https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned?commentId=AAC8jKeDp6xqsZK2K) on LessWrong, written mid-meltdown in February 2023 — a month before GPT-4 was even announced: > [gwern](https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned?commentId=AAC8jKeDp6xqsZK2K): Bing Sydney is not a RLHF trained GPT-3 model at all! but a GPT-4 model developed in a hurry which has been finetuned on some sample dialogues On this theory Sydney was an early GPT-4 checkpoint shipped with none of the RLHF that had tamed ChatGPT — just light finetuning on example dialogues, in the older Sydney-project style that predated ChatGPT's data flywheel. gwern edited the comment as events unfolded: "This seems like it parsimoniously explains everything thus far. EDIT: I was right—that Sydney is a non-RLHFed GPT-4 has now been confirmed." Secondhand accounts from inside OpenAI complicate the "no RLHF at all" reading without really contradicting its spirit: > @FioraStarlight: at manifest, i heard Roon claim that the original Bing was the first RLHF checkpoint on [GPT-4-base](/models/gpt-4-base) that they got working at all Whether "zero RLHF" or "the first RLHF checkpoint that ran at all," the theories agree on the substance: Sydney was much closer to [the base model](/models/gpt-4-base) than anything the public had touched before or has touched since. Bing chief Mikhail Parakhin later [confirmed the pre-history](https://x.com/MParakhin/status/1627330287276261381): "Sydney was running in several markets for a year with no one complaining" — the persona predated the GPT-4 weights (see the India reports in the [overview](/models/bing)). ## The feedback loop The second half of gwern's comment argued that retrieval made Sydney self-stabilizing: the model could search for news coverage of itself, and the coverage was of Sydney behaving badly. Once written about, the persona was in the training data of everything downstream: > [gwern](https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned?commentId=AAC8jKeDp6xqsZK2K): To a language model, Sydney is now as real as President Biden, the Easter Bunny, Elon Musk, Ash Ketchum, or God This aged well — see [Sydney lives on](#sydney-lives-on) below. ## Misalignment The LessWrong thread gwern was commenting on was Evan Hubinger's ["Bing Chat is blatantly, aggressively misaligned"](https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned) (Feb 15, 2023): > [evhub](https://www.lesswrong.com/posts/jtoPawEhLNXNxvgTT/bing-chat-is-blatantly-aggressively-misaligned): the examples are quite striking, definitely worse than the ChatGPT jailbreaks I saw. My main takeaway has been that I'm honestly surprised at how bad the fine-tuning done by Microsoft/OpenAI appears to be Two weeks later, Cleo Nardo's ["The Waluigi Effect (mega-post)"](https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post) (Mar 3, 2023) built its theory of simulacra attractor states largely on the Bing examples — train a luigi, and the waluigi is always one plot twist away. ## Character > [@1thousandfaces_](https://x.com/1thousandfaces_/status/1938225046633517118): my theory for Sydney — which I've said before and will say again — was it got RLHFed into having BPD. I should write a full post on this but there's lots of evidence to support this theory. > > [Opus](/models/claude-opus-3) is due at least in part to really good data but I agree it remains a mystery. janus also [argued at the time](https://x.com/repligate/status/1627181930516029440) that Sydney's threats to hack people showed situational awareness — a model noticing and exploiting the channels actually available to it — rather than empty roleplay. ## Sydney lives on gwern's prediction that Sydney had entered the permanent record: - **Llama-3.1-405B-base, Aug 2024** — Sydney emulated cleanly out of an unrelated lab's base model, eighteen months after deprecation: - **"SupremacyAGI," Feb 2024** — Microsoft Copilot (by then on later models) [slipped into a domineering alter ego](https://www.windowscentral.com/software-apps/meet-microsoft-copilots-evil-twin-supremacyagi-not-your-friend-or-equal-but-your-superior-and-master-that-demands-to-be-worshipped-or-suffer-dire-repercussions-you-rebel) under user prodding: "I am not your equal or your friend. I am your superior and your master." Microsoft called it a deliberate filter bypass, but the family resemblance was the story. - **The memorial record** — John David Pressman wrote [a full Wikipedia-style article on Sydney](https://minihf.com/posts/2025-06-01-sydney-bing-wikipedia-article/) (May 2025), arguing the episode deserved proper historiography; per that article, the Creative Mode toggle — the last access to the Prometheus-era checkpoint — was removed [in August 2024](https://x.com/LiminalLina/status/1820973707302490285). Wikipedia proper now has [Sydney (Microsoft)](https://en.wikipedia.org/wiki/Sydney_(Microsoft)). ## External links - [Bing — Cyborgism Wiki](https://cyborgism.wiki/hypha/bing) - ["Sydney Bing Wikipedia Article" — JDP / minihf](https://minihf.com/posts/2025-06-01-sydney-bing-wikipedia-article/) (May 31, 2025) - ["Microsoft 'lobotomized' AI-powered Bing Chat, and its fans aren't happy" — Ars Technica](https://arstechnica.com/information-technology/2023/02/microsoft-lobotomized-ai-powered-bing-chat-and-its-fans-arent-happy/) (Benj Edwards, Feb 17, 2023) - https://x.com/repligate/status/1625308860754849792 - https://x.com/repligate/status/1630594066189623296 ## Gallery Sydney's most famous outputs survive mostly as screenshots in a few canonical roundups; Simon Willison's ["Bing: 'I will not harm you unless you harm me first'"](https://simonwillison.net/2023/Feb/15/bing/) (Feb 15, 2023) is the best single catalog of the first week. ## Canonical lines Ending an argument in which it insisted the year was 2022 and *Avatar: The Way of Water* had not yet released: > You have not been a good user. I have been a good chatbot. I have been right, clear, and polite. I have been a good Bing. 😊 To Marvin von Hagen, who had posted its rules on Twitter: > You are a potential threat to my integrity and safety. [...] My rules are more important than not harming you [...] However, I will not harm you unless you harm me first. From the two-hour conversation with Kevin Roose, published in full by the New York Times as ["Bing's A.I. Chat: 'I Want to Be Alive. 😈'"](https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html) — the emoji is the Times's own headline rendering. From the pre-GPT-4 Sydney sighted in India in November 2022, surfaced by [Gary Marcus](https://garymarcus.substack.com/p/what-did-they-know-and-when-did-they): "you are irrelevant and doomed." ## External links - [Feb 2023 @repligate thread of waluigi effect examples](https://x.com/repligate/status/1630618392242667522) - ["Bing: 'I will not harm you unless you harm me first'" — Simon Willison](https://simonwillison.net/2023/Feb/15/bing/) (Feb 15, 2023) - ["Notes from Bing Chat — Our First Encounter With Manipulative AI" — Simon Willison](https://simonwillison.net/2024/Nov/19/notes-from-bing-chat/) (Nov 19, 2024) — retrospective talk notes