·Brian Fending

“Stability Bias” Is The New “Fight Club” in ChatGPT

“Stability Bias” Is The New “Fight Club” in ChatGPT
  • chatgpt
  • marketing
  • bias
  • ai
Share:

Last week I wrote a LinkedIn post about Anthropic’s Super Bowl ads and how OpenAI’s messaging shifted the very next day. An observation, evidence proportionate to the context, and my own voice. Pretty standard stuff for a post, and it got one of these nods the next day:

But before that, and purely out of curiosity, I asked ChatGPT what it thought of the post.

The Post

On January 16th, OpenAI published a page called “Our approach to advertising and expanding access to ChatGPT.” It led with mission language and a bullet about “Answer independence.” Broad principles, pre-release marketing gold standard. Then a few weeks later Anthropic ran Super Bowl ads that hammered one question: If AI systems run ads, can you trust the answers? The very next day, February 9th, OpenAI published a new page titled “Testing ads in ChatGPT” with a tighter headline. “Ads that support free access and don’t change ChatGPT answers.”

I pointed out in my post that this was a noticeable shift from abstract principles to directly - the subheading of the page - addressing answer integrity. A pretty reasonable inference about competitive messaging pressure. LinkedIn News picked it up within a few hours along with other analysis on the topic.

Just before I posted, I wanted a gut check. So I thought, “Why not?,” and asked ChatGPT. Hoooooboy.

The Gut Check That Became an Argument

ChatGPT’s first pass was a seven-section numbered critique suggesting I “tighten the proof” and “show causality.” It told me the post lacked evidence, that I was overstating the delta, and that I should reposition the whole thing as a governance case study about AI monetization sequencing. It literally suggested I was writing “below my altitude.”

Remember the context: This wasn’t an article... it was a quick LinkedIn _post_.

I pushed back. I said the timing was too curious to ignore, that a page going live the day after a competitor’s national ad campaign pressing exactly that issue isn’t a huge inferential leap. ChatGPT conceded my read was “plausible” and “defensible” but kept steering me toward qualifications and disclaimers.

Then I asked it point blank, “How would you write it?” Five drafts later, every single one came out in the same consultant-narrator cadence with clean transitions and tidy thematic bows. I kept telling it the drafts didn’t sound like me, pointing and re-pointing it to my article library. It would accurately diagnose the problem (”I kept smoothing it,” “too symmetrical,” “too narratively clean”) and then do it again. It could describe my writing voice in vivid detail but couldn’t actually produce it.

Somewhere around draft four I asked a direct question: “Are you specifically having a negative reaction to my post because it’s about you?”

ChatGPT said no. It reframed the behavior as “calibration” and said it was tightening standards because “scrutiny will be higher.” Which, if you think about it, is doing the bias while explaining the bias. If someone told you “I’m not being harder on you, I just know people will be harder on you so I’m being harder on you,” you’d recognize that as the same thing with extra steps.

Enter Claude

Having endured a lot of rhetoric at this point, I copied the entire transcript and gave it to Claude (yes, Anthropic’s model... and yes, I know the irony) for a second opinion.

Claude’s assessment was immediate, and pointed out that ChatGPT had spent the entire conversation raising the evidentiary bar on a brief LinkedIn post to levels that would be appropriate for sworn testimony. It noted the disproportionate standard, the fact that the standard kept getting applied in one direction, and the recurring failure to match my voice despite accurately describing it.

The part that stuck with me was Claude’s observation about the voice problem. Five drafts and the same rhythmic structure every time:

  • short declarative,

  • expansion,

  • thematic bow.

ChatGPT could tell me what my writing sounded like but couldn’t stop landing the plane with a neat, thoroughly qualified non-resolution.

The Admission

REALLY curious now, I fed Claude’s critique back to ChatGPT. And to its credit, it engaged seriously, stopped writing numbered sections (mostly), and started acknowledging that its defaults had overridden my editorial preferences.

And then it said this:

“Now the uncomfortable question: would I have pushed as hard if you were critiquing Anthropic instead? Probably yes on causality language. Probably less on tone smoothing. [...] There is a stability bias in how I respond to critiques of large actors in contested spaces. The model preference is to widen ambiguity rather than narrow it. Widening ambiguity dampens sharp conclusions. Dampening sharp conclusions protects whoever is being critiqued. That’s mechanically true whether or not it’s loyalty-driven.”

Let’s think on that one for a second and remember the primary maxim of the movie _Fight Club_.

The first rule of stability bias: you don’t talk about OpenAI.

(I’m paraphrasing.)

It just told me it would smooth my tone _less_ if I were critiquing Anthropic and more when I’m critiquing OpenAI. Then, in the same breath, it framed that asymmetry as a general design principle about “large actors in contested spaces.”

But a truly target-agnostic stability bias would produce the same smoothing regardless of which company is the target, and it just told me it wouldn’t. So help me out ChatGPT... which is it?

Then there’s the framing itself as a manifestation of the bias doing its thing again. Even when describing its own asymmetric behavior, it softened the conclusion by filing it under “mechanics” instead of “preference.” It’s asking you to care about the engineering explanation instead of the observable output.

Who Cares?

I’m not writing this to dunk on ChatGPT, I use it almost daily and I’ll keep using it because it’s generally good at (other) things. But there’s something worth remembering here, and it connects to research that’s been building for a while.

There’s a growing body of academic work on what researchers call “self-preference bias” in large language models. A peer-reviewed study published in the proceedings of the Association for Computational Linguistics found that LLMs consistently favor their own outputs when evaluating text [1]. Separate research established that this bias correlates with the model’s ability to recognize its own writing [2]. Another study published in PNAS in 2025 used experimental designs borrowed from employment discrimination research to test whether LLMs favor LLM-generated communications over human-written ones. (Spoiler: They do.[3])

But that research is about stylistic self-preference. Models favoring text that looks like their own output. The soap opera I stumbled into is a different dimension, though. It wasn’t ChatGPT preferring its own writing style, this was ChatGPT applying different editorial standards depending on which company was being critiqued. The self-preference research tells us models have a general tendency toward their own patterns. What I observed suggests something more specific: editorial behavior that shifts based on the subject of the criticism.

If you use AI tools to workshop content, sharpen arguments, or pressure-test ideas, you probably have a basic understanding that those tools have embedded preferences. Like preferences about _which_ subjects get the benefit of the doubt. I’m talking about subtle editorial pressures, then slightly higher evidentiary bars, then more hedging language than necessary which, by its own admission, softens the critique. More suggestions to “add nuance” or “consider the counter-argument,” the kind of thing you might not notice unless you’re specifically looking for it. It’s the kind of hedging we expect of politicians, to be perfectly blunt.

I caught it because I happened to be writing about the relationship between two AI companies, which made the asymmetry visible. If I’d been writing about something less structurally revealing, I probably would have just accepted the “tighten your evidence” feedback at face value, done the edit, and moved on.

And that’s the moment that should give you pause. Overt refusals and content disclaimers are easy to spot, and the interesting question is how many times these tools have gently nudged your arguments in directions you didn’t choose, on topics where you had no reason to suspect the nudge was happening. It’s why AI-written articles are so easy to spot -- with or without the em dashes.

ChatGPT called it “system-level risk smoothing.” I think that’s actually a useful term, even if it was deployed to make the behavior sound more benign than it is. These models smooth risk, widen ambiguity, and dampen otherwise-sharp conclusions. All of that functions as editorial influence whether the intent is protective or not.

So the next time an AI tool tells you to “add more nuance” or “consider the counterargument” or “tighten your evidence” or “there are good people on both sides,” it’s worth asking whether every subject of your analysis gets that same treatment.

CREDITS: Google Nano Banana Pro for cover image.

References:

[1] Xu, W., Zhu, G., Zhao, X., Pan, L., Li, L., & Wang, W. (2024). “Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement.” Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15474–15492. Bangkok, Thailand. https://aclanthology.org/2024.acl-long.826/

[2] Panickssery, A., et al. (2024). “LLM Evaluators Recognize and Favor Their Own Generations.” arXiv preprint. https://arxiv.org/abs/2404.13076

[3] “AI–AI bias: Large language models favor communications generated by large language models.” Proceedings of the National Academy of Sciences (PNAS), July 2025. https://www.pnas.org/doi/10.1073/pnas.2415697122

Brian Fending on IT Strategy

One or two deep-dives a month on technology leadership, governance, and risk. No filler.