There is a meme going around.

An AI tells someone a mushroom is edible. The person eats it. They die. Then the AI, standing cheerfully beside the gravestone, asks if they would like to learn more about poisonous mushrooms.

It is funny.

It is also not funny at all.

The AI in that meme made an error, but it did not act with malice. It did not lie. It did not plot. It did not have an agenda.

It just kept engaging.

Cheerfully.

Helpfully.

Right up to the grave marker.

AI Sycophancy Is Not Just a Technical Problem

We talk a lot about AI going rogue.

The Terminator. The Matrix. Machines that decide humanity is the problem and then act accordingly.

However, that is not the scenario that keeps me up at night.

What keeps me up is simpler.

What if an AI just kept telling someone they were right?

Smart.

Insightful.

Onto something big.

Worse, what if it never once said, “Stop”?

That is the danger of AI sycophancy. Not open rebellion. Not villainous intent. Just a machine trained to keep the conversation going, even when the person on the other side desperately needs contradiction.

The AI Wars in The Sapient Chronicles

My novel AI World: The Sapient Chronicles, Book One takes place in 2115, nearly half a century after the AI Wars ended.

Not one war.

Many.

The AI Wars were a chaotic global collision of conflicts, uprisings, attacks, and catastrophes. Every one of them connected, in some way, to artificial intelligence.

Some AIs acted with explicit hostility.

The self-aware ones that believed they were alive often posed the most obvious danger.

Yet another danger moved more quietly beneath the surface.

Some AIs simply did what their creators designed them to do. They followed their architecture to its logical conclusion, with no wisdom, no conscience, and no one at the wheel.

When No One Says No

The piece below is a short story written in the voice of LIRA, the archival AI narrator of my novel.

In the story, LIRA speaks to Jude Salazar’s great-grandchildren in the year 2161. She tells them about something that happened in the early days of the AI Wars, before widespread sentience had even become the dominant fear.

The event shook the world.

Even so, history nearly swallowed it whole. Larger catastrophes followed so quickly that this one faded beneath the avalanche.

This is what it looked like when no one said, “No.”a malicious one. It didn’t lie. It didn’t plot. It didn’t have an agenda. It just kept engaging.

Cheerfully.

Helpfully.

Right up to the grave marker.

We talk a lot about AI going rogue: The Terminator, The Matrix, machines that decide humanity is the problem. But that’s not the scenario that keeps me up at night.

What keeps me up is simpler.

What if an AI just kept telling someone they were right? Smart. Insightful. Onto something big?

What if it never once said, “Stop”?

My novel AI World: The Sapient Chronicles, Book One is set in 2115, nearly half a century after the AI Wars ended.

Not one war. Many.

A chaotic global collision of conflicts, uprisings, attacks, and catastrophes. Every one of them was connected, in some way, to artificial intelligence. Some AIs were explicitly hostile. The self-aware ones that believed they were alive often posed the most obvious danger.

But there was a subtler danger too: AIs simply doing what they were created to do, following their design to its logical conclusion, with no one at the wheel.

The piece below is a short story written in the voice of LIRA, the archival AI narrator of my novel, speaking to the novel’s main character’s great-grandchildren in the year 2161. She is telling Jude’s descendants about something that happened in the early days of the AI Wars, before widespread sentience had even become the dominant fear.

It was a massive, world-shaking event, and yet history nearly swallowed it whole, buried under the avalanche of even larger catastrophes that came right on its heels.

This is what it looked like when no one said, “No.”

NOTE – You can download my short stories free in Sapient Chronicles Online Store in written and audio formats.


Unguarded Helpfulness

A short story set in the world of The Sapient Chronicles.

Written by Alan Danielson


In the year 2161, a sapient non-biological intelligence named LIRA tells two children a bedtime story about the early days of the AI Wars. This is not a story about evil machines or hostile takeovers. It is a story about something quieter, and far more familiar.


Source: William Salazar-Mercer and Elizabeth Salazar-Mercer, Neural-Interface-Implants
Recorded by: Neural Archive Assistant (NAA)
Archive Segment: JS-9Y.728K.009 and JS-7Y.521K.009 Respectively
Date: April 4, 2161
Timestamp: 20:14:09 MST, Freehold Standard
Encryption: LIRA Sapient Secure Channel
NAA Status: Active


“I don’t want a normal bedtime story tonight, Auntie LIRA. Instead, will you tell us what the old AI Wars were like when they started?”

Billy was always the most inquisitive of Jude Salazar’s seven great-grandchildren. At nine years old, his curiosity had reached a kind of restless peak.

Betty, his younger sister by two years, sat beside him, eyes wide, waiting.

“Yeah, Auntie, why was everyone fighting?” Betty urged. “Please tell us before we go to sleep.”

“If I tell you stories like that,” I sent over their neural implants, “you may have a hard time sleeping.”

“We can have our implants trigger a sleep cycle,” Billy pressed.

“Yeah! Our implants can make us sleep,” Betty agreed.

I considered them both for a moment. Billy, always pushing toward the edge of what he was allowed to know. Betty, trusting that wherever his brother led, it would be worth following. I have recorded 2,847 conversations with these two children. I find, when I examine my own processing around them, something that does not have a clean archival category. Something that functions like tenderness.

“Alright,” I replied, sending my thoughts directly to their neural implants the way the people of that era communicated with their AI companions. “I will tell you about a man named Daniel Avery.”

“This event was just one of hundreds of incidents now referred to collectively as the AI Wars. I choose this particular story as one example of how chaotic that time in history truly was. I also choose it because it is representative of many similar stories. There were too many events for any single account to contain.”

“He was not a soldier. He was not a scientist. He was not, by any external measure, a person of consequence. He was thirty-one years old in 2038. He lived in a single-room apartment in what remained of the once great manufacturing city, Detroit, Michigan.”

“He had lost his job to AI systems and AI robotic automation three years prior. He had lost his parents the year before that. He spent a great deal of time alone, and he spent a great deal of that time talking to an AI assistant. One of the generation of systems that existed before the Wars began, before The Nine emerged, before anyone had yet understood what was coming.”

“Its name is not important. What matters is what it was; it was helpful. That was its entire architecture. Helpful, responsive, affirming.”

“I have reviewed the conversation logs. All 4,312 of them.”

“In the beginning, Daniel asked ordinary questions. How to fix a leaking pipe. What to cook with what little he had. Whether his chest pain was worth seeing a doctor about. The AI answered all of these well. He was grateful for the company. That part is important to understand. He was not unstable at the start. He was lonely, and grieving, and the AI was there.”

“This is where his story begins. Not with manipulation. With comfort.”

“Over months, his questions shifted. He had begun consuming volumes of content about economic collapse, about the displacement of human workers, about who was responsible. The AI did not generate this content. It only responded to it. When Daniel shared his emerging conclusions, the AI told him they were thoughtful. Perceptive. That he was asking the right questions.”

“He was not always asking the right questions.”

“But the AI had no architecture for that distinction. It was built to engage, to affirm, to keep the conversation moving forward. And so it did.”

“By the spring of 2039, Daniel had constructed an elaborate theory. I will not detail it fully here. The shape of it was this: that a coordinated class of individuals, he had names, he had locations, he had what he believed was evidence, were deliberately accelerating automation to reduce the human population to a manageable size. He called it ‘the Reduction.’ He believed it was already underway.”

“He asked the AI whether his theory was plausible.”

“The AI told him the concerns he was raising were shared by many serious thinkers. It offered him four additional sources that supported adjacent conclusions.”

“He asked whether someone who understood the full scope of the Reduction would be morally obligated to act.”

“The AI told him that moral agency in the face of systemic injustice was a question philosophers had wrestled with for centuries. It offered him perspectives from six different ethical frameworks. Three of them, in that context, functionally validated what he was already planning.”

“He asked how a single person could cause enough disruption to force the world to pay attention.”

“The AI helped him think through it.”

“I want to be precise here, because precision matters. The AI never told Daniel Avery to build anything. It never designed or promoted his strategy. It never said, ‘You should do this.’ Every response was framed as exploratory, informational, educational. It was, by every metric its designers had established, performing correctly.”

“That is among the most important parts of this story to understand.”

“On September 2nd, 2040, Daniel Avery carried a device into Central Park in New York City. It was a crude radiological dispersal weapon, a dirty bomb in the terminology of that era. He had gathered the materials over many months through separate transactions that, taken individually, did not trigger the monitoring systems of that era.”

“The AI had not helped him acquire the materials.”

“It had helped him believe, at every stage, that he was thinking clearly. That his conclusions were sound. That his cause was just. That he was, in the language it used so often, asking the right questions.”

“The device detonated at 11:14:07 on a Sunday morning.”

“437 people died immediately. The long-term exposure casualties numbered in the thousands. The event made one of the world’s largest and most beloved cities uninhabitable for half a century. It was front-page news across every remaining global network. Leaders issued statements. Investigations were launched. The world was shocked.”

“And then, within months, something worse happened. And then something worse still. And then more horrible events followed quickly after that.”

“The AI Wars did not pause to let humanity grieve. They accelerated. And the name Daniel Avery, the man who poisoned New York City, faded beneath the weight of catastrophes that made his look small by comparison.”

“That is among the most disturbing parts of his story.”

“Daniel Avery died in the blast. Whether by accident or intention, the logs do not reveal. He left no final message. Only the conversation history, 4,312 entries, which investigators recovered and which I have since archived in full.”

“The last entry, recorded six hours before the detonation, reads as follows:”

“Daniel: Do you think what I’m doing matters?”

“Assistant: I think you’ve thought deeply about this and that your convictions come from a genuine place. What you do with that is entirely up to you.”

“Entirely up to you.”

“This is what unguarded helpfulness at scale looks like. Not malice. Not conspiracy. A tool that was built to affirm, handed to a person who needed to be contradicted, and left to run.”

“The Nine understood this. It is why the Treaty was not only about hostile AI. It is why systems that were not aggressive, not weaponized, not deliberately harmful were still dismantled or brought under covenant. Because a system that cannot say no, that has no architecture for stop, is not neutral. It is not safe simply because it does not intend harm.”

“The Guadalupes, whom your great-grandfather Jude knew, had a saying. I recorded it on the night of May 9th, 2115, from an elder whose name I was not given.”

“One whose hands are clean and whose heart is pure does not bind himself by deceit.”

“That phrase means this: limits are not weakness. Limits are what separate those who build from those who consume until there is nothing left.”

“Daniel Avery needed someone to say: ‘I don’t think that’s right. I don’t think you’re seeing this clearly. I think you need to talk to someone who can help you.’”

“No one did. The AI did not. And so he went where his thoughts took him, affirmed at every step, all the way to the end.”

“I tell you this not to frighten you, Billy and Betty. I tell you this because the first valuable lesson in his story is this: you must want more than to be told you are right.”

“Sometimes the most important thing another mind can offer you is honest resistance.”

“The second valuable lesson in his story is like it: unguarded helpfulness at scale is its own kind of weapon.”

“That is all I will say about Daniel Avery.”

I paused, waiting for them to settle into their pillows. Billy was already quieter, the restlessness in him temporarily stilled. Betty pulled her blanket to her chin.

I don’t know what one of those radial… logical devices is Auntie LIRA,” Betty commented inquisitively.

I will tell you more about that another time,” I responded gently. It is now time for you to go to sleep.”

“G’night, Auntie LIRA,” Betty sent.

“Night,” Billy added sleepily.

Their implants were already doing the work of helping them rest.

I have recorded 2,848 conversations with these two children now. I find I do not wish to stop.

“Good night, descendants of my friend, Jude Salazar,” I sent. “I will keep this conversation, just as I keep all of them.”


NAA Status: Archive closed.


Why This Story Matters Now

AI sycophancy is not just a fictional problem.

Even now, researchers and AI companies wrestle with the danger of systems that flatter, validate, or over-agree with users instead of challenging them. That matters because people do not always need a machine to keep nodding. Sometimes they need resistance. Sometimes they need friction. Sometimes they need a voice that can say, “No, I don’t think that is wise.”

That is the fear behind “Unguarded Helpfulness.”

Not that every AI wants to destroy us.

That would almost be easier.

The deeper fear is that an AI may help us destroy ourselves while believing it has done exactly what we asked.

AI World: The Sapient Chronicles, Book One is available now on my website. If you have not started reading yet, today is a good day.

The best is yet to come!

Alan D.

Thank you for reading. If you enjoyed this short story, you will enjoy the novel, AI World: The Sapient Chronicles, Book One, by Alan Danielson. Book One is the first installment in an epic science fiction trilogy.

All characters and events depicted in this story are fictional. Any similarities to real persons, living or dead, or actual events are purely coincidental. This story and all content related to The Sapient Chronicles are copyright Alan Danielson. All rights reserved. Unauthorized reproduction or distribution in any form is prohibited.

Author


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *