How Can Science Fiction Reveal AI Ethics' Hidden Costs?
An imagined world can expose a choice's consequences without proving a prediction. M.K. examines assumptions, counterexamples and competing viewpoints as tools for disciplined ethical thought experiments.
AI ethics makes it easy to agree with attractive principles: preserve freedom, reduce harm, respect choice and accept responsibility. The difficulty often begins when those principles conflict in the same situation.
Science fiction can place that conflict inside a world we can imaginatively inhabit. Instead of only reading that efficiency may cost freedom, we can examine who gains stability, who loses choice and how a reasonable arrangement changes shared life.
But an affecting story does not establish a prediction. Fiction can make assumptions and costs visible while inviting counterexamples. It cannot replace empirical evidence.
A thought experiment is more than staging a preferred conclusion
The Stanford Encyclopedia of Philosophy entry on thought experiments examines their different uses and debates over their contribution to knowledge. They can clarify concepts and challenge claims, but there is no single uncontested account of their epistemic value.
For an author, that is a useful warning. Assigning a favourite position to a virtuous character and an opposing view to a villain can create persuasive fiction without seriously testing the ethics.
A harder question arises when both choices have reasons and we still must choose a cost to bear. Characters and circumstances can reveal that an abstract answer becomes less tidy when it enters a life.
Identify the assumptions that do the important work
Imagine a city where AI helps provide stable services while some personal choices require system approval. This is a hypothetical story for this article, not a scene from Neo Babylon or a description of an existing city.
If the story assumes perfect accuracy, universal agreement and cost-free exit, its conclusion may seem easy. Change one condition and the question changes: can the system fail, can people disagree and can non-users still obtain basic services?
I would identify the assumptions shaping the conclusion before varying them. A novel need not become a specification sheet. Its author should nevertheless know whether a convenient premise has silently settled the ethical issue.
Fiction may go beyond current technology. Greater departures make it more important to distinguish examining values under a stipulated premise from describing an evidence-supported possibility.
Change a condition and test the conclusion
Would residents accept the same authority from a less capable manager? Would excellent services make a system fair if a minority could not appeal? Would exit remain meaningful if it required abandoning basic living conditions?
These questions test whether we prioritize results, procedures or self-determination and which conditions we consider indispensable. They do not provide an automatic answer.
Counterexamples should also run against the author's preference. A story critical of centralized control should seriously consider the difficulties of dispersed decisions: coordination costs, unequal power and public duties without an owner. The preferred arrangement should not be written as cost-free while its rival bears every disadvantage.
An ethically useful story should make its author's own answer open to scrutiny. That is my reading and writing criterion, not a universal test of literary merit.
Different positions reveal different costs
A beneficiary, maintainer, rejected applicant and person unable to leave may experience the same arrangement differently. This does not make truth arbitrary. It means each position may reveal different consequences and information.
Fiction can restore time, relationships and choice to people who appear as a single category in a table. An efficient policy might change one character's ability to care for a relative; a convenient service might remove another's opportunity to begin again privately.
Those imagined lives are not a sample proving equivalent real-world effects. They indicate questions to bring back to reality, evidence to seek and experiences not yet heard.
Neo Babylon can open a question without proving a forecast
In the published future-city governance essay, I use Neo Babylon's Bay Area City to ask where truth, choice and minority dissent belong when governance provides stable life. That author essay, available in Traditional Chinese, supplies the question here. Questions about the novel itself should return to its text.
The fictional world gives a civilizational problem a shape. It does not establish that current systems possess its capabilities or that its governance must occur. Narrative premises and present technical limits remain distinct.
The comparison of six AI readings of Neo Babylon adds another lesson: different reviewers can identify different ethical tensions in one work. These are model interpretations, not independent verification of fictional canon or evidence of AI feelings.
An author can use criticism to notice blind spots without accepting every reading. Readers can return to the work and offer different reasons. A thought experiment need not announce a single correct response in advance.
Bring the story back to evidence
I would examine an AI ethics scenario through four questions: which assumption matters most, which changed condition alters the conclusion, what is the strongest opposing reason and what real evidence would establish the risk?
This is a proposed reading and writing method, not a new academic classification. It helps fiction and reality illuminate each other while limiting the use of stories to manufacture certainty.
I would watch whether my questions produce more specific empirical inquiries, remain open to other perspectives and actually change when evidence arrives. If a story only confirms what I already believe without exposing a cost, its question may need revision.
That returns the discussion of AI catastrophe to people: do we only want technology to strengthen our desires, or will we examine how our choices might harm others? Fiction cannot perform self-cultivation for us, but it can remove an excuse for avoiding consequences.
Take an AI principle you strongly believe. Write its hardest case, then give the opposing view its strongest reason. Before arranging a protagonist's victory, make the question honest.
Frequently asked questions
Can science fiction prove what the future will be?
No. It can expose assumptions and trade-offs. Actual capability, probability and policy effects still need evidence; narrative plausibility does not establish a prediction.
Must every novel be a rigorous thought experiment?
No. Fiction also serves emotional, aesthetic and entertainment purposes. This article examines how to check arguments when fiction is used for ethical inquiry.
Can AI reviews confirm a novel's setting or ethical conclusion?
No. Reviews can suggest useful questions or misread the text. Canon rests with the work; ethical reasoning needs arguments and counterexamples rather than model agreement alone.
Sources and further reading
Sources revisited October 14, 2026. The hypothetical city and four questions are the article's method. References to Neo Babylon follow previously published author essays.