Truth Under Machine Conditions

1. The Argument

Public discourse about AI talks a great deal about truth, but usually in a form that is too narrow.

It asks: Is the model hallucinating? Are the sources genuine? Is an image fake? Is a video a deepfake? Is an answer correct? Can a claim be fact-checked?

These questions are necessary. But they do not capture the deepest transformation.

For truth is not merely a property of individual sentences. Truth is also a process. It does not arise solely because a statement corresponds to a state of affairs. Socially, it emerges through institutions, practices, and roles: through scientific review, journalistic research, judicial procedures of evidence, archives, source criticism, methodical doubt, dissent, correction, and public traceability.

The article therefore proposes viewing AI not merely as a risk to individual truths but as an intervention in a truth ecology.

This means that a society possesses social and institutional conditions under which statements can be examined, justified, contested, corrected, and made publicly robust. These conditions cannot be taken for granted. They can be strengthened, weakened, or reshaped by technical forms of mediation.

The article makes three proposals.

First: AI-related risks to truth must not be reduced to hallucinations, deepfakes, and disinformation.

Second: Generative AI changes the form of access to knowledge because it increasingly converts research, verification, and interpretation into synthetic answer formats.

Third: AI governance must therefore protect not only correct outputs but also the integrity of truth practices: source plurality, paths of verification, institutional responsibility, invisibilities, correction mechanisms, and time for criticism.

2. Why Truth Is Not Merely Content

In everyday life, truth is often treated as though it were a property of statements: This sentence is correct or incorrect. This report is true or false. This image is authentic or manipulated. This source is robust or it is not.

That is not wrong. But it is incomplete.

Social epistemology has shown that knowledge does not arise only in isolated minds. In Knowledge in a Social World, Alvin Goldman demonstrated that knowledge is anchored in social processes, institutions, and practices: in science, law, democracy, education, and testimony. [1] With the concept of epistemic injustice, Miranda Fricker showed that credibility is socially distributed and that people can be harmed as subjects of knowledge when they are systematically given less credence or lack the concepts needed to make their experiences publicly intelligible. [2] In his social history of truth, Steven Shapin showed that even scientific credibility has historically rested on social norms, relationships of trust, and institutional practices.

Democratic theory and theories of the public sphere also point in this direction. Jürgen Habermas describes the public sphere as a space in which private individuals exchange public reasons and form political opinion. In “Truth and Politics,” Hannah Arendt insisted that factual truths are vulnerable within political communities because power and opinion can shift shared reality. In Why Trust Science?, Naomi Oreskes argues that the trustworthiness of science lies precisely in its social organization: criticism, professional communities, institutionalized review, and collective self-correction. [6]

These lines of thought differ considerably in theoretical terms. But they share one point: Truth is not merely the quality of statements. Truth needs a social infrastructure.

It is precisely this infrastructure that I call a truth ecology.

3. The Concept: Truth Ecology

A truth ecology is the totality of the social, institutional, and technical conditions through which statements become verifiable, correctable, and publicly robust.

It includes at least eight elements.

First: sources. Who says something? On what basis? With what proximity to the subject?

Second: testimony. Who can report, observe, document, or object?

Third: methods. How was something examined? Which procedures are considered robust?

Fourth: institutions. What roles do science, journalism, courts, archives, statistical offices, public authorities, and education play?

Fifth: time. Truth often requires delay: research, cross-checking, peer review, access to records, dissent.

Sixth: dispute. Truth does not arise through harmony but through an ordered capacity for conflict.

Seventh: correction. Error is not the end of truth. What matters is whether correction is possible, visible, and consequential.

Eighth: trust. Not blind trust, but reasoned trust in processes, roles, and institutions.

A truth ecology is therefore not a state of perfect truth. It is a system for the robust production of truth under conditions of error, power, interest, and conflict.

AI intervenes in this ecology because it not only generates content but also changes the paths by which people arrive at statements, sources, interpretations, and judgments.

4. What AI Changes: From Search to Answer

Traditional digital search had already intervened massively in truth ecologies. Search engines organize visibility, relevance, and access to information. But as a rule, they still displayed a list of results and thus multiple possible paths. At least to some extent, the user could see: There are sources, competition, provenance, different providers, and different levels of authority.

Generative AI changes this form.

It increasingly transforms search into an answer. It synthesizes. It smooths. It formulates. It creates a linguistic surface that conveys the impression of settled knowledge. Instead of a landscape of sources, a coherent answer appears.

That is convenient. But it shifts the epistemic situation.

A list of results says: Here are possible paths. A generative answer says: Here is the result.

This shift is not merely aesthetic. It changes the burden of verification. With traditional search, the user had to do more of the selecting, comparing, checking, and weighing. With a generative answer, part of that work is moved upstream and made invisible. The system decides which sources are relevant, which conflicts are mentioned, which uncertainties remain visible, and which differences disappear within a synthesis.

Audit studies of generative search systems demonstrate precisely this problem. In their study of several generative search engines, Liu, Zhang, and Liang found that although answers appeared fluent and informative, they often contained unsupported claims and inaccurate citations. [10] Li and Sinnamon examined ChatGPT, Bing Chat, and Perplexity as new search systems and showed that such systems do not merely reproduce answers on matters of public interest but can establish authority, select sources, and generate bias. [11]

The point is not that generative search is always worse. The point is that it assumes a different role. It becomes not merely a means of access to sources but a preliminary authority on plausibility.

5. Hallucination Is Only the Visible Tip

The term “hallucination” dominates many debates about generative AI. It generally means that a system generates false, invented, or unsupported information. This is a real problem. False court rulings, fabricated studies, false sources, invented names, and plausible-sounding but unsupported claims can cause concrete harm.

But the concept of hallucination narrows our view.

It suggests that the problem exists only where the system is visibly wrong. For truth ecologies, that is not enough. Even a formally correct answer can be epistemically problematic if it obscures sources, smooths over disputes, renders minority positions invisible, removes context, or masks uncertainty.

The most dangerous AI-related truth problem is not necessarily a grossly false claim. A grossly false claim can be checked, disproved, and corrected. More difficult is a plausible condensation that is not wrong enough to be noticed immediately but smooth enough to displace scrutiny.

One might say:

Guiding PrincipleHallucination damages individual statements. Plausibility smoothing damages habits of verification.

This is precisely where the greater danger lies. If people become accustomed to synthetic, linguistically authoritative answers, truth may increasingly be confused with comprehensibility, speed, and coherence.

But truth is not what sounds smooth. Truth is what holds up under scrutiny.

6. Disinformation Is Not Merely False Content

Disinformation, too, is often understood too narrowly. Public debates focus on false news, fabricated images, bots, deepfakes, or coordinated campaigns. These problems are real. With their report on Information Disorder, Wardle and Derakhshan established an important distinction among mis-, dis-, and malinformation. [7] Lazer et al. showed in Science that fake news is not merely a technical problem but an interdisciplinary one involving platform architectures, psychological processing, political communication, and research access. [8] Lewandowsky et al. showed that misinformation can continue to have an effect even after correction. [9]

But here, too, a content-based logic is not enough.

The problem is not merely that false content circulates. The problem is that a society’s capacity to stabilize truth collectively comes under attack. Disinformation can operate on at least three levels.

First: It can create false beliefs.

Second: It can destroy trust in institutions that verify truth.

Third: It can create an atmosphere in which truth appears to be merely a matter of opinion.

The third level is particularly dangerous. A society does not have to believe every lie to be damaged. It is enough for a sufficient number of people to believe that everyone lies, everything is manipulated, and no institution can any longer distinguish between what has and has not been verified.

AI can intensify this dynamic—not only through deepfakes but through scalable text production, synthetic sources, automated commentary, personalized persuasion, apparent expert voices, and floods of information in which verification itself becomes overwhelming.

At that point, the issue is no longer merely fakes. It is the robustness of shared reality.

7. Authority Without Responsibility

A distinctive feature of generative AI is its simulation of authority. The system often speaks fluently, methodically, objectively, and confidently. It can generate footnotes, adapt its tone, summarize controversies, and manage uncertainty through language.

This form creates trust.

The problem is not that AI can never be helpful. The problem is that its communicative form can generate authority without bearing institutional responsibility.

A scientific article exists within an order of publication. Journalistic research exists within an editorial organization. A court ruling exists within a procedure. Official statistics exist within a public authority. A medical opinion exists within professional responsibility. These institutions are not perfect. But they create roles, liability, paths for correction, and accountability.

A generative AI answer can appear to perform similar functions. In reality, however, it is often a synthetic surface without corresponding institutional embeddedness. It answers, but it does not assume responsibility. It formulates, but it does not bear witness. It explains, but it does not participate in a procedure.

This is a core problem of truth ecology under machine conditions:

Guiding PrincipleAI can imitate the form of institutional authority without assuming its duties.

This is not a moral accusation against the machine. It is a problem of social order for the society that deploys such systems.

8. Epistemic Injustice Through Generative AI

Truth ecologies are never neutral. They also determine whose knowledge becomes visible, whose experiences appear credible, and which concepts are available for describing social reality.

This is where the debate about epistemic injustice enters. Fricker distinguishes testimonial injustice and hermeneutical injustice: People can be harmed as subjects of knowledge when they are assigned less credibility or lack collective interpretive resources for making experiences intelligible. [2]

Generative AI can intensify such injustices.

Kay, Kasirzadeh, and Mohamed speak of generative algorithmic epistemic injustice. They show that generative AI can undermine the integrity of collective knowledge and democratic discourse, including through manipulative testimonial injustice, hermeneutical ignorance, and access injustice. [12]

This is central to truth ecology. AI does not merely generate false sentences. It can determine which linguistic spaces, forms of knowledge, and perspectives appear dominant. Models strongly shaped by particular languages, institutions, cultural contexts, and digital sources can marginalize other forms of knowledge. This applies especially to smaller languages, local bodies of knowledge, Indigenous perspectives, non-Western epistemologies, and the experiences of marginalized groups.

The question, then, is not merely: Is the answer correct? It is also: Whose knowledge was included as relevant in the order of the answer in the first place?

9. The Three Shifts in Truth Ecology

The preceding sections can be consolidated into three central shifts.

9.1 From Verification to Plausibility

Generative AI provides linguistically persuasive answers. Plausibility can thereby take the place of verification. This is particularly dangerous because plausibility is a cognitive shortcut. It feels like understanding but can arise without a robust path of verification.

The danger lies not only in people overlooking individual errors. It lies in verification itself becoming a secondary activity.

9.2 From Sources to Syntheses

Traditional knowledge work often began with sources. Generative AI often begins with syntheses. Sources then appear downstream, decorative, or merely selective. This shifts the structure of epistemic responsibility. The source no longer supports the answer; instead, the answer seeks a connection to sources after the fact.

This can weaken scientific, journalistic, and political scrutiny.

9.3 From Institutions to Interfaces

Truth has previously been stabilized through institutions: science, journalism, law, education, and archives. Generative AI increasingly shifts access to truth into interfaces. The interface becomes the first point of contact with knowledge about the world.

The problem is not the interface itself. The problem arises when the interface makes institutional processes invisible, replaces them, or simulates them.

10. Empirical Fields

10.1 Science and Research

Scientific truth does not arise through isolated genius. It arises through procedures: literature review, methodology, peer review, replication, criticism, data access, citation, and institutional responsibility.

Generative AI can support these procedures. It can open up the literature, generate hypotheses, structure data, and improve texts. But it can also damage citation practices when sources are adopted without review, fabricated references are generated, or existing literature is synthesized incorrectly.

The problem is not merely fraud. It is epistemic erosion. If scientific texts look plausible but their sources, methods, and evidence do not reliably support them, the surface of science becomes detached from its internal scrutiny.

10.2 Journalism and the Public Sphere

Journalism is not mere content production. It is an institutional practice of research, selection, verification, contextualization, and responsibility. When generative AI intervenes in news search, summarization, and distribution, it changes more than the speed of information. It changes which sources become visible, which editorial standards apply, and how readers recognize the provenance of statements.

Generative search can ease the burden of journalistic work. But it can also obscure the path between the original source, the editorial organization, and the public. If users see only an AI summary, it becomes less clear which journalistic institution verified the information, what uncertainty existed, and where corrections can be made.

10.3 Law and Public Administration

Legal and administrative truth arises through procedures. Records, deadlines, jurisdictions, duties to state reasons, hearings, appeals, and judicial review are not bureaucratic aesthetics. They are technologies of truth and responsibility.

When AI is used in public administration or law, it must not replace these procedures with plausible preliminary decisions. An automatically generated summary can be helpful. But if it determines which parts of a file become visible, which arguments are considered central, and which cases appear similar, it intervenes in the procedure’s order of truth.

The question is whether the procedure retains the ability to establish truth in opposition to the machine’s preliminary structuring.

10.4 Education

Education is not merely the transmission of knowledge. It is training in verification, formulation, doubt, criticism, and judgment. When AI immediately provides learners with polished answers, finished outlines, and plausible explanations, it can support learning processes. But it can also reduce the friction through which judgment develops.

The harm does not necessarily arise because pupils or students “cheat.” At a deeper level, the question is whether they lose the practices through which knowledge becomes their own judgment: reading, comparing, citing, disagreeing, rephrasing, failing, and checking again.

AI in education must therefore be assessed not only for cheating. It must be measured by whether it strengthens or replaces epistemic abilities.

11. Counterpositions

11.1 Counterposition: AI Makes Knowledge More Accessible

That is true. AI can break down barriers, explain texts, translate languages, make complex research accessible, and help people who would otherwise have no access to certain forms of knowledge.

But accessibility is not the same as a capacity for truth. A system can make knowledge more accessible while weakening paths of verification. The task is not to exclude AI from knowledge processes but to connect accessibility with source clarity, correction, plurality, and institutional responsibility.

11.2 Counterposition: False Information Has Always Existed

That is also true. Rumors, propaganda, forgeries, and errors are older than AI.

But AI changes scale, personalization, speed, cost, and form. It can produce not only false content but plausible knowledge surfaces in large quantities. It can also position itself as a layer of mediation between the user and the source. This is qualitatively significant.

11.3 Counterposition: Fact-Checks Are Enough

Fact-checks are important. But they are not enough.

Fact-checks address individual claims. Truth ecology asks about the conditions under which claims become visible, verifiable, correctable, and institutionally accountable in the first place.

A society can produce many fact-checks and still lose its truth ecology if no one knows any longer whom to trust and for what reasons.

11.4 Counterposition: The Solution Is Better AI

Better models, better retrieval systems, better citations, and better evaluations are necessary. But they do not resolve the problem of social order on their own.

Even perfect answer quality would not eliminate the question of whether verification, source plurality, and institutional responsibility are preserved. The central question is not merely how well AI answers. The central question is which human and institutional practices it replaces, shifts, or strengthens.

12. Consequences for AI Governance

If AI changes truth ecologies, governance must do more than reduce errors.

First: a duty to provide sources instead of source aesthetics. Citations must not be decorative. They must genuinely support statements, be discoverable, and distinguish among primary, secondary, and synthetic sources.

Second: clarity about uncertainty. AI systems must not only provide answers but also make uncertainty visible: missing sources, disputed findings, weak evidence, minority positions, and temporal limitations.

Third: preserve paths of verification. Interfaces should not merely supply users with results but keep paths to verification open: original sources, counterpositions, methodological notes, and context.

Fourth: protect institutional roles. Science, journalism, law, education, and public administration must not be hollowed out by generic answer systems. AI may support these institutions but must not simulate their roles of responsibility.

Fifth: secure epistemic plurality. No single model and no single provider logic may become the invisible default authority for public reality.

Sixth: make correction visible. When AI systems make mistakes, corrections must be traceable, retrospectively verifiable, and compatible with institutional processes.

Seventh: limit synthetic feedback. AI-generated content that flows back into AI systems as source material can narrow truth ecologies. Governance must detect, label, and limit such loops.

13. Conclusion: Truth Needs More Than Correct Answers

The central danger of artificial intelligence lies not only in the fact that it lies. It also lies in the fact that it puts truth into a form that is too easy to consume.

Fast. Smooth. Plausible. Authoritative. With no visible path.

That is useful. And precisely for that reason, dangerous.

A society does not first lose truth because every answer becomes false. It loses truth when the practices through which correct answers become robust grow weaker: source work, dispute, correction, testimony, institutional responsibility, and time.

AI can strengthen these practices. But only if it is deliberately designed and limited to do so.

The question is not merely: Is this AI answer correct? It is: Are the conditions under which truth can still be examined collectively being preserved?

If these conditions disappear, what may remain is a world full of answers.

But no public sphere that still knows why it should believe them.