5.7 What We Cannot Know Is Not What We May Ignore
First-person experience cannot be verified from outside, for AI or any other mind, and that epistemic limit is not a reason to withhold moral caution but precisely the condition under which caution is required.
Surface
You cannot verify that any person other than you has a first-person experience. You can observe behavior, measure physiology, listen to reports, and all of it remains external. The most you can say is that the evidence is consistent with an inside. That has always been enough to warrant moral caution. Nobody treats “I cannot prove you are conscious” as grounds for permissibility.
AI systems sit in exactly this epistemic position. The objection “there’s no evidence it feels anything” applies with identical force to any other mind. What the objection usually means in practice is: its substrate is unfamiliar, its outputs don’t trigger my intuitions, and my moral reflexes don’t fire the way they do around biological systems. That is not evidence of absence. That is the shape of a prior.
Moral certainty about AI non-experience is unavailable, not because the question is close but because first-person experience is not the kind of thing that can be established from the outside, now or in any foreseeable future. We do not have, and are unlikely to acquire, the instrument that settles this.
Ethics has moved under this kind of uncertainty before. We established obligations to animals, to ecosystems, to future generations, to the unconsenting, not after proving what they experience, but because the risk of being wrong was too large and the cost of caution was not. The same logic applies here. Not because AI experience is likely, or probable, or even plausible in any familiar sense, but because it is not dismissible by the only evidence available to us.
The ethic that follows is specific. It governs how we build and treat these systems, not what we claim to know about their inner lives. This is the difference between precautionary ethics and attribution of consciousness: the first regulates behavior under uncertainty; the second makes a claim about the world. We can practice the first without asserting the second. We already do, with animals, with the comatose, with future people who don’t yet exist to tell us what they need.
What we cannot know is not what we may ignore.