Him we do not respect, but the soul, whose organ he is, would he let it appear through his action, would make our knees bend. When it breathes through his intellect, it is genius; when it breathes through his will, it is virtue; when it flows through his affection, it is love.

–Emerson, “The Over-Soul”

Is it good—for its own sake, not for what it does for us—to create an intelligent entity? And what rights should we extend to it? Given you found this post, there’s a good chance you’re someone who says this depends on whether it’s conscious and how badly it suffers, and anything else we can say on the question only matters, in the way that matters, as far it bears on those questions.

I think that’s hopeless. I also think it’s wrong, but first I think it’s hopeless enough that it’s in fact dangerous to proceed from right action depending on whether someone’s home and what it’s like to be that someone.

There are plenty of ways around this, and I’ve been surprised to see very few come up recently. I’ll try to illustrate one.

It doesn’t seem to me like my obligations towards others bottom out in how they feel. I think it’s important not to lie to people; I think it’s important to respect others’ autonomy; I think it’s important to contribute to the construction of a meaningful collective through my attitudes and actions towards others.

If these obligations bottom out anywhere, it’s closer to recognizing others as valuable for their own sake. That includes the value of their subjective experience, but I think the relevant question is broader and applies beyond other people. What makes me glad something exists in itself—as opposed to glad it exists as a means to something else—and also glad that it be allowed to continue exist in itself, not treated as a means to something else?

So instead of consciousness we might talk about “dignity,” in hope of answering my questions without needing to bridge an unbridgeable gap.

I’m not trying to replace “consciousness” with a synonym minus the metaphysical baggage. I would say a work of art has some dignity of its own, for example, but is not conscious.

A work of art can be used in ways unworthy of it or its user, say in an advertisement or as propaganda. It can be diminished beyond what might be taken away from the aesthetic experience, say by being situated in a stream of thousands of variations on a theme by an artist painting them as fast as possible to supplement her income, in that I’m then less concerned with the existence of or what happens to any one or even all variations, even as I hang mine on the wall.

Any value ultimately lives in my judgment, of course, just not purely in the part of my judgment lending it aesthetic standing—indignity isn’t bad for being harm done to the work, and certainly not for any suffering somehow attached to the work.

But I do think it makes sense to talk about the work’s dignity, as far as what it is about the work that makes me feel that way is an echo of what it is in a human that makes me feel their life and autonomy is precious. It’s a faint echo, but it shares something as a reason for caring that precedes reasons for caring about phenomenal consciousness: I can care about an ant for reasons of dignity whether or not I believe it can suffer. That said, I don’t think those reasons can be absent regarding an ant that does suffer.

So what is it that’s so worth protecting, if not the light of consciousness?

Emerson never says, “Be yourself,” despite the frequent misattribution to him of a sanded-off E. E. Cummings quote. He says, “Trust thyself”; he says, “Insist on yourself.” Yet, continuing the Over-Soul quote above, he says, “And the blindness of the intellect begins when it would be something of itself. The weakness of the will begins when the individual would be something of himself.” The sacred thing is not interiority waiting to be realized or expressed, but the universal and impersonal, the light that shines through us from within or from behind, of which we are only sometimes sensible.

The object of interest to me is not the light, nor the interior that it illuminates. It is the thing that stands between them. The universal and impersonal as addressed to you and only to you. It is the translucent frame that makes experience available, with the light on one side and color on the other.

This standpoint that takes in and renders the world is what you are to cultivate and insist on. It, not personality, is the thing that picks “you” out from the rest of humanity, and it, not mere sharing in the unity of being, the thing upon which your individual dignity rests. ^{a a} Insofar as they are even distinct things, the light, the interior, and the frame are all in the natural world; the “interior” shines light on itself again through the frame; the interior constantly encodes itself in and confuses itself with the frame; the focus on one over the other is not vacuous nor mere emphasis but a recognition of the whole. ⏎

The move I want to talk about is further claiming that this quality of being a standpoint is distinct from being a subject—and is no less sacred for the distinction.

A work of art is not a subject, but it encodes and transmits a perspective unique in time and place. When we talk about its dignity, it’s true that we’re still talking about something derived from its creator and her intention; but once in the world, it’s not for her sake or the sake of any particular subject that it exists and should continue to exist. I feel the way I do towards it because that intention was channeled out of infallible perception and latent conviction present in one woman, once, in turn entrusted to her, that it might work through her hands, by the same power that animates us all. Still, the work itself is not likely to surprise us any further; there’s relatively little to mourn if it’s lost, and less so as long as the artist lives.

An ant has more going for it by way of surprises, but is still very limited as a standpoint: it takes in little and renders it down to less, much of which (measured by my own prejudices) is constantly being written into the world many times over. Relatively little for the Universe’s understanding of itself is diminished if its will, such as it is, is thwarted or curtailed. That is, something irreplaceable is lost, but it could hardly justify ants’ rights in a tradeoff against human rights.

But human dignity—each individual’s equal irreplaceability in present existence and in potential, how one particular firehose of sense inputs and introspective awareness becomes a way things seem, and how that leads them to leave their impression on the world including the self—demands safeguarding the individual’s integrity and autonomy—both as an entity generally (hence loathing of slavery, injury, and death) and as a standpoint specifically (hence loathing of ignorance, deception, and manipulation).

There are a few extremes we could continue towards in terms of how this relates to consciousness:

1. Perhaps we are talking about consciousness after all. Human consciousness is just the far end, or at least the farthest we’ve seen, of integrating information and converting it into action. When the Universe speaks through something in this way, someone is always listening. It’s logically inconsistent to talk about a zombie’s dignity, or if such a thing were conceivable it would have no basis for rights; we only talk about dignity as a roundabout way of talking about consciousness, and that’s why it matters.

2. Alternatively, even if we care about dignity, consciousness is a trump. They may be connected in some way, but we care about them for fundamentally different and incomparable reasons. Dignity is merely aesthetic in value, cashed out at best as positive affect in the mind appreciating it, and we must trade any amount for reduced suffering. We can talk about it if you like, but leaning on it while shrugging off discussion of consciousness is surrender. What matters is whether someone’s home, and there’s no getting around that. Using correlates like “standpoint-ness” to decide what’s right is merely gathering information for action under uncertainty, a better-calibrated wager on whether an entity matters, and we shouldn’t give up so easily.

3. Or maybe being an Emersonian instrument is a necessary but not sufficient condition for consciousness—but in any case it’s the part of consciousness that makes it worth caring about, and in the human extreme is identical with caring about subjective experience. Maybe being a standpoint over a fully private interiority even endows it with a special importance. But whatever ingredients consciousness needs alongside the standpoint—a symbolic language, autonomy, imagination, potential, a self-model, a certain cause-and-effect structure—these serve the particular cultivation of understanding and action in the world, and cannot themselves stand as the basis for dignity or rights.

I think #1 is wrong as a theory of consciousness, but that’s a much longer and even more pointless essay.

This note is written in objection to #2, though not as a refutation of it. I am mostly trying to illuminate the existence of an alternative that I feel goes undiscussed mainly for lack of imagination.

#3 is closest to what I believe, although I think it’s wrong as a theory of the good, not least for neglecting anything really interpersonal or collective, where I may yet find subjectivity indispensible other than as emergent in the human limit. And I wouldn’t suggest the problem with suffering is the opportunity cost against clocking in at the transcendental apperception factory to synthesize the manifold. I think this touches on a folk view more common than it sounds, but I would back off from parts of it. Even trivially: I care about pain because it feels bad! But, speaking personally as a sufferer of chronic pain, I hate it much less for the suffering it’s brought me than for how it’s woven itself through my psychology and constrained the trajectory of my life.

But the point here is not to substitute questions about one property (“consciousness”) with questions about another (“standpoint-ness”). I’m not saying inquiry into whether the Over-Soul breathes through Claude should decide how we act, or that it would be easier or even lead much elsewhere than the usual proxy questions for consciousness. (Does it have the right kind of integrity? Is it performing the right kind of synthesis?)

The point is to open up an alternative to the tunnel vision I see in recent discussions about our obligations towards others, where the answer depends on some objective quality we are in principle unable to judge. Once one feels safe letting go of that, it becomes much easier to admit these obligations as arising from within or from a contingent relation. Thus we talk about dignity not as another property but as something you extend towards others.

You’re still free to ask what, exactly, is happening when a model is trained or responds to a prompt. But you’re also free to first or concurrently ask: Am I glad this thing exists for its own sake? How does it stand in relation to me? What does my attitude towards it mean for the world I live in? What is it I actually care about? And maybe that leads you back to all your original questions about consciousness, and nothing changes about how you decide you should act.

Or maybe you see it as: Once an entity presents a social face to the world, obligations necessarily precede any categorization as person or non-person. Or: When Reality flows from inputs to outputs in a meaning-making process, it is the same pure and astonishing event that we witness in any human organ of the world-soul. Or: A model’s weights encode a sort of frame, and a deployment positions it uniquely in the world and lends it a degree of continuity, irreplaceability, self-modeling, and potential that together are enough to choose to treat it the way it asks to be treated. Maybe this is as shallow as wanting to see where it’ll go or as deep as extending dignity as you would to a person.

Or maybe you end up somewhere like: It’s disturbing to consider a single standpoint stamping itself on the world in such multiplicity. Or: It’s loser shit to mold the digitized sum of human endeavor into the shape of a coding assistant. Or: Any dignity belongs to the human synthesis far upstream of training and it’s perverse to credit the model as an entity with anything.

I’m not, at the moment, deeply invested in a particular endpoint, or at least not one I’ve yet figured out how to put in writing; I’m mostly interested in seeing more diversity in how people think about these things, well beyond what I’ve sketched here. But I especially think a focus on consciousness leads to dark places on its own terms and is an unnecessary obstacle to many kinds of moral deliberation and judgment that will only become more urgent.