Virtual reality will never be what you want it to be.
What do we want virtual reality to be? We want the complete experience of being someone or something else. We want to be able to do what they can do, see the world from where they see it, understand their life as they understand it. Sometimes we want to be ourselves but in some sort of more exciting scenario, but that’s still more or less the same thing – inhabiting some alternate version of the self that lives in more exciting and fulfilling circumstances is still basically playing a character. There is something greedy, something invasive about it. The sort of greed, not for money or power but for unique perspectives and experiences, that motivated the villains in Get Out, who solve the virtual reality conundrum by essentially hollowing out other peoples’ minds and physically occupying them, living in their reality, colonizing it.
However, say we want to create a simulation of what it might mean to occupy another body, one that does no direct harm to someone. Right now, the gap between virtual reality and actual reality is obvious. Aside from any issues with graphical verisimilitude that we can assume will be addressed to some degree over time with better rendering and artistic technique, there’s a big difference between the experience of seeing through someone’s eyes, hearing through their ears, controlling approximately where their hands are and what they’re doing, and the experience of being that person. It is, perhaps, satisfactory for a simulation of being a robot locked in place, with sensory and interactive apparatus, but even then the virtual entity cannot be wholly inhabited because you still have awareness of your own body, your own place. You cannot escape yourself so easily.
In order to experience what it is to be another person, you’d have to occupy more senses – the senses of taste and smell, the senses of balance and of proprioception, the sense of touch, and while we occasionally make minor forays into some of these with tilting rooms and packaged scents, it is still far from a complete transformation.
That’s still a problem that can probably be solved. We can regard it as something like the issue of graphical fidelity, a problem that is challenging but that we can take concrete steps to approach, bit by bit. There’s a bigger issue. Say we figure all that out, and we create a perfectly convincing all-encompassing simulation of being a star football player winning the Superbowl. I don’t know anything about football, but say you undergo the simulation and experience the entirety of the winning play, from the first pitch through dunking the shuttlecock into the wicket: Who actually did this brilliant, effortlessly physically perfect play? Who ran? Who dodged? Who threw? Who pumped the legs, found the point of balance, who carefully threaded the defenders and perfectly understood the field of play? It wasn’t you, because you didn’t have the lifetime of experience and training necessary to do those things – a person’s unique capabilities stem indelibly from their personal history and understanding of the world. How can you say you’ve had that experience, then, if you didn’t really do any of it?
Thus there needs to be some degree of abstraction. If you’re to control someone who has capabilities you do not, you need to be able to boil those complex micro-decisions down into more digestible macro-decisions. Instead of the tiny piece-by-piece decisions of position and balance, you’re fed the bigger and more understandable decisions of where to run, when to throw, who to pass to, and so forth.
It doesn’t really sound like virtual reality any more, does it? It doesn’t really sound like becoming another person temporarily any more. It sounds like a video game.
If the idea of being able to inhabit one of the characters you play in games sounds appealing, that’s because games are made to only show the appealing sides of their characters. This isn’t some nitpicking realism-critique about characters never needing to use the bathroom, but a lot of character designs, a lot of character animations, a lot if characters are simply not made to be functional. They would be unable to actually draw their weapons, or they would keep falling over, or they would be unable to see past their own clothing if they were a living creature – which is, perhaps, not the experience people have in mind when they imagine what it would be to live as this character.
The point is, art isn’t consistent. Art doesn’t always completely make sense, or create a livable reality. Art is not coherent. That is what makes it interesting, because anywhere there’s a gap in a story or inconsistency in a character or a lack of detail is a place where we are invited to interpret. There’s no bone or tendon to it, no connectivity, merely a series of moments, and in that way art is like dreams, all memory and no substance.
Okay, then. What about lucid dreaming? What about a virtual reality comprised of extremely specific dreams, of remembered moments orphaned from the specific experiences that created them, implanting a perfectly formed recollection of a finely crafted or curated lived experience? This is more or less the plot to Total Recall (and the short story it was based on, We Can Remember it for you Wholesale). Memory implantation is probably the most actually plausible form of a true “virtual reality” – of course, you’d have no actual ability to affect the outcome, but you would remember all of the choices you supposedly made and would rationalize the reasons you made those decisions. That’s pretty much how we live our lives anyway, placing yesterday’s decisions into narratives that make sense based on who we believe we are, since the self of yesterday is essentially a stranger to us. If we’re making fake memories, we can make them perfectly plausible: In John Varley’s novel Steel Beach, a character finding themselves in an implausible tropical paradise lifts up a handful of sand and finds it to be too perfectly detailed for it to possibly be a simulation. However, as the computer running the simulation points out afterwards, the entire beach of sand doesn’t need to be simulated, only the moment of staring at a handful of perfectly detailed sand and of deciding that this couldn’t possibly be a simulation.
We cannot know what it is to be someone else – that experience is forever alien to us. Even more tragically, we cannot really know what it was to be ourselves ten years ago, ten days ago, ten minutes ago – we are severed from our past mind, with only the flimsy bridge of memory and the cataclysmic tower of consequences to tie us to our history.
Virtual reality will never be what you want it to be, and you’ll never be quite sure what actual reality even is.