All The Graphics In The World Won't Save You

“Fiction is an ancient virtual reality technology.”
~ Jonathan Gottschall, The Storytelling Animal

Higher resolution. Lower latency. More polygons, more pixels, more motion controls, more, more, more.

I'm from game development. There's a lot of focus there on improving graphical fidelity, frame rates, and photorealism. And now with the rise of virtual reality, throw into the mix: head & hand tracking, haptic feedback, and other various devices to imitate reality reality.

I have a few aesthetic objections to photorealism - of all the visual styles possible with a computer screen, a digital canvas totally under your control, you choose... to imitate the look of the world that's already all around us all the time? That just seems like a lack of imagination and a waste of potential. But above all...

Surface-level "realism" does not last.

I'm reminded of the old, old 1896 film, Train Pulling into a Station, a 50-second movie where a train moves towards the camera. Yeah, that's it. As urban legend has it, when audiences first saw the film, they panicked and fled to the back of the room. But of course, with time, we all got used to it. Same thing once film had sound, color, special effects, HD, and even 3D. We all get used to it. (in the case of 3D movies - for many peeps including myself, bad 3D sickens me and undos any immersion I had.)

I can't help but feel that the current state of virtual reality is like Train Pulling into a Station. Sure, it's impressive now, but soon the surface-level stuff won't matter. But think about what film theory is today! It's not about how to create the most spectacular photorealistic scenes. It's about the framing, the editing, the everything that's done in service of visual storytelling.

Surface doesn't last. Story does.

It's the classic "form over function" mistake.

I guess it's a problem all new artists & mediums go through. At first, everyone can only see the surface-level stuff, because, well, it's the surface. It's the first thing we notice. And all of us go through this stage - I know I'm always going through it, as I try to learn new art forms and mediums.

I hope this post doesn't come off as dismissive of virtual reality. On the contrary, I'm pretty excited for it, and I've experimented with it myself! So while it's good we have continuing research into graphics performance and motion control precision, I think that's all surface, and instead, we should be thinking more about how all this new technology lets us better communicate stories.

For example, in VR, the viewer controls the camera. How does this constrain us, and how can we use this fact to enhance storytelling? I've written before about a few techniques I saw in the very-not-photorealistic-and-yet-stunningly-brilliant Windy Day, such as...

  • Using your head's inertia for comedy
  • Changing things subtly when you're looking away
  • Pausing the action if you're looking away

...just to just name a few.

When films got colour & sound, it took us a while to make full use of these new additions. Sure, first we replicated reality, but then as the medium matured, we learnt to manipulate lighting and do musical scoring to convey moods, to better communicate the stories.

So what's new in VR and VR games?

  • Viewer control of camera
  • Spatial awareness of a scene
  • Depth perception
  • Direct manipulation
  • Kinesthetics
  • Interactivity
  • Eye contact

...and more. I'm excited to figure out what new functions these additions can serve. I wonder how they can help us tell stories better, and with luck, tell better stories.

When I was at GDC, one quote from a VR storywriter stuck with me:

“Immersion is a narrative problem.”

And in hindsight, it's kind of obvious, isn't it? If a skilled writer can make a bunch of ink symbols on dead paper feel immersive, that's a strong testament to the power of story.

I wonder if all that money and research going into higher resolutions and higher framerates, in an attempt to solve the problem of "presence", couldn't just be solved by looking below the surface, and looking for better stories.

Coz if you think "looking around in 3D" will still be impressive in five years, I got two words for you:

choo choo