The moment it hits you, is the moment when you hear the plane flying overhead [Youtube 360video: opens in new tab]. The drone of the engines get louder and louder, and around you the refugees become increasingly agitated. It’s hard to locate the plane against the sky, but suddenly there it is. Small packages burst out from it, tumbling to the ground, and the people make a break for them, even whilst the supplies are impacting the ground; dust and soil spraying everywhere.
I’m sitting in a small one person spacecraft. In front of me lies the place we call home; Earth. As I watch, the Moon rotates into view. Just at the edge of my vision, I sense something and turn. It is vast and beautiful and awesome and terrifying… It is the blistering heart of the solar system; Sol, our star, almost blinding me with the light it puts out. A quick flick of my head and the music swells as my craft heads out into the solar system.
This is VR.
Virtual Reality.
And I’m not using some fancy headset bolted to a thousand dollar PC, I’m using my phone (Chinese; unlocked; cost less than £300), and a VR headset that cost £50 (I was so impressed I upgraded from the £12 headset I started with). This is the future of something.
The first example I described, is “The Displaced”, the inaugural VR documentary published by the New York Times. They shipped a million Google Cardboard kits to their print subscribers so their audience could experience this. The app that plays the video is free, and the documentary is absolutely superb. Sure, you can watch it on Youtube, and pan around to get the 360 view, but in VR, you can be there. There are moments that will move you with their intimacy and power.
The second example is arguably the leading demo for practical uses of virtual reality in an educational context. “Titans of Space” is an absolute gem. Its premise is to show the scale of the bodies of the solar system in relation to each other. You know that the Sun is the overwhelming mass of our system, with Jupiter in a distant second place. “Titans of Space“ shows you what that really means. When you are craning your head up to try and see the upper edge of Jupiter, or surfing across the surface of a star so large, you cannot see but a fraction of it, you truly get a feel for the planets and moons of our galactic neighborhood. The image abovet is me demonstrating it to a group of publishing types at a conference recently.
Virtual Reality has, to put it bluntly, a pretty poor history. The VR of the nineties most certainly over-promised and massively under-delivered. But Moore’s law, coupled to increasingly higher density LCD screens and the arrival of cheap and accurate inertial sensors has changed the game. VR is possible today precisely because of the revolution wrought by the mobile phone. And (and this bit gets missed all too often), the environment generators of today’s modern games industry.
This is a fundamentally new metaphor for interacting with Information.
My experiments so far have involved exploring the applications above (uses cases: education; social commentary; documentaries; journalism). But there’s more. With the application of some desktop software, I’m able to turn my phone into an extended display and have 3d views of various protein structures (a major upgrade on rasmol for any ex-molecular biologists in the house) as well as walk through archaeological data obtained via LIDAR (the amphitheatre at Pompeii [WARNING: this will stress test your browser]), and using the new android version of Google Street View I’ve walked the lost city of Angkor Wat (use cases: 3D molecule viewing; archaeology; geography; history). Take a moment and think to yourself how many scientific visualizations in books, journals or other outputs could benefit from a VR rendering, allowing the user to truly inspect the data, not have to rely upon some two dimensional rendering of a far more complex dataset.
The Google Cardboard iteration of VR is quite frankly a work of genius – probably by luck more than design. The fact is, long VR experiences can become difficult. Being on the receiving end of a highly stimulating, realistic visual and sonic experience that does not match what the inner ear is reporting can be a challenge. But the beauty of Cardboard VR is that it’s intended to be a more casual, shorter scale experience on the order of a few minutes, perhaps 15 at most. The ridiculously cheap cardboard phone holder is surprisingly capable, and as I’ve already indicated, more sophisticated headsets aren’t exactly going to break the bank. Oh, by the way – the cardboard design is open to anyone to reprint, so distributing a version with your branding on it is simplicity in itself.
It’s still early days. The game engines (Unity, Unreal Engine in particular) are still working out some of the kinks in terms of support for the various different environments (Cardboard, Oculus Rift, Gear VR and more) but I have more than a sneaking suspicion that this will play out as follows: A couple of ‘High End’ devices such as Oculus Rift will support the equivalent of the audiophile user, whilst cardboard has the clear potential to occupy the more casual space. And the casual space is where we should be.
Alongside the 2d representation of some data, it’s not beyond our capabilities to enable the reader of our various outputs the option to slide their phone into a journal/conference provided cardboard viewer, and see and interact with a far better representation of the data. Why couldn’t a paper about an archaeological investigation offer up the chance for the reader to virtually experience the dig.
Like this maybe:
This isn’t a gimmick. I think there are some really compelling uses cases for VR in precisely our areas of interest. Let me know what you think. Below are some links and explanation for those who want to dig into this a bit more.
So you want to know more about VR:
- It works on a surprisingly wide number of phones (even an ‘ancient’ iPhone4s*), the major issue is having a physical screen that’s big enough to form a decent field of view for your eyes. You want something with a 4.5 inch screen or larger ideally. More pixels gives a better quality VR image.
- I’m properly short sighted. Honestly, the experience can vary a bit here. My current viewer (a freefly VR headset) works fine without glasses. The cardboard viewers allow for ‘manual adjustment’ (move it till you get focus), but some other viewers can be a bit of challenge.
- Headphones can really add to the experience
- Sitting down on a swivel chair isn’t a bad idea.
- Do take breaks.
- Get your viewer (or make it! go on!)
- Download the Google Cardboard VR app. You can find it on both Google Play and iOS stores.
- Then go and download the New York Times VR app to experience the documentary.
- Watch the documentary “The displaced”
- Go and get “Titans of Space”
- Download the Google Street View for Android/iOS app to experience places like Angkor Wat (or just about anywhere else where you can enable street view).
- And don’t forget to visit Youtube for the 360 degree videos (which vary from ‘gimmick’ to really very compelling indeed; I recommend the discovery channel videos with sharks).
- Make sure your friends and family can’t take ‘hilarious’ pictures of you with a cardboard box strapped to your face.
Enjoy!
*You will go cross-eyed trying to see anything with an iPhone4s, the screen isn’t big enough.
Discussion
13 Thoughts on "Virtual Reality and the Scholarly Publisher"
Good news indeed. At one point I experimented with 3D fly thru, as opposed to flat earth walk thru. It is great but 3D navigation is difficult. There are potential scientometric applications. The tangled looking citation and co-author networks we see today are really 2D projections of 3D structures. Interacting with them in 3D might tell us a lot about the structure of the scientific community, including how it is evolving. This may also be true for scientific communication.
There’s another example to add to my list. It’s also worth noting that game developers have basically solved the issues of navigating easily through complex 3d environments.
Good to know. How is the navigation done?
Basically Good Heads Up Display UI; Way-points, Maps that build where you’ve been vs unexplored areas, and sometimes, ‘ghost’ paths that indicate possible routes or where you’ve been
Another potentially useful 3D structure that I did a lot of research on is something I call the “issue storm.” When a major issue arises in a community or an organization, there is a lot of message traffic. Messages are links between people, so this traffic can be visualized as an evolving set of links. The point is that different issues can have very different patterns of links. The nature of the pattern for a given issue storm tells us a lot about the issue and how to deal with it, including predicting its course.
Our lives are increasingly issue driven, which means we are constantly dealing with issue storms. This is due to modern communication technologies, which can involve large numbers of people very quickly. Traditional management techniques, many of which were developed long ago, are inadequate for dealing with issue storms. Seeing the patterns is potentially very helpful.
Just to elaborate (since no one else has), in 3D fly thru you can go forward, backward, left, right and up or down. You can also look forward, up, down, left and right. With all these commands it is easy to lose track. My favorite error message was “you cannot see this object because it is behind you.” My wife created a game called Find the Refrigerator. It was a relatively small multi-room house, suspended in space, with various rooms of furniture. It turned out to be very hard not to get lost. Mind you once she put the refrigerator under the car in the garage, which I thought unfair. We gave the game to the Air Force Academy, to test potential navigators with.
“There are a lot of reasons to believe the next major computing platform is going to be more directly connected to vision.” This interview with Mark Zuckerberg on the trends that led Facebook to bet on VR/AR as the next platform is worth viewing:
This seems to be the classic Hype Cycle in action: let’s say Second Life was the peak of inflated expectations; we’re now starting up the slope towards ‘productivity’.
Also worth mentioning Microsoft Hololens in terms of making VR/AR (“mixed reality”) mainstream. MSFT’s investment in gaming looks like a very strategic investment. [10:30 in http://www.microsoft.com/october2015event/en-us/microsoft-live-event%5D
Great and timely post, thanks!
And thanks for the link to the video and the call out on the hololens, which is one very much to watch!
360 cameras are becoming much more affordable and ‘mainstream’. Ricoh Theta and Bubl are two that I’ve researched more in depth. I think over the next year or two we are going to see some really interesting and innovative initiatives being brought forth with this technology.
They are. There’s a Kodak (branded) 4k camera that has just shipped, and I know of a guy who has persuaded 3 of the cheaper gopros (original heros) in a custom caddy, to deliver 360 video. There a Google+ (yes, srysly!) Community for cardboard. Worth keeping an eye on.
Another thing worth considering–VR seems to work better for men than for women, with a suggested mechanism involving how different brains do spatial perception:
http://qz.com/192874/is-the-oculus-rift-designed-to-be-sexist/
I do wish people knew what the term sexist actually meant… But that’s a digression for what is a fascinating article. Cardboard doesnt have the power (yet) to run into those problems I suspect, and in theory it only would matter when you are rendering an environment, so video etc should be fine… I know the rift R&D has basically discovered yhat latency is the killer here, the time has been spent getting that problem solved.
