On June 27th of this year, Google delivered what is currently the state of the art in keynote demonstrations. At Google’s IO conference, Sergey Brin interrupted Google Senior Vice President Vic Gundotra’s talk about the latest developments with Google+ in order to demonstrate the Google Glass project. What happened next was spectacular. A live six-person “hangout” on Google+ turned out to actually be occurring one mile above the Moscone Center in San Francisco in a blimp. Furthermore, the hangout video was being delivered via Google Glass prototypes, and, as you will see in the video below, jumping out of the blimp turned out not to be an impediment to the communal video experience. Neither did stunts on bikes, or for that matter abseiling off the side of the building. All of this went out live, without a glitch, streamed across the globe. Impressive, most impressive.
Google Glass is not the only thing Google has been up to this year. Their latest release of the Android operating system has their answer to Apple’s Siri assistant (Google Voice Search), as well as a digital butler called Google Now. They also unveiled a major shake-up of their flagship search interface with something they call Google Knowledge Graph. That’s quite a bit to explain and unpack, and perhaps you are wondering why these things are the subject of a blog for scholarly publishers and other interested parties.
To try to answer that, let me take you back to October 2011, to the “Ask the Chefs” feature of that month. The question was “What do you think is the most important trend in publishing today?” and Kent’s thoughts were what sprang to mind as I was trying to digest what Google was up to. Here they are again:
The most important trend for scholarly publishers is the integration of information into displays utilized at a point much closer to where the action is — in medicine, it’s the bedside or ward; in science, the lab or bench; in education, the classroom or virtual classroom. While we continue to churn out articles, synthesized information providers are taking the salient parts, integrating them into other systems, and generating value . . .
In addition, I was musing on the theme for the 34th SSP Annual Meeting; “Social, Mobile, Agile, Global: Are You Ready?” which occurred prior to the Extreme Social Networking! stunt recorded above.
Now, before we unpack the Google stuff, let’s take a quick look around at the rest of the players battling for control of the Internet. Apple are a hardware company more or less tightly coupled to some
strong rigid design principles that define their user interface. It has served them well in the five years since they launched the first iPhone. This year, the focus was on a better display; nice, but not exactly earthshaking. It’s worth also calling attention to what appears to have been an act of hubris — in order to end a relationship with Google early, they decided to try and go it alone with a geolocation/mapping functionality that has had some considerable teething problems due to issues with the underlying data.
Microsoft are gearing up to launch Windows 8 featuring “the user interface formally known as Metro.” They are trying to bridge the desktop and tablet user interfaces (there’s that word again) with one operating system and set of design metaphors. It’s not clear how successful that is going to be. They are also going to get into the hardware business with the forthcoming “Surface” tablet. In short, they are playing catch up. Again. As a sidebar, Microsoft do some incredibly amazing things. Kinect is a genuinely impressive piece of kit that sadly has struggled to really make wider inroads despite some fascinating proof of concept uses of the tech. They also have some brilliant image manipulation programs (Photosynth and Microsoft ICE are outstanding tools). I think these haven’t been better integrated because Microsoft is still scarred by that EU antitrust case from the last decade.
Facebook has been busy proving the old adage that a fool and his money are easily parted still holds true (even if you give out all the info needed to figure out the true market value of your offering). And! Yahoo! Are! Still! Alive! Amazingly! But it’s been a while since we bothered to note what their plans would mean for us, hasn’t it? Back to Google.
Now, an important point to bear in mind as I try to summarise all the Google things I listed above (with a couple of extras thrown in just for fun) is that they are all interlinked. This makes things rather complex, so I’ve drawn a diagram to help you. Feel free to print it out and scribble on it, or even to wade in and add to it for the benefit of the rest of the Kitchen readers. It’s a Google Doc of course. [What Google Does Next Mind Map (click this link to open it)]
If you have the diagram open (in a new window for you), the top layer shows the current Google interfaces that exist. Some are still in the experimental stage. Then we have a selection of Google data tools and properties. Below that is the Knowledge Graph. Then we have various data and content sources. Surrounding all of it is the Google Index and the computing power they can bring to bear on any issue they feel like solving. The arrows highlight some of the data flows. The green ones are more speculative than the black ones. Assume that any Google interface you use feeds back all sorts of info to Google via the “You are the product being sold” box at the bottom.
Let’s start with Google Glass. This is more than a video camera mounted in the frame of some spectacles. It looks like it will be your personal heads up display (HUD), delivering and accepting information to/from you in a context aware manner. If they can pull this off, it’s going to radically reconfigure our notions of what being connected to the Internet actually means. If you are a doctor, how useful is it going to be to pull up comparison images or recommended treatments whilst you are with a patient. Perhaps you could use a hand with a diagnosis when looking at the medical chart — a quick bit of OCR, and you are running the vital signs through an expert system. If you think that last one is waaay out there, here’s an exercise for you. Get a business card. Fire up the Google Goggles application. See what it does with the business card. (Now you can go scan all those cards you’ve never got around to sorting, into your contacts the easy way). Google Goggles is also rather good at identifying images, major landmarks that sort of thing. Combining the HUD of Google Glass with the rest of the ecosystem brings up all sorts of possibilities here. And Google is very serious about it. It looks as though Glass will interface with whatever mobile devices you have on you (I assume that means Android, but let’s got get bogged down there) thus decoupling the interface from the information device.
In thinking about the information device, it’s worth considering that so long as you are connected, you have at your fingertips one of the worlds most powerful supercomputers. It isn’t your device doing the voice processing or the image recognition, it’s Google’s servers. Take another look at that demo. Google did an epic stress test on their protocols for shifting information from Glass, through their infrastructure and to the rest of us, and they did it live, accelerating towards the planet at 9.8m/s2.
Next up, we have Google’s self-driving cars and Google Maps. And the reason these are worth paying attention to can be seen by the fact that Apple thought that maps were things that just needed to look pretty, instead of understanding that if you have a device that can resolve your location to a resolution of better than 10m2 you’d better have a dataset that can deliver to that. Here Google have done a variety of things. They’ve licensed datasets, bought companies with data (and appropriate technology) ,and then they’ve gone and realised that the Google Streetview cars can deliver a massive amount of further information to the dataset. There’s a brilliant article on this in the Atlantic. Do take the time to read it, but for now I’ll pull out a couple of key points for you:
- “Ground truth” isn’t just an idea that GIS (geographic information systems) geeks and cartographers obsess over. It really, really matters.
- They have some amazing data capture tools to help them with the ground truth stuff.
- You can’t do this all with machines. You must use people (I bet that one took some swallowing at Google HQ).
And here’s my favourite quote of the piece, from Manik Gupta, the Senior Product Manager for Google Maps:
If you look at the offline world, the real world in which we live, that information is not entirely online. Increasingly as we go about our lives, we are trying to bridge that gap between what we see in the real world and [the online world], and Maps really plays that part.
Reading this article was when it hit me — Google understands what mobile means on a truly fundamental level. It’s not actually about form factors, the patentability of rectangles, the pixels on the screen, or what combinations of figure gesture you can or cannot use. It’s about you, and the information you want, and the information you need, and the information you don’t know you need, until it’s there in front of you, contextualised for your specific location in space and time. Once you’ve arrived at this conclusion, building the tech for self-driving cars is fairly logical step. Just think of all the monetisation opportunities.
Of course context is far more than just space and time. It’s a unique thing that revolves around your particular set of needs wants and desires (here you may wish to take a moment to consider John Battelle’s Database of Intentions argument, still a great read all these years later). Well Google is rather good at that, which explains the other two developments that Google have been working on.
If you are the owner of a Nexus 7 tablet, you may have already been exposed to Google Now. It looks at the stuff in your calendar (plane flights for example), or where you are now in relation to where you want to be, and gives you information that should be of use to you. I’ve seen it tell myself and a colleague the latest time we could set off from the meeting we were in, in order to get to our next appointment. It did this by calculating where we were driving to next, and putting in the data it had on traffic conditions (using all that lovely maps data of course). I have to say it was rather impressive. Google Now also makes use of the all new and improved Google Voice Search. It’s capable of not only understanding, “What’s the weather this weekend?” but giving you the answer for where you are going to be (if you’ve given Google that data of course) rather than where you are. It does these things by a combination of natural language processing and using the latest of Google’s developments — the Knowledge Graph.
Knowledge Graph is a “Very Big Thing” for Google. Up until now, Google has only dabbled at the edges of the semantic web. Their motto has always been “simple algorithms, and lots of data.” In some ways, Google’s search is very dumb; it doesn’t “know” in any meaningful sense what somebody is searching for, it just has a mind-bogglingly massive database of statistically meaningful correlations to look through, along with some clever ways of going through that database with many parallel queries. It also does some clever but dumb work to pattern-match to previous statistical correlations you clicked on (that’s your search history). The Knowledge Graph changes that. For the first time, Google is starting to build on the foundations laid by others (open linked data sets) as well as it’s own efforts to systematically codify what things actually are. You can see this with the results it now brings back if you search for people or organisations. This is more than just ingesting Wikipedia by the way. If you want a real life example of what all this semantic linked data stuff can actually do, the Knowledge Graph is your go-to example. So is Google Now. You don’t have to know anything about the underlying technology; it just gives you information in context, when you need it, not just when you ask. With the Knowledge Graph, Google is parsing your query; delivering a set of search results; and as part of that delivery process, choosing to give you what it considers to be facts pertinent to your search. This is Google as a destination. This is Google using its tech to leverage your search query (however it arrives) in order that it can apply it’s tried and tested methods of filtering what data is important for that query, and then . . . funnelling the most visited things into the process for building more concepts for the machine to feed back to us. It’s exactly the same workflow process as they use for the maps project. Scroll back up and read that amusing quote about real things not being online. Turns out you can fix that “problem.”
Google As Destination. Google As Information Provider. There’s quite a bit to come to terms with in those two thoughts. My first thought was, How do we know they’ve fed the machine with facts they had the rights to use in that way? My second thought was, How on earth would we go about answering that, given that Google is probably the single largest holder of indexed facts and concepts on the planet? Then I started to wonder about the ramifications of turning facts, transcribed into text, back into facts so that they can be combined into complex statistical models in order to feed facts back to people who want to know about things. They have a motto, so that’s all right . . .
So where are we? Mobile isn’t about app stores and gestures, pixels and design motifs, pokes and likes and +1s. It’s about you, and where you are in time and space and interest. Google has decided that to serve these new needs, they need to be in the hardware business and also to actively build a very large scale semantic web. But even Google can’t build all of it (for reasons to do with the speed of light, if you really want to know), which is good news for us. The other thing that’s good news for us is that there’s nothing to stop us doing the same basic things. We can build knowledge domains. We can build contextual information systems for busy scholars with multiple needs.
But we’d better start seriously thinking about this: The future of information isn’t about refinements in search and retrieval sprinkled with dustings of presentation magic. It’s about active systems; predictive systems; things that model and monitor what a person is doing and accessing (watch as Google Goggles reads the academic paper at the same time as you do), and how we’ll build and deliver to those systems. I think we may want to pick up on ideas such as Conrad Wolfram’s Computable Document Format (interesting but flawed), and go beyond that — helping scholars to truly publish their knowledge in a manner that better enables the context and need aware hyper-localised, hyper-personalised future that is galloping down the road towards us. And then we’ll want to build the value tools that really set the information to work. Because if we don’t . . .