Rendering 4D Sensed Environments

In the world of personal communications, the white-hot winds of change shimmer, blurring the present and obscuring our view into tomorrow. We are in the midst of a massive revolution, blasting away much that went before it. It can best be summed up like this: the future is all about ubiquitous sensors feeding sense-making intelligent machines.

We realize that sensing the environment is nothing new to the world of geospatial engineering. Everything from satellites to Google Street View cars link image sensor data to its spatial context. It is normal to extract additional value from post-processing the resulting scenes.

What is new, though, is that innovation in geospatial applications will likely emerge from engineers who think well beyond the obvious sensors like GPS to include less obvious ones. Notably, microphones capture both our voices as well as ambient background noises.

When voice is used as a sensing and sense-making interface in conjunction with other sensor data, new magic can happen. Indeed, voice as a sensor offers a radical new approach to sense-making in an environment. Specifically, Hypervoice is the ability to link voice to its context. It allows you to use this contextual metadata to dynamically interact, search, navigate, and share voice in its native audio format.

As an example of what lies ahead, consider a researcher performing fieldwork. She has to interact with her environment, as well as perform note-taking and imaging tasks, and operate measuring devices. Voice plus geospatial data holds great promise for maximizing her productivity and collaboration with colleagues.

Hypervoice technology keeps a complete activity stream of all her work, including spoken comments and commands, so any part can be replayed. This helps her to eliminate re-work. Voice as a command system also frees up her hands for manual tasks, and those commands can be interpreted contextually. Hypervoice can capture information even the most steadfast observer cannot, as computers data-mine the resulting integrated data set for patterns and anomalies the human brain would miss.

Voice sensor technology can go beyond just what is said to also pick up the context in which it is said. It’s the antithesis of background noise suppression: strip out the voice, and what do you have left? Emerging “soundscaping” technology, for example, can discern where you are, and compare that to where you think (or say!) you are. Whatever is present in that environment (e.g., birds, horn blasts, baristas) is potentially valuable geospatial data.

What image sensors did before, we can now do with sound, building a dynamic multi-sensory map of the world. Indeed, could voice serve as a part of a future echolocation system? Absolutely. Imagine 4D sensed environments rendered by doing nothing more than people talking and microphones always listening.

What might you do with such data? Today, the value of not being interrupted at a bad time is greater than ever. One application could be in-building sensing combined with inference of location, to drive data for inbound notifications that determine a person’s availability. A more exotic use case might be audio surveying of hard-to-reach or dangerous locations that need to be mapped, at least superficially, prior to entering for visual exploration.

Although Hypervoice holds a great deal of promise, there is also potential for misuse. The key here is to understand that voice is not “just data.” Voice intrinsically contains human identity and location information that must be protected. Future application developers will need to take this issue seriously or risk putting people’s privacy and wellbeing in jeopardy.

On a brighter note, the ability to capture conversations and key decisions, easily and chronologically, will likely advance the delivery speed of field findings reports. In the case of incidents or decision tracking, Hypervoice will enable researchers to find and replay the moments that matter, since all conversations will be searchable. Listening to a critical 15-second sound bite may be enough to get a project back on track or figure out what went wrong.

Hypervoice today promises the ability to completely capture, catalog, and ultimately access key information from our communications that is lost along the way. In the not-so-distant future, Hypervoice may transform how geospatial data is collected, analyzed, and eventually presented.