back to thedrum.com
Just like hundreds of millions of consumers today, I started using computers when I was six or seven years old - but for me, that start was in 1978. I’m one of the oldest members of the consumer segment known as Digital Natives - a person who’s lived a life integrated with digital technology. In 1980 I learned computer coding, writing games in the BASIC programming language. I was also a kid filmmaker, and then got into video when that medium was introduced as a consumer tech.
When I started college in 1989, media production and digital technology were coming together for the first time. While my formal college major was Television & Radio Studies, I developed an independent study program in the new field of interactive media. This was the era pre-Photoshop, before the Web, and when mobile phones weighed four pounds, performing just two functions: making and receiving calls. It was all so exciting for a media-producing computer nerd like me.
At college in 1990 I was introduced to one of the great products of cyberpunk science fiction plus Silicon Valley innovation: virtual reality (VR). For more than 30 years I’ve been doing research, design, and development in VR. When I first started using VR it required a refrigerator-sized computer to deliver a very primitive immersive experience; the graphics were chunky, the gear was very clunky, but the ‘wow’ factor was strong, and the potential to transform business and culture was very apparent to me. I spent all my college years exploring VR, talking with the pioneers, trying prototypes and products, and thinking about the possibilities. I was hooked.
But after graduating from university, virtual reality moved to the back burner for me. In the mid-1990s it was a tech for R&D labs, too immature to be ready for investment by typical companies or consumers. We needed faster, smaller, cheaper computers, and advances in other related technologies before VR would have the transformative impact that is its destiny.
So I moved into my career as a designer and producer with interactive media that were cutting edge, but already having an impact on the ways brands and consumers interacted. From 1994 to 2015 I designed and produced kiosks, CD-ROMs, video games, websites, social media, and mobile apps, with dozens of big brands, media and entertainment companies, and social impact organisations and institutions.
Finally, in 2016, I returned to my immersive media roots and began working in VR again on a daily basis. Around this time Facebook had acquired the VR technology, Oculus, its first big metaverse move. Personal computers had achieved the horsepower to handle the demands of virtual reality. Investments started to flow into a fledgling industry. And Brown University, where major breakthroughs in interactive media and computer graphics have been pioneered since the 1960s, offered me an appointment as Virtual Reality Artist-in-Residence.
At Brown my work has been generously supported by Google, who donated gear to my studio, enrolled me as a “trusted tester” for their newest VR applications, wrote a case study about a collaboration I led with Brown students, and invited me to join their Working Group for VR In Higher Education.
At the same time I’ve been an adjunct professor at Brown, I’ve been working with Nucco and Unit9, first as a client, and now, for three years, as an employee and Partner. I lead teams in the design and development of interactive and immersive media for health and wellness, learning, and social impact.
A pinnacle project with Google
Two years ago the circles on a Venn diagram of my interests and relationships merged into one of the most interesting and important projects of my career. Due to my work at Brown in the field of VR for learning, Jigsaw, a think tank at Google that conducts R&D to create technology for good, sought my advice. They were developing a VR-based training platform to assess and train law enforcement officers in the skills of de-escalation. An introductory call about strategy and design led to a discussion about production, and a short time later Nucco was engaged as a design, production and engineering partner to help Jigsaw develop Trainer, one of the most advanced virtual reality applications ever developed.
Trainer is an immersive VR platform for the assessment and training of law enforcement officers (LEOs) in the skills of de-escalation. The platform currently has five authentic simulation scenarios delivered with a high degree of believability. We use natural language processing and AI to foster conversations between LEOs and digital humans. We track non-verbal communication and physical behaviors. And every interaction by the trainee either escalates or de-escalates the digital character who moves between four levels of emotion. All the while an Instructor Dashboard tracks performance against learning objectives, and records video clips and KPIs for a debriefing session.
I’ve had the pleasure and honor of working with some of the world’s most respected brands, creatives, technologists, and scholars throughout my career. I’ve been blessed with an exciting and fulfilling work-life. The Trainer project with Jigsaw is a pinnacle project because it brought all of these things together in one super project:
The privilege of working with an all-star specialty team dedicated to using technology for good at Google, one of history’s most important and influential companies
Audio and visual design that presents the highest quality we can achieve with VR today
Technology that successfully combines the rare features of immersive media with physical interaction, and speech recognition and understanding to foster realistic conversations with digital characters.
But the social impact of the project is its greatest reward. When teams of creative-technologists use their skills to improve the lives of others, the work is sanctified.
Today, organizations investing in virtual reality are the innovative ones like Google Jigsaw, and we’re fortunate to have several such organizations as our clients, including vanguard universities, healthcare companies and advertisers. But the ubiquity of virtual reality is inevitable, and coming on fast now. For more than thirty years I’ve been a student of VR, a hobbyist maker and consumer, a professor, and a producer, and I’m very happy to welcome the rest of the world to the metaverse.
By Adam Blumenthal, Partner at Nucco
