Building resilience against tsunamis with 360-degree video and virtual reality

Yale-NUS faculty from the science and humanities disciplines have teamed up to create a Virtual Reality for Disaster Resilience (VR4DR) platform where experts can have a realistic view of earthquake and tsunami prone neighbourhoods in Padang, Indonesia. Using the observations made possible through this platform, research experts, investors and community leaders can work together to maximise community assets to help these neighbourhoods build resilience against tsunamis.

The approximately two-year project was the brainchild of Professor of Science (Environmental Studies and Geophysics) Brian G McAdoo, Instructor of Humanities (Documentary, Photojournalism and Visual Communication) Tom White and Lecturer of Humanities (Visual Arts) Chen Yanyun. The team hopes to release a prototype for VR4DR later this year.

Prof McAdoo has been working with non-governmental organisation (NGO) Komunitas Siaga Tsunami (KOGAMI) since 2006 to find ways to reduce the impact of tsunami losses at Padang after the region faced the devastating Indian Ocean tsunami in 2004. In 2017, after working with faculty members from various disciplines, Prof McAdoo envisioned a new dimension in the project. Then the Rector of Elm College, he hosted Virtual Reality (VR) expert and founder of Immersively, Mr Lionel Chok, at a Rector’s Tea. Looking at Mr Chok’s work on 360-degree video, both Prof McAdoo and Mr White, who also attended the session, saw the opportunity for VR to capture an environment in its entirety, hence enabling researchers to document the full scope of a tsunami-prone site in Padang. Unlike the traditional modes of videography and photography, spaces that might be important for research will not be left out.

“For example, as a geologist, I might only take photos of rocks, and miss the health clinics. An investor charged with supporting businesses would perhaps notice the markets and shops, but miss the temples and mosques,” said Prof McAdoo. “If we were to walk through a neighbourhood with a 360-degree video camera…[that would] paint a more complete picture of the build and social environment that would one day have to withstand the onslaught of a disaster.”

Mr White, whose role is to produce photographic and videographic content for the platform, believes that this allows experts who are not at the site to assess the area more comprehensively. They are able to “look around” and “possibly spot important information that may have otherwise been missed”. Such extensive documentation gives researchers a clearer idea of the plausible damage a community would have to withstand during a tsunami. Mr White is currently incorporating narratives and content from the Padang communities and experts into the platform.

The project was pushed to the next level when Lecturer of Humanities Chen Yanyun (Visual Arts) devised a way to make it more aesthetically engaging – the 360-degree videos could be put into a gaming environment where the viewer could move around and explore. A floating orb within the platform would feature another 360-degree video explaining the narrative of the space being explored.

To put things into perspective, “imagine a Google Street View with interviews with locals that you come across as you explore,” said Prof McAdoo.

A model of Purus, a coastal area in Padang. Image provided by Brian McAdoo.

Using her artistic expertise, Dr Chen created a sketch deck for the VR4DR platform. The team is now working with Jirasiri “Nummy” Techalapanarasme (Class of 2019) and Singapore-based computer graphics professional Mr Alex Scollay, who has worked on movies such as Avengers: Infinity War and Solo: A Star Wars Story towards releasing a prototype by the end of the year.

The aim is to put VR4DR online, where a curated team of experts can comment on key elements of resilience and vulnerability in earthquake and tsunami prone neighbourhoods. Examples include an NGO identifying a busy health clinic where they can deploy vaccinations, or structural engineers spotting buildings that could serve as refuge shelters. These observations will be shared with the investors and leaders of these communities who can then decide how their communities’ assets can be maximised to build resilience against tsunamis, and how they can strengthen liabilities that weaken their neighbourhoods.

Padang in Indonesia sits in a zone vulnerable to the threat of earthquakes and tsunamis. Image by Tom White.

Prof McAdoo believes that these well-informed investments will build resilience in communities. After a disaster hits, communities can rebuild their lives in an area with less losses and achieve a more rapid and better recovery. Additionally, VR4DR serves as a bridge between marginalised communities and investors as a more accurate profile of the communities’ socio-cultural norms would be visible.

The team intends to expand the project to other parts of Asia. In May, Prof McAdoo and a research associate will travel to Pokhara, Nepal, working with Kathmandu Living Labs to help map sections of the city.

VR4DR reflects the multifaceted nature of the faculty’s work and the benefits that can arise when different fields are merged together. “This project requires skillsets from different fields, and enriches a user’s experience when different technologies, narrative devices, levels of engagement, and journalistic approaches come together,” said Dr Chen. Such interdisciplinary collaborations that can take place in a liberal arts and sciences college are progressively more important in the face of increasingly complex challenges.

Mr White said, “The question for me is not whether the work we should do be interdisciplinary or not, but how interdisciplinary can it be, what are the connections that can be made, and how can those collaborations operate?”

He emphasised: “Interdisciplinary activity is the natural state of the world!”