Article
Towards using Audiovisual Scene Analysis to Evaluate Cognitive Performance in Classroom Scenarios (en)
* Presenting author
Abstract:
In the project ECoClass-VR we successively increase the realism of an existing test paradigm referred to as ”audio-visual scene analysis” by Ahrens et al. Here, subjects listen to stories told by a set of talkers surrounding them. The listeners are asked to identify the content of the talkers and their specific locations, as measures of cognitive performance. We modified the existing test paradigm towards a classroom-type virtual environment and increased its realism mainly with regard to the visual aspect. We present two different types of scene visualization: immersive 360◦ video, with individual talkers captured using a 360◦ video recording and virtual production techniques, and Computer Generated Imagery (CGI) based scenes generated using SketchUp and Unity. We describe the challenges and the technical solutions to achieve close-to-photorealism for two virtual representations and to enable the flexibility required for conducting the targeted cognitive performance tests. We present a subjective test design which involves both visual presentation modes, 360◦ video and CGI. To underline the importance of an accurate acoustic reproduction, we contrast a monaural presentation with a dynamic binaural rendering. The generated IVEs and first results of the series of subjective tests are presented and discussed.