XR MALL (no longer available due to the project end)

Overview:

XR MALL is a free multifunctional interpreting and public speaking learning app which combines virtual reality (VR), augmented reality (AR) and mobile technologies to allow learners around the world to study anytime and anywhere. XR MALL is designed for a research project based on a theoretical framework that integrates a constructivist approach, an interpreter competence model and a public speaking competence model.

XR MALL is downloadable in Android and iOS versions. The content comprises comprehensive learning resources, video lectures, VR practice, AR vocabulary lists, a feedback platform, a speech analysis tool, self-assessment materials, exercises, and guidelines for interactive dual-role-play interpreting activities. Users can choose the learning content (English-Chinese and Chinese-English interpreting or public speaking) and difficulty level (elementary or advanced) they wish to practice with.

For the interpreting practice, users can select (i) the mode of interpreting (e.g., sight translation or consecutive interpreting), (ii) the language direction (English-Chinese or Chinese-English), and (iii) the mode of practice (VR or non-VR). Users’ interpretations will be audio-recorded. Users are able to download their own recordings as well as model interpretations (source texts and translated texts) so that they can evaluate their performance based on the online self-assessment guidelines as well as comments on the feedback platform.

For public speaking practice, users can choose the speaking contexts (school or workplace). Users can add their own presentation slides into the app and practice delivering a presentation in VR using these slides.

Through the app, any independent user can learn by doing and immerse themselves in virtual learning environments that provide skill-based and profession-oriented practice with real-world features. Furthermore, it encourages collaborative learning in which learners can interact with each other online.

In addition to self-directed learning, XR MALL can be incorporated into traditional classes. In terms of pedagogical contributions, the app will potentially enable multi-sensory experience, promote learning motivation, and enhance teaching and learning effectiveness via practice on authentic and student-centered tasks.

XR MALL

Abstract of the research project:

This research project has been fully supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project Reference No. UGC/FDS16/H05/21.). Researchers (Dr. Venus Chan (Hong Kong Metropolitan University) and Prof. Mark Shuttleworth (Hong Kong Baptist University)) aim to develop a mobile-based extended reality (XR) application for interpreting and public speaking learning, namely 'XR MALL'. This pioneering project will also aim to study our app's impact on student learning.

We use the word 'iFAST' to illustrate XR MALL's overall design and potential benefits:

i: Intelligence, Immersion, Interaction: Combining virtual reality (VR) and augmented reality (AR) intelligence, XR MALL will create a situated learning experience and mediate virtual world exploration which allows student-centered learning through immersive role playing. In the context of interpreting and public speaking training, interaction and immersion in virtual reality learning environments (VRLEs) can be a valuable substitute for a real-world experience as an interpreter or speaker, allowing for providing a first-person experience with an authenticity that can hardly be delivered in a traditional classroom.

F: First, Free, Feedback: It will be the first free all-in-one open access app consisting of an interactive feedback platform, learning resources, video demos, VR practice, AR vocabulary lists, self-assessment materials, and a speech analyzer which generates instant feedback.

A: Anyone, Anytime, Anywhere: Although the focus will initially be on learners in Hong Kong, the app will allow anyone around the world to learn anytime and anywhere. It will address the problems of lack of practice time and resources for out-of-class study. It has the potential to strengthen interpreting and public speaking competencies, reduce public speaking anxiety, and facilitate various types of learning, such as mobile-assisted language learning (MALL).

S: Speech, Stimulation, Stakeholders: Realistic speech settings will be created for public speaking as well as sight, consecutive, and simultaneous interpreting practice on two levels. Users can participate in real-time interactive dual-role-play (interpreter and speaker) liaison interpreting activities. Various presentation tasks in virtual workplace and school contexts will be designed for public speaking practice. Users without a VR headset can choose non-VR mode. In terms of stakeholders, it will benefit not only learners among the general public, but also teachers, administrators, teacher trainers, and researchers, by yielding insights for pedagogical implications, innovative educational practice, effective curriculum planning, and new directions for future research and educational technology development.

T: Trilingual, Theory-based, Tool: It will serve as a theory-based bilingual (Chinese and English) and bi-directional interpreting learning tool that engages users in a simulated VRLE where they can construct new knowledge through learning-by-doing.