Talking CT Head

The Talking CT Head is an integration of ps-medtech desktop virtual reality, IC3D Media game technology, and the Luminis PulseOn e-learning system. SELEMCA agent technology will be applied in the next phase.
Ps-medtech is the company that developed the VR hardware and made the 3D CT scans navigable. Medical Technology Inholland provided the expert knowledge on, for example, radiology. Their students and teachers are the end-users of this system. IC3D Media is a company dedicated to the development of game engines and avatar technology. They designed the mouth layer over the CT scan head. They moreover developed the audio so that the head can talk about itself. Luminis/PulseOn is a company that delivered the e-learning platform for personalised learning.

What is it, for whom?
The Talking CT Head is a product-service system that supports e-learning in higher education, designed in a student and tutor-centered way, with great opportunities for scientific outreach to the public at large (e.g., NEMO, Corpus) and public health education. It induces the much-wanted “wow” effect, raising curiosity and instigating the motivation to learn.

Commonly, e-learning in higher education is associated with downloadable lectures, online tutorials, Podcasts, tablets, dedicated discussion forums, and sometimes even e-mail. The benefits are sought in solving logistical problems of the organisation (e.g., large-scale distribution of content and large-scale assessment of students) as well as measurable cost savings. The students can work more efficiently, because they are not constricted by a timetable and do self-assessments independent of time, place, or tutor. True and desirable as this may be, SELEMCA studies and designs an e-learning system that goes beyond the ‘digital filing cabinet,’ concentrating on pedagogical interventions on the fly.
In collaboration with Inholland and companies from the Creative Industries, SELEMCA developed a whole new learning concept for students of Medical Imaging and Radiation Therapy (MBRT). Here, a 3D Virtual Reality technique is used to visualise several scans of the human body (MRI, CT, etc.) such that the scans can be manipulated, while being educated through SELEMCA-designed emotionally adaptive “agent technology” (work in progress). In the 3D scans, students can “touch” organs and body parts while the system interprets and explains what they see. This way, the students learn anatomy and how it is represented on different scans. Knowledge can be tested, using the same system. This innovation makes students much more aware of the 3D aspect of treatment (i.e. radiotherapy) and diagnosis in the body. This applies not only to students of MBRT, but also for higher-vocational students of HBO-V and the medical masters ANP and PA. In time, this project offers good chances to qualify for external funding and exporting the concept to various other medical schools outside the university. The system will also attract media attention, serving scientific outreach and general health education.

How it works
The ps-medtech C-Station is a desktop virtual reality system - a 3D viewing station - designed for the medical professional for the analysis of complex 3D en 4D data such as MRI, CT, and Ultrasound in a way not previously possible in a traditional computing environment. Through optical tracking of multifaceted “dice,” the student can explore the brain architecture (or any other body part) in 3D. Enriched with IC3D Media animation and sound, Luminis’ logging and adaptation tools for personalised e-learning, and SELEMCA’s affective understanding of the student (work in progress), the educational content becomes self-explicating, interesting, and decidedly motivating.
SELEMCA’s emotion-simulation software will be applied to an animated talking head built of CT-scans, explaining what the student sees and what to look for. Enriched with exercises, the student’s progress can be logged by the PulseOn system and content matter can be adapted to personalised learning styles. The emotion simulations will help to motivate the student. SELEMCA’s inhouse designed creativity-software connects the educational contents to special interest areas of the student (e.g., a journey through the winding lobes is framed as a hike through canyons and gulches).

Next step
While the virtual reality system offers educational content (e.g., to diagnose trauma in a CT scan), in the next phase, agent technology will be connected to electrodes and sensors, measuring and responding to the student’s behavioural and electro-cortical patterns. Suppose, certain teaching styles become predictable after a while, then alpha-wave activity increases. Traditionally, this would lead to a wake-up call to stir arousal and insisting on focused attention, not changing the teaching strategy but changing the student’s state of mind. In creative teaching, however, the moment of reverie is where problems are solved in an associative style rather than rationally so that the system changes its teaching strategy and offers associatively related contents from which the student can solve the problem from a completely different angle. For example, if the challenge is to recognise brain-tissue softening in a range of CT scans and the student fails repeatedly or falls asleep, the system offers a cooking app to bake a pudding with gelatine and tofu. Then it returns to the original question, asking the student to mentally apply the molten gelatine images to the CT scan. Performance is measured through PulseOn and the strategy that renders most correct responses is continued. In other words, e-learning new style is a merger between neuroscience, interactive multimedia, pedagogy, and creativity.