The research project “3D Projections to Support Medical Training and Interventions” has been carried out by OVGU Magdeburg, Stimulate Campus Magdeburg and domeprojection.com®. It aimed at investigating the benefits and usage of augmented reality projections in the context of medical diagnosis & planning and computer assisted surgery, as well as education and training of medical students and staff.
The med projection setup allows to project and manipulate 3D medical volume data, e.g. anatomy, MRT data or derma structures projected on a physical 3D body surface. The volume data are projected onto the surface corresponding to the users viewing perspective and they are adapted to the users movements in real time.
A set of interaction techniques, e.g. 3D gestural interactions to manipulate medical data, have been explored and evaluated by our research partner.
Benefits of projected AR
Augmented 3D projections allow exploring and interacting with medical volume data in an analogue 3D context. There are no additional VR Glasses required.
The user can identify anatomical structures and their spatial relations easier than using 2D slice images only. The cognitive load of matching 2D image data on a flat display with the according 3D structures in a physical body is reduced. Developing mental maps is facilitated.
In comparison to pure VR technologies AR projections retain the context of reality. Difficulties of users regarding virtual reality sickness are avoided.
Advantages of augmented 3D projections:
- exploring and interacting with medical volume data in its physical 3D context
- easier identification of spatial relations compared to 2D slices views
- no additional monitors are required
- no additional head sets are required
- no VR sickness
- Projection of VR and AR Scenes with Unity3D and Unreal
- Projection of 3D Object surfaces and volume data
- Projection volume: 1600 x 1600 x 650 mm (extendable)
- Resolution approx. 1,25 mm / Pixel
- Integration of HTC Vive Tracker and Controller
- 3D high resolution scanning
- Fast recalibration
Bornemann, K.,Heinrich, F., Lawonn, K., Hansen, C. (2020)
Exploration of Medical Volume Data in Projective Augmented Reality: An Interactive Demonstration
Proceedings of Mensch und Computer, Magdeburg, Germany, pp. 507-509
video on acm
Heinrich, F., Bornemann, K., Lawonn, K., Hansen, C. (2020)
Interacting with Medical Volume Data in Projective Augmented Reality
Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Lima, Peru, 12263, pp. 429-439
Heinrich, F., Bornemann, K., Lawonn, K., Hansen, C. (2019)
Depth Perception in Projective Augmented Reality: An Evaluation of Advanced Visualization Techniques
Proceedings of ACM Symposium on Virtual Reality Software and Technology (VRST), Sydney, Australia
video on acm