Haptic Augmented Reality System

In our current research we examine the integration of haptic interfaces into augmented reality setups. The ultimate target of these endeavours is the application of the framework to training of manipulative skills in surgical environments. To this end, highly accurate calibration, system stability, and low latency are indispensable prerequisites. Therefore, we developed a new calibration method to exactly align the haptic and world coordinate systems. Moreover, a distributed framework was created, which ensures low latency and component synchronization. Finally, to demonstrate our results, we integrated all elements into an augmented reality haptics ping-pong game. (Video 1)

Publication: G. Bianchi, B. Knörlein, G. Székely and M. Harders, "external page High Precision Augmented Reality Haptics", Eurohaptics 2006, July 2006

The driving force of our research is the precise combination of real and - possibly indistinguishable - virtual interactive objects in an augmented reality environment. This requires an interactive, multimodal simulation, as well as stable and accurate overlay of the computer-generated objects. This paper describes several methods to improve accuracy and stability of our hybrid augmented reality system. In a comparison of two approaches to hybrid head pose refinement, we show the superior performance of Quasi-Newton optimization for image space error minimization. Moreover, a 3D landmark refinement step is proposed, which significantly improves robustness of the overlay process. The enhanced system is demonstrated in an interactive AR environment, which provides accurate haptic feedback from real and virtual deformable objects. Finally, the effect of landmark occlusion on tracking stability during user interaction is also analyzed.

Publication: G. Bianchi, C. Jung, B. Knörlein, M. Harders and G. Székely, "High-fidelity visuo-haptic interaction with virtual objects in multi-modal AR systems", ISMAR 2006, October 2006.