Augmented Reality is increasingly being explored as the new medium for two-way remote collaboration applications to guide participants more effectively and efficiently through visual instructions. As users strive for more natural interaction and automation in augmented reality applications, new visual recognition techniques are needed to improve the user experience. Although simple object recognition in Augmented Reality is often used to achieve this goal, most collaborative tasks are too complex for such recognition algorithms to suffice.

In this project, we propose an active and fine-grained visual recognition approach for mobile Augmented Reality, which uses 2D video frames, 3D feature points and camera position data to detect different states of an object. We demonstrate the value of our approach through a mobile application designed for hardware support that automatically detects the state of an object to present the right set of information in the right context.

Appeared in ACM MobiCom'18: https://dl.acm.org/doi/10.1145/3241539.3267771
IBM tech blog: https://www.ibm.com/blogs/research/2020/11/ar-tech-support/

Please take the opportunity to connect and share this video with your friends and family if you find it helpful.