ManiWAV

ManiWAV is a research project aimed at learning robot control skills from audio and visual data in the wild. It collects synchronized audio and visual feedback of human demonstrations and learns robot control strategies directly from demonstrations through corresponding policy interfaces. The model demonstrates the capabilities of its system through four contact-rich manipulation tasks that require the robot to passively sense contact events and patterns, or actively sense the material and state of an object surface. Additionally, the system is able to generalize to unseen wild environments by learning from diverse human demonstrations in the wild.

The target audience of ManiWAV is mainly researchers and developers in the fields of robotics and artificial intelligence. They can use ManiWAV to improve the robot's control skills in complex environments, especially when visual information is unclear or incomplete, and to enhance the robot's perception capabilities through audio information.