AI learning

ManiWAV is a research project aimed at learning robot control skills from audio and visual data in the wild. It collects synchronized audio and visual feedback of human demonstrations and learns robot control strategies directly from demonstrations through corresponding policy interfaces. The model demonstrates the capabilities of its system through four contact-rich manipulation tasks that require the robot to passively sense contact events and patterns, or actively sense the material and state of an object surface. Additionally, the system is able to generalize to unseen wild environments by learning from diverse human demonstrations in the wild.