3D modeling

SpatialLM is a large-scale language model designed for processing 3D point cloud data, capable of producing structured 3D scene understanding output, including semantic categories of architectural elements and objects. It is capable of processing point cloud data from a variety of sources including monocular video sequences, RGBD images, and LiDAR sensors without the need for specialized equipment. SpatialLM has important application value in autonomous navigation and complex 3D scene analysis tasks, significantly improving spatial reasoning capabilities.

This is a multi-modal language model framework developed by a Stanford University research team that aims to unify verbal and non-verbal language in 3D human movements. The model is capable of understanding and generating multimodal data including text, speech, and motion, which is critical for creating virtual characters that can communicate naturally and is widely used in games, movies, and virtual reality. Key advantages of this model include high flexibility, low training data requirements, and the ability to unlock new tasks such as editable gesture generation and predicting emotions from actions.

Genie 2 is a large-scale basic world model developed by Google DeepMind that can generate endless, actionable, and playable 3D environments based on a single cue image for training and evaluating embodied agents. Genie 2 represents a major advance in the field of deep learning and artificial intelligence. It demonstrates a variety of emergent capabilities in large-scale generative models by simulating virtual worlds and the consequences of their actions, such as object interaction, complex character animation, physics simulation, etc. Genie 2 research drives new creative workflows for prototyping interactive experiences and opens up new possibilities for future research on more general AI systems and agents.

WonderJourney is a modular scene generation framework that can generate a series of diverse and coherent 3D scenes through location information provided by users to form an imaginary "wonderful journey". Generate textual descriptions of scenes by leveraging language models, generate coherent 3D scenes using a text-driven point cloud generation process, and validate the generated scenes with large-scale verification models. WonderJourney showcases a variety of visuals, including scenes of various types and styles.