AI Agents

Adept Fuyu-Heavy is a new multi-modal model designed specifically for digital agencies. It performs well in multimodal reasoning, particularly in UI understanding, while also performing well on traditional multimodal benchmarks. Furthermore, it demonstrates our ability to extend the Fuyu architecture and obtain all associated benefits, including processing images of arbitrary sizes/shapes and efficiently reusing existing transformer optimizations. It also has the ability to match or exceed the performance of models of the same computational level, albeit requiring some of the capacity to be devoted to image modeling.

LEO is a multi-modal, multi-task omnipotent agent based on a large language model, capable of sensing, localizing, reasoning, planning and executing tasks in the 3D world. LEO is implemented through two stages of training: (i) 3D visual language alignment and (ii) 3D visual language action command adjustment. We carefully curate and generate a large-scale dataset containing object-level and scene-level multi-modal tasks that require deep understanding and interaction with the 3D world. Through rigorous experiments, we demonstrate LEO's outstanding performance on a wide range of tasks including 3D subtitles, question answering, reasoning, navigation, and robot operation.

AndesGPT Andes large model is a personalized and exclusive large model and agent released by OPPO. It is based on the device-cloud collaboration architecture design and provides a variety of model specifications with different parameter sizes. It supports technical features such as dialogue enhancement, personalization and device-cloud collaboration. OPPO will lay out large-scale forward-looking technologies, cooperate with the Intelligent Computing Joint Laboratory established by the University of Science and Technology of China, and open source the agent framework to support the efficient incubation, hosting and application of agents.

Motif is a PyTorch-based project that trains AI agents on NetHack by deriving reward functions from the preferences of LLMs (Large Language Models). It can generate behaviors that are intuitively consistent with human behavior and can be guided by cue modifications.