Tag: LLM

MemU is an intelligent memory layer designed for AI companions that provides higher accuracy, faster retrieval speed and lower cost. It is an open source AI memory framework suitable for machine learning, neural networks, conversational AI, chatbot memory, AI agents and autonomous memory.

mcp-use is an open source MCP client library designed to help developers connect any large language model (LLM) to MCP tools and build custom agents with tool access without using closed source or application clients. The product provides an easy-to-use API and powerful functions that can be applied in multiple fields.

Basic Memory is a knowledge management system that builds persistent knowledge through natural dialogue with LLM and saves it in local Markdown files. It solves the problem that most LLM interactions are short and knowledge is difficult to retain. Its advantages include local priority, bidirectional reading and writing, simple structure, the ability to form knowledge graphs, compatibility with existing editors, and lightweight infrastructure. Positioned to help users build personal knowledge bases, it adopts the AGPL-3.0 license and has no clear price information.

OpenAI Agents SDK is a framework for building multi-agent workflows. It allows developers to create complex automated processes by configuring instructions, tools, security mechanisms, and handoffs between agents. The framework supports integration with any model that conforms to the OpenAI Chat Completions API format and is highly flexible and scalable. It is mainly used in programming scenarios to help developers quickly build and optimize agent-driven applications.

Awesome-LLM-Post-training is a resource library focused on large language model (LLM) post-training methods. It provides an in-depth look at post-LLM training, including tutorials, surveys, and guides. This resource library is based on the paper "LLM Post-Training: A Deep Dive into Reasoning Large Language Models" and aims to help researchers and developers better understand and apply LLM post-training technology. This resource library is free and open and suitable for academic research and industrial applications.

l1m is a powerful tool that leverages large language models (LLMs) through agents to extract structured data from unstructured text or images. The importance of this technology lies in its ability to convert complex information into an easy-to-process format, thereby increasing the efficiency and accuracy of data processing. The main advantages of l1m include no need for complex prompt engineering, support for multiple LLM models, and built-in caching functions. It was developed by Inferable Company to provide users with a simple, efficient and flexible data extraction solution. l1m offers a free trial and is suitable for businesses and developers who need to extract valuable information from large amounts of unstructured data.

LLMs.txt Generator is an online tool powered by Firecrawl designed to help users generate integrated text files for LLM training and inference from websites. It provides high-quality text data for training large language models by integrating web content, thereby improving model performance and accuracy. The main advantage of this tool is that it is simple and efficient to operate and can quickly generate the required text files. It is mainly aimed at developers and researchers who need large amounts of text data for model training, providing them with a convenient solution.

hugo-translator is an article translation tool driven by a large language model (LLM). It can automatically translate articles from one language to another and generate new Markdown files. This tool supports OpenAI and DeepSeek models, and users can quickly complete translation tasks through simple configurations and commands. It is mainly aimed at users who use the Hugo static website generator to help them quickly generate and manage multilingual content. The product is currently free and open source, aiming to improve the efficiency of content creators and lower the threshold for publishing multi-language content.

Aviator Agents is a programming tool focused on code migration. By integrating LLM technology, it can connect directly to GitHub and support multiple models, such as Open-AI o1, Claude Sonnet 3.5, Llama 3.1 and DeepSeek R1. This tool can automatically perform code migration tasks, including searching for code dependencies, optimizing code, generating PR, etc., greatly improving the efficiency and accuracy of code migration. It is mainly aimed at development teams to help them complete code migration efficiently and save time and energy.

llm-commit is a plug-in designed for LLM (Large Language Model), used to generate Git commit information. This plug-in automatically generates concise and meaningful submission information by analyzing differences in Git's staging area and using LLM's language generation capabilities. It not only improves developers' submission efficiency, but also ensures the quality and consistency of submitted information. This plug-in is suitable for any development environment using Git and LLM, is free and open source, and is easy to install and use.

Crawl4LLM is an open source web crawler project that aims to provide efficient data crawling solutions for the pre-training of large language models (LLM). It helps researchers and developers obtain high-quality training corpus by intelligently selecting and crawling web page data. The tool supports multiple document scoring methods and can flexibly adjust crawling strategies according to configuration to meet different pre-training needs. The project is developed based on Python, has good scalability and ease of use, and is suitable for use in academic research and industrial applications.

This product is an open source project developed by Vectara to evaluate the hallucination rate of large language models (LLM) when summarizing short documents. It uses Vectara’s Hughes Hallucination Evaluation Model (HHEM-2.1) to calculate rankings by detecting hallucinations in the model output. This tool is of great significance for the research and development of more reliable LLM, and can help developers understand and improve the accuracy of the model.

VisionAgent is a powerful tool that uses artificial intelligence and large language models (LLM) to generate code to help users quickly solve vision tasks. The main advantage of this tool is its ability to automatically convert complex visual tasks into executable code, greatly improving development efficiency. VisionAgent supports multiple LLM providers, and users can choose different models according to their needs. It is suitable for developers and enterprises who need to quickly develop visual applications and can help them implement powerful visual solutions in a short time. VisionAgent is currently free and aims to provide users with efficient and convenient visual task processing capabilities.

OmniParser V2 is an advanced artificial intelligence model developed by Microsoft's research team, designed to transform large language models (LLM) into intelligent agents capable of understanding and operating graphical user interfaces (GUIs). This technology enables LLM to more accurately identify interactable icons and perform predetermined actions on the screen by converting interface screenshots from pixel space into interpretable structural elements. OmniParser V2 has made significant improvements in detecting small icons and fast inference, achieving an average accuracy of 39.6% on the ScreenSpot Pro benchmark when combined with GPT-4o, far exceeding the original model's 0.8%. In addition, OmniParser V2 also provides the OmniTool tool, which supports use with a variety of LLMs, further promoting the development of GUI automation.

Supametas.AI is a platform focused on unstructured data processing, designed to help enterprises quickly convert data in multiple formats such as audio, video, pictures, text, etc. into structured data suitable for the LLM RAG knowledge base. By providing a variety of data collection methods and powerful pre-processing functions, the platform greatly simplifies the data processing process and lowers the threshold for enterprises to build industry data sets. Its ability to be seamlessly integrated into the LLM RAG knowledge base enables enterprises to more efficiently use data to drive business development. Supametas.AI is positioned to become the industry's leading LLM data structured processing and development platform to meet the needs of enterprises in terms of data privacy and flexibility.

This product is a full-stack application that uses LLM (Large Language Model) and LangChain technology, combined with LangGraph to achieve retrieval and analysis of stock data and news. It utilizes ChromaDB as a vector database, supports semantic search and data visualization, and provides users with in-depth insights into the stock market. This product is mainly aimed at investors, financial analysts and data scientists, helping them quickly obtain and analyze stock-related information and assist in decision-making. The product is currently open source and free, and is suitable for users who need to efficiently process financial data and news.

OpenDeepResearcher is an AI-based research tool that, by combining services such as SERPAPI, Jina, and OpenRouter, can automatically conduct multiple rounds of iterative searches based on query topics entered by users until sufficient information is collected and a final report is generated. The core advantage of this tool lies in its efficient asynchronous processing capabilities, deduplication function and powerful LLM decision support, which can significantly improve research efficiency. It is mainly aimed at scientific researchers, students and professionals in related fields who need to conduct large-scale literature searches and information sorting, helping them quickly obtain high-quality research materials. The tool is currently available as open source and users can deploy and use it as needed.

DocETL is a powerful system for processing and analyzing large amounts of text data. By leveraging the power of large language models (LLM), it can automatically optimize data processing processes and seamlessly integrate LLM and non-LLM operations. The main advantages of the system include its declarative YAML definition, which allows users to easily define complex data processing processes. In addition, DocETL also provides an interactive playground to facilitate users to experiment with prompt projects. Product background information shows that DocETL launched DocWrangler in December 2024, a new interactive playground designed to simplify prompt engineering. Price-wise, although not clearly marked, judging from the use cases provided, the cost of running and optimizing data processing processes is relatively low. Product positioning is mainly to provide services for users who need to process large amounts of text data and extract valuable information from it.

DocWrangler is an open source interactive development environment designed to simplify the process of building and optimizing large language model (LLM)-based data processing pipelines. It provides instant feedback, visual exploration tools, and AI-assisted features to help users more easily explore data, experiment with different operations, and optimize pipelines based on findings. This product is built based on the DocETL framework and is suitable for processing unstructured data, such as text analysis, information extraction, etc. It not only lowers the threshold for LLM data processing, but also improves work efficiency, allowing users to more effectively utilize the powerful functions of LLM.

mlabonne/llm-datasets is a collection of high-quality datasets and tools focused on fine-tuning large language models (LLMs). The product provides researchers and developers with a range of carefully selected and optimized datasets to help them better train and optimize their language models. Its main advantage lies in the diversity and high quality of the data set, which can cover a variety of usage scenarios, thus improving the generalization ability and accuracy of the model. In addition, the product provides tools and concepts to help users better understand and use these data sets. Background information includes being created and maintained by mlabonne to advance the field of LLM.

FlashInfer is a high-performance GPU kernel library designed for serving large language models (LLM). It significantly improves the performance of LLM in inference and deployment by providing efficient sparse/dense attention mechanism, load balancing scheduling, memory efficiency optimization and other functions. FlashInfer supports PyTorch, TVM and C++ API, making it easy to integrate into existing projects. Its main advantages include efficient kernel implementation, flexible customization capabilities and broad compatibility. The development background of FlashInfer is to meet the growing needs of LLM applications and provide more efficient and reliable inference support.

llmstxt-generator is a tool for generating website content integration text files required for LLM (Large Language Model) training and inference. It crawls website content and merges it into a text file, supporting the generation of standard llms.txt and complete llms-full.txt versions. This tool is powered by firecrawl_dev for web crawling and uses GPT-4-mini for text processing. Its main advantages include the ability to use basic functions without the need for an API key, while providing a web interface and API access for users to quickly generate the required text files.

CodebaseToPrompt is a simple tool that converts local directories into structured prompts for large language models (LLM). It helps users select files that need to be included or ignored, and then outputs them in a format that can be copied directly into LLM, suitable for code review, analysis, or documentation generation. The main advantages of this tool are that it is highly interactive, easy to operate, and can be used directly in the browser without uploading any files, ensuring data security and privacy. Product background information shows that it was developed by the path-find-er team and aims to improve the efficiency of developers when using LLM for code-related tasks.

Document Inlining is a composite AI system launched by Fireworks AI that can convert any large language model (LLM) into a visual model to process images or PDF documents. This technology enables logical reasoning by building an automated process to convert any digital asset format into an LLM-compatible format. Document Inlining provides higher quality, input flexibility and ultra-simple usage by parsing images and PDFs and inputting them directly into the LLM of the user's choice. It solves the limitations of traditional LLM in processing non-text data, improves the quality of text model inference through specialized component decomposition tasks, and simplifies the developer experience.

IdentityRAG is a tool for building LLM chatbots based on customer data, capable of retrieving unified customer data from multiple internal source systems such as databases and CRMs. The product handles misspellings and inaccurate information through real-time fuzzy search, delivering accurate, relevant and consistent responses to customer data. It supports rapid retrieval of structured customer data, construction of dynamic customer profiles, and real-time updates of customer data, enabling LLM applications to access unified and accurate customer data. IdentityRAG is trusted by fast-growing, data-driven enterprises for its fast response times, real-time data updates, and easy scalability.

PromptWizard is a task-aware prompt optimization framework developed by Microsoft. It uses a self-evolution mechanism to enable large language models (LLM) to generate, criticize and improve their own prompts and examples, and continuously improve through iterative feedback and synthesis. This adaptive approach is fully optimized by evolving instructions and contextually learning examples to improve task performance. The three key components of the framework include: feedback-driven optimization, critique and synthesis of diverse examples, and self-generated Chain of Thought (CoT) steps. The importance of PromptWizard is that it can significantly improve the performance of LLM on specific tasks, enhancing the performance and interpretability of the model by optimizing prompts and examples.

ElevenLabs Conversational AI is a voice agent product that can be quickly deployed on the web, mobile device or phone. It features low latency, full configurability and seamless scalability, supports turn-taking and interruption processing in natural conversations, and is suitable for unpredictable conversations in noisy environments. The product combines speech-to-text, large language model (LLM) and text-to-speech technology, supports multiple languages ​​and custom voices, and is suitable for various scenarios such as customer support, scheduling, and outbound sales.

SiteSpeakAI - llms.txt Generator is an online tool for generating llms.txt files. This file provides large language models (LLMs) with the necessary information so that they can more effectively use your website when inferring. The importance of this tool lies in its ability to help webmasters and developers optimize their websites to make them more suitable for interaction with artificial intelligence language models, improving website functionality and user experience. SiteSpeakAI provides a free online generator that allows users to quickly generate the required llms.txt file without complex programming knowledge.

aisuite is a product that provides a simple, unified interface to access multiple generative AI services. It allows developers to use multiple large language models (LLM) through a standardized interface and compare the results. As a lightweight Python client library wrapper, aisuite allows authors to seamlessly switch and test responses from different LLM providers without changing their code. Currently, the library focuses on chat completion functionality, with plans to expand to more use cases in the future.

Model Context Protocol (MCP) is an open protocol that allows seamless integration between large language model (LLM) applications and external data sources and tools. Whether building an AI-powered integrated development environment (IDE), enhancing chat interfaces, or creating custom AI workflows, MCP provides a standardized way to connect LLMs with the context they require. The main advantages of MCP include standardized connection methods, easy integration and expansion, and strong community support. Product background information shows that MCP aims to promote developers to build more intelligent and efficient applications, especially in the fields of AI and machine learning. MCP is currently free for developers to use.

ollama-ebook-summary is a project that leverages large language models (LLM) to create bullet point summary summaries for long texts. This project is particularly suitable for books in epub and pdf formats, and is able to automatically extract chapters and split them into small chunks of about 2000 tokens to increase the granularity of the response. The context for the product was that the creators wanted to quickly summarize a series of books to integrate psychological theory and practice and build a coherent argument based on this information. Key benefits of this tool include improved content combing efficiency, support for custom question queries, and the generation of detailed summaries of each text section.

HumanLayer is an API and SDK that allows AI agents to contact humans for feedback, input and approval. It ensures human oversight of high-risk function calls via approval workflows on channels such as Slack, email, and more, enabling secure connection of your choice of LLMs and frameworks with AI agents to the world. HumanLayer is supported by Y Combinator and is compatible with a variety of popular frameworks and LLMs, including OpenAI, Claude, Llama3.1, and more. It provides a platform to enhance the capabilities of AI agents and improve their reliability and efficiency through human-in-the-loop methods. HumanLayer's pricing strategy includes free, paid and customized enterprise plans to meet the needs of different users.

Mistral Moderation API is a content moderation service launched by Mistral AI, designed to help users detect and filter unwelcome text content. The API is the same technology used in the auditing service in Le Chat and is now open so that users can customize and use the tool according to specific applications and security standards. The model is an LLM (Large Language Model) based classifier capable of classifying text input into 9 predefined categories. Mistral AI's API is natively multilingual and specifically trained for Arabic, Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian and Spanish. The main advantages of this API include improving the scalability and robustness of auditing, as well as providing detailed policy definitions and startup guidance through technical documentation to help users effectively implement system-level security protection.

askrepo is a source code reading tool based on LLM (Large Language Model). It can read the content of text files managed by Git, send it to the Google Gemini API, and provide answers to questions based on specified prompts. This product represents the application of natural language processing and machine learning technology in the field of code analysis. Its main advantages include the ability to understand and interpret code, helping developers quickly understand new projects or complex code bases. Product background information shows that askrepo is suitable for scenarios that require an in-depth understanding of the code, especially during the code review and maintenance stages. The product is open source and free to use.

Chat Nio is a leading one-stop enterprise solution for LLM (Large Language Model) in China. It provides powerful AI integration tools and supports 35+ mainstream AI models, covering fields such as text generation, image creation, audio processing and video editing, and supports privatized deployment and transfer services. It provides customized AI solutions for developers, individual users and enterprises, including but not limited to multi-tenant token distribution, billing management system, deep integration of Midjourney Proxy Plus painting function, comprehensive call logging system, etc. With its versatility, flexibility and ease of use, Chat Nio meets the diverse needs of enterprises and teams, helping them develop and deploy AI applications efficiently.

Laminar is an open source, full-stack platform focused on AI engineering from first principles. It helps users collect, understand and use data to improve the quality of large language model (LLM) applications. Laminar supports tracking of text and image models, and will soon support audio models. Key benefits of the product include zero-overhead observability, online evaluation, dataset construction and LLM chain management. Laminar is completely open source and easy to self-host, suitable for developers and teams who need to build and manage LLM products.

Dabarqus is a Retrieval Augmented Generation (RAG) framework that allows users to feed private data to large language models (LLMs) in real time. This tool enables users to easily store various data sources (such as PDFs, emails, and raw data) into semantic indexes, called "memories", by providing REST APIs, SDKs, and CLI tools. Dabarqus supports LLM-style prompts, enabling users to interact with memories in a simple way without having to build special queries or learn a new query language. In addition, Dabarqus also supports the creation and use of multi-semantic indexes (memories) so that data can be organized according to topics, categories, or other groupings. Dabarqus’ product background information shows that it aims to simplify the integration process of private data and AI language models and improve the efficiency and accuracy of data retrieval.

GitHub to LLM Converter is an online tool designed to help users convert project, file or folder links on GitHub into a format suitable for Large Language Model (LLM) processing. This tool is critical for developers and researchers who need to work with large amounts of code or document data, because it simplifies the data preparation process so that the data can be used more efficiently for machine learning or natural language processing tasks. This tool was developed by Skirano and provides a simple user interface. Users only need to enter the GitHub link to convert with one click, greatly improving work efficiency.

Meta Lingua is a lightweight, efficient large language model (LLM) training and inference library designed for research. It uses easily modifiable PyTorch components, allowing researchers to experiment with new architectures, loss functions, and data sets. The library is designed to enable end-to-end training, inference, and evaluation, and provide tools to better understand model speed and stability. Although Meta Lingua is still under development, several sample applications are provided to demonstrate how to use this code base.

Prompt Engineering is a cutting-edge technology in the field of artificial intelligence that changes the way we interact with AI technology. This open source project aims to provide a platform for beginners and experienced practitioners to learn, build and share Prompt Engineering technology. The project contains a variety of examples from basic to advanced and is designed to promote learning, experimentation and innovation in the field of Prompt Engineering. In addition, it also encourages community members to share their innovative technologies and jointly promote the development of Prompt Engineering technology.

Llama-3.1-Nemotron-70B-Instruct is a large language model customized by NVIDIA, focusing on improving the helpfulness of answers generated by large language models (LLM). The model performs well on multiple automatic alignment benchmarks, such as Arena Hard, AlpacaEval 2 LC, and GPT-4-Turbo MT-Bench. It is trained on the Llama-3.1-70B-Instruct model by using RLHF (specifically the REINFORCE algorithm), Llama-3.1-Nemotron-70B-Reward and HelpSteer2-Preference hints. This model not only demonstrates NVIDIA's technology in improving the helpfulness of common domain instruction compliance, but also provides a model transformation format that is compatible with the HuggingFace Transformers code library and can be used for free managed inference through NVIDIA's build platform.

LlamaIndex.TS is a framework designed for building large language model (LLM)-based applications. It focuses on helping users ingest, structure and access private or domain-specific data. This framework provides a natural language interface for connecting humans and inferred data, allowing developers to enhance their software capabilities through LLM without becoming experts in machine learning or natural language processing. LlamaIndex.TS supports popular runtime environments such as Node.js, Vercel Edge Functions and Deno.

ComfyUI LLM Party aims to develop a complete set of LLM workflow nodes based on the ComfyUI front-end, allowing users to quickly and easily build their own LLM workflows and easily integrate them into existing image workflows.

promptic is a lightweight, decorator-based Python library that simplifies the process of interacting with large language models (LLMs) through litellm. Using prompt, you can easily create prompts, handle input parameters, and receive structured output from LLMs with just a few lines of code.

Epsilla is a no-coding RAG-as-a-Service platform that allows users to build production-ready Large Language Model (LLM) applications based on private or public data. The platform provides a one-stop service, including data management, RAG tools, CI/CD-style assessments, and enterprise-grade security measures, designed to reduce total cost of ownership (TCO), increase query speed and throughput, while ensuring the timeliness and security of information.

MemoryScope is a framework that provides long-term memory capabilities for large language model (LLM) chatbots. It enables chatbots to store and retrieve memory fragments through memory databases and work libraries, thereby achieving personalized user interaction experience. Through operations such as memory retrieval and memory integration, this product enables the robot to understand and remember the user's habits and preferences, providing users with a more personalized and coherent conversation experience. MemoryScope supports multiple model APIs, including openai and dashscope, and can be used in conjunction with existing agent frameworks such as AutoGen and AgentScope, providing rich customization and scalability.

Weavel is an AI prompt engineer that helps users optimize the application of large language models (LLM) through functions such as tracking, data set management, batch testing and evaluation. Used in conjunction with the Weavel SDK, Weavel automatically records and adds LLM-generated data to your dataset, enabling seamless integration and continuous improvement for specific use cases. In addition, Weavel can automatically generate evaluation codes and use LLM as an impartial referee for complex tasks, simplifying the evaluation process and ensuring accurate and detailed performance indicators.

RAG_Techniques is a collection of technologies focused on Retrieval-Augmented Generation (RAG) systems, aiming to improve the accuracy, efficiency, and context richness of the system. It provides a hub for cutting-edge technology, driving the development and innovation of RAG technology through community contributions and a collaborative environment.

GitHub Models is a new generation AI model service launched by GitHub, aiming to help developers become AI engineers. It integrates industry-leading large and small language models directly into the GitHub platform, allowing more than 100 million users to access and use these models directly on GitHub. GitHub Models provides an interactive model playground where users can test different prompts and model parameters without paying a fee. In addition, GitHub Models integrates with Codespaces and VS Code, allowing developers to seamlessly use these models in the development environment and achieve production deployment through Azure AI, providing enterprise-level security and data privacy protection.

RouteLLM is a framework for serving and evaluating large language model (LLM) routers. It intelligently routes queries to models of varying cost and performance to save costs while maintaining response quality. It delivers router functionality out of the box and has shown up to 85% cost reduction and 95% GPT-4 performance on widely used benchmarks.

vLLM is a fast, easy-to-use, and efficient library for inferring and serving large language models (LLM). It provides high-performance inference services by using the latest service throughput technology, efficient memory management, continuous batch processing of requests, CUDA/HIP graph fast model execution, quantization technology, optimized CUDA kernels, etc. vLLM supports seamless integration with the popular HuggingFace model, supports multiple decoding algorithms, including parallel sampling, beam search, etc., supports tensor parallelism, is suitable for distributed reasoning, supports streaming output, and is compatible with the OpenAI API server. Additionally, vLLM supports NVIDIA and AMD GPUs, as well as experimental prefix caching and multi-LORA support.

RAGElo is a toolset that uses the Elo scoring system to help select the best retrieval-augmented generation (RAG)-based large language model (LLM) agents. As generative LLM becomes easier to prototype and integrate in production, evaluation remains the most challenging part of the solution. RAGElo provides a good overview of which settings work and which don't by comparing answers to multiple questions from different RAG pipelines and prompts, calculating rankings for different settings.

Nerve is an LLM tool that can create stateful agents, allowing users to define and perform complex tasks without writing code. It enables agents to plan and step through the actions required to complete a task by dynamically updating system prompts and maintaining state across multiple inference processes. Nerve supports any model accessible through ollama, groq or the OpenAI API, with a high degree of flexibility and efficiency while focusing on memory safety.

Unify AI is a platform designed for developers that allows users to access and compare large language models (LLMs) from different providers through a unified API. The platform provides real-time performance benchmarking to help users select and optimize the most suitable models based on quality, speed and cost efficiency. Unify AI also provides custom routing capabilities, allowing users to set constraints on cost, latency, and output speed based on their needs, and define custom quality metrics. In addition, Unify AI's system updates every 10 minutes based on the latest benchmark data, sending queries to the fastest providers to ensure continuous peak performance.

Siri-Ultra is a cloud-based intelligent assistant that runs on Cloudflare Workers and works with any large language model (LLM). It utilizes the LLaMA 3 model and obtains weather data and online searches through custom function calls. This project allows users to use Siri through Apple Shortcuts, eliminating the need for dedicated hardware devices.

ScrapeGraphAI is a Python web crawling library that uses LLM (Large Language Model) and direct graph logic to create crawling pipelines for websites, documents, and XML files. Users only need to specify the information they want to extract, and the library will do the job automatically. The main advantage of this library is that it simplifies the process of network data crawling and improves the efficiency and accuracy of data extraction. It is suitable for data exploration and research purposes, but should not be misused.

Reka Core is a GPT-4 level multi-modal large language model (LLM) with powerful context understanding capabilities for images, videos and audios. It is one of only two commercially available comprehensive multimodal solutions currently on the market. Core excels in multi-modal understanding, reasoning capabilities, coding and agent workflows, multi-language support, and deployment flexibility.

Tools4AI is a Large Action Model (LAM) implemented 100% in Java and serves as an LLM agent for enterprise Java applications. This project demonstrates how to integrate AI with enterprise tools or external tools to convert natural language prompts into actionable actions. These prompts may be called "action prompts" or "executable prompts". By leveraging AI capabilities, it simplifies user interaction with complex systems, improving productivity and innovation.

Langtail is a platform designed to simplify large language model (LLM) hint management. With Langtail, you can enhance team collaboration, improve efficiency, and get a deeper understanding of how your AI works. Try Langtail to build LLM applications in a more collaborative and insightful way.

karpathy/llm.c is a project that uses simple C/CUDA to implement LLM training. It aims to provide a clean, simple reference implementation, but also includes a more optimized version that can approach the performance of PyTorch, but with significantly less code and dependencies. Currently under development are direct CUDA implementations, optimized CPU versions using SIMD instructions, and support for more modern architectures such as Llama2, Gemma, etc.

Tara is a plug-in that can integrate large language models (LLM) into Comfy UI, supports simple API settings, and integrates LLaVa models. It includes the TaraPrompter node to generate accurate results, the TaraApiKeyLoader node to manage API keys, the TaraApiKeySaver node to securely store keys, and the TaraDaisyChainNode node to serially output to implement complex workflows.

Building UI components is often a tedious job. OpenUI aims to make this process fun, fast and flexible. This is also what we at W&B use to test and prototype the next generation of tools for building powerful applications on top of LLM. You can use your imagination to describe the UI and then see the rendering in real time. You can request changes and convert HTML to React, Svelte, web components and more. It's like an open source and less polished version of V0.

OPT2I is a T2I optimization framework that utilizes large language models (LLM) to improve cue-image consistency. Optimize the generation process by iteratively generating revised prompts. It can significantly improve the consistency score while maintaining FID and increasing the recall rate of generated data and real data.

Apple released its own large language model MM1, which is a multi-modal LLM with a maximum scale of 30B. Through pre-training and SFT, the MM1 model achieved SOTA performance in multiple benchmark tests, demonstrating attractive features such as in-context prediction, multi-image reasoning, and few-shot learning capabilities.

ELLA (Efficient Large Language Model Adapter) is a lightweight method to equip existing CLIP-based diffusion models with powerful LLM. ELLA improves the model's prompt following capabilities, enabling text-to-image models to understand long texts. We design a time-aware semantic connector to extract time-step related conditions for various denoising stages from pre-trained LLM. Our TSC dynamically adapts to the semantic features at different sampling time steps, helping to freeze U-Net at different semantic levels. ELLA performs well in benchmarks such as DPG-Bench, especially in dense prompts involving multiple object combinations, different attributes and relationships.

LongRoPE is a technology introduced by Microsoft that can expand the context window of a pre-trained large language model (LLM) to 2048k (2 million) tokens, achieving expansion from short context to long context, reducing training cost and time, while maintaining the original short context window performance. It is suitable for improving the understanding and generation capabilities of language models on long texts, and improving tasks such as machine reading comprehension, text summarization and long article generation.

LangSmith is a unified DevOps platform for developing, collaborating, testing, deploying and monitoring LLM applications. It supports all stages of the LLM application development life cycle and provides end-to-end solutions for building LLM applications. The main functions include: link tracking, prompt tools, data sets, automatic evaluation, online deployment, etc. Suitable for developers building LLM-based AI assistants and ChatGPT applications.

Fennel Bean is a domain-specific knowledge assistant based on LLM that can handle complex scenarios in group chats and answer user questions without causing a flood of messages. It provides an algorithm pipeline to answer technical questions and has low deployment cost. It only needs the LLM model to meet four characteristics to answer most user questions. Fennel Bean can handle various problems in running scenarios, and you are welcome to join their WeChat group to experience the latest version.

React Flow is an open source visual editor that allows users to create agent workgroups through drag-and-drop for customizing business logic. Users can drag and drop agents from the gallery into the workspace, connect them, define initial tasks, and export Python scripts to run on the local machine. We provide cloud support for enterprises through customized operating systems so that they can run LLM. Feel free to contact our Enterprise Support team for more information.

Athina AI is a tool for monitoring and debugging LLM (Large Language Model) models. It can help you discover and fix hallucinations and errors of LLM models in production environments, and provide detailed analysis and improvement suggestions. Athina AI supports multiple LLM models and can configure customized evaluations to meet different usage scenarios. You can use Athina AI to detect incorrect output, analyze cost and accuracy, debug model output, explore conversation content, and compare the performance of different models.

LLM Context Extender is a tool designed to extend the context window of large language models (LLMs). It helps LLMs effectively adapt to larger context windows by adjusting the base frequency of RoPE and scaling attention logits. The tool validates the superiority of its approach in terms of fine-tuning performance and robustness, and demonstrates extraordinary efficiency in extending the context window of LLaMA-2-7B-Chat to 16,384 with only 100 samples and 6 training steps. Furthermore, we explore how data composition and training sessions affect context window expansion for specific downstream tasks, suggesting fine-tuning of LLMs with long conversations as a good starting point.

Portkey's AI Gateway is the interface between applications and managed LLM. It uses a unified API to optimize API requests for OpenAI, Anthropic, Mistral, LLama2, Anyscale, Google Gemini, etc., enabling smooth routing. The gateway is fast, lightweight, has a built-in retry mechanism, and supports multi-model load balancing to ensure application reliability and performance.

Kindllm is a distraction-free LLM chat web app optimized for Kindle and is your perfect reading companion. Powered by Mixtral by Mistral AI. Mainly tested on Kindle Paperwhite. Why? The author tried to make this app before, but it didn't work well on older versions of the Kindle browser. Surprisingly, Amazon recently updated some of the Kindle's web browsers, and it now seems to be good enough to run simple interactive applications like this!

This is an efficient LLM inference solution implemented on Intel GPUs. By simplifying the LLM decoder layer, using a segmented KV caching strategy and a custom Scaled-Dot-Product-Attention kernel, the solution achieves up to 7x lower token latency and 27x higher throughput on Intel GPUs compared to the standard HuggingFace implementation. Please refer to the official website for detailed functions, advantages, pricing and positioning information.

Confident AI is an open source evaluation infrastructure that provides confidence in LLM (Language Model). Users can evaluate their LLM applications by writing and executing test cases and measure their performance using a rich set of open source metrics. By defining expected output and comparing it to actual output, users can determine whether the LLM is performing as expected and identify areas for improvement. Confident AI also provides advanced difference tracking capabilities to help users optimize LLM configurations. In addition, users can take advantage of comprehensive analytics to identify focus use cases and move LLM into production with confidence. Confident AI also provides powerful capabilities to help users confidently take LLM into production, including A/B testing, evaluation, output classification, reporting dashboards, dataset generation and detailed monitoring.

LobeChat is an open source scalable high-performance chatbot framework that supports one-click free deployment of private ChatGPT/LLM network applications. With functions such as custom models, multi-language support, Plugins system, and knowledge extraction, it can help users quickly build private, safe and controllable AI assistants and knowledge management tools.

Essential AI has developed full-stack AI products to significantly improve enterprise work efficiency by automating boring workflows. For example, their technology can make data analysts 10 times more productive and provide business users with the tools to become independent data-driven decision-makers themselves. It can also identify the biggest risks in an organization’s supply chain and recommend improvements. With artificial feedback and technological breakthroughs, Essential AI's LLM will empower users to solve increasingly difficult tasks, unlock key skills, and expand the organization's impact on society.

Algomax simplifies the evaluation of LLM and RAG models, optimizes tip development, and provides unique insights into qualitative metrics through intuitive dashboards. Our evaluation engine accurately evaluates LLMs and ensures reliability through extensive testing. The platform provides comprehensive qualitative and quantitative metrics to help you better understand the behavior of your model and provide specific suggestions for improvement. Algomax has a wide range of uses and is suitable for various industries and fields.

Agent M is a powerful large language model or ChatGPT driven master agent development framework that allows you to create multiple LLM based agents. Agent Mbetween orchestrates between multiple agents that perform various tasks, such as natural language-based API calls, connect to your data and help automate complex conversations.

Teammate Lang is a comprehensive LLM App development and operation solution. Provides codeless editor, semantic caching, Prompt version management, LLM data platform, A/B testing, QA, playground and more than 20 models, including GPT, PaLM, Llama, Cohere, etc.

Vellum is a development platform for building LLM-driven applications. It has tools such as prompt engineering, semantic search, version control, testing and monitoring, which can help developers introduce LLM functions into the production environment. It is compatible with all major LLM providers, and developers can choose the most suitable model or switch at any time to avoid relying too much on a single LLM provider.

llamafile is a tool that packages an LLM (Large Language Model) model and its weights into a self-contained executable file. It combines llama.cpp and Cosmopolitan Libc, which allows complex LLM models to be compressed into an llamafile, which can be run locally on most computers without any installation and configuration. The main advantage is to make open source LLM models more accessible to developers and end users.

Bind is a collaborative Generative AI application development platform that helps developers quickly build and deploy powerful language model applications. Provides a wealth of tools and functions, including prompt scenarios for real-time testing and debugging of LLM responses, easy deployment of LLM assistants and other platforms that can be applied to production environments.

Hippocratic AI is the healthcare industry’s first safety-oriented LLM. It uses state-of-the-art technology to exceed the performance of GPT 4 on 105 medical exams and certifications. It comes with great features and benefits and provides details like pricing and positioning.

Vellum is a development platform for building LLM applications. It provides tools for rapid engineering, semantic search, version control, testing and monitoring, and is compatible with all major LLM providers. Vellum can help you bring LLM capabilities into production, enabling rapid development and deployment of LLM models while providing features such as quality testing and performance monitoring. Please refer to the official website for pricing and positioning.

PlugBear connects your LLM application to tools like Slack in seconds without writing any code. It supports mainstream LLM frameworks, allowing you to develop an LLM application once and connect to various popular collaboration and communication tools.

LLM Spark is a development platform that can be used to build LLM-based applications. It provides rapid testing of multiple LLMs, version control, observability, collaboration, multiple LLM support and other functions. LLM Spark makes it easy to build smart applications such as AI chatbots and virtual assistants, and achieves superior performance by integrating with provider keys. It also provides GPT-driven templates to accelerate the creation of various AI applications while supporting customized projects from scratch. LLM Spark also supports seamless uploading of datasets to enhance the functionality of AI applications. Compare GPT results, iterate and deploy smart AI applications with LLM Spark's comprehensive logging and analytics. It also supports simultaneous testing of multiple models, saving prompt versions and history, easy collaboration, and powerful search capabilities based on meaning rather than just keywords. In addition, LLM Spark also supports the integration of external data sets into LLM and complies with GDPR compliance requirements to ensure data security and privacy protection.

Lookahead Decoding is a new reasoning method used to break the sequential dependence of LLM reasoning and improve reasoning efficiency. Users can use Lookahead Decoding to improve their code by importing the Lookahead Decoding library. Lookahead Decoding currently only supports two models: LLaMA and Greedy Search.

Klu is an all-in-one LLM application platform that can quickly build, evaluate and optimize applications based on LLM technology on Klu. It provides a wide selection of state-of-the-art LLM models, allowing users to select and adjust according to their needs. Klu also supports team collaboration, version management, data evaluation and other functions, providing a comprehensive and convenient development platform for AI teams.

Pulze.ai is a one-stop LLM development automation platform that provides a single API to plug all the best LLM into your products and streamline your LLM feature development in minutes. Pulze.ai's API follows LLMOps best practices and makes it easy for your team to use. Pulze.ai allows you to test all your best models at once to speed up development. You can dynamically control budget and cost targets within Pulze.ai and protect your profits as you scale. Pulze.ai also provides enterprise-grade security to manage data privacy and security of all user data. Pulze.ai provides multiple function points, such as uploading data sources, optimizing results, one-click deployment, real-time tracking and version control, etc.

ArguflowChat is a retrieval-enhanced LLM chatbot that can hold conversations with Garry Tan. It has the following features and benefits: provide customized solutions, chat with Garry Tan, contact via email.

Freeplay is an LLM prototype building tool that can help product teams prototype, test and optimize functions faster. It empowers teams to use LLM to build faster.

Humanloop is a collaborative platform for building and monitoring production-grade applications based on large language models. It provides a complete set of tools that can help developers develop AI from prototype to production environment more quickly while ensuring system reliability. The main functions include: prompt engineering, which can iterate and version prompts to improve hit rates; model management, which supports various models and tracks them; content evaluation, which collects feedback and performs quantitative analysis; and a cooperation platform, which allows non-technical personnel to participate in AI application development. Typical application scenarios include building chatbots, automating customer support, and generating marketing content. Humanloop has been favored by thousands of developers and used by many well-known companies.

Langroid is a lightweight, extensible and principled Python framework that makes it easy to build LLM-based applications. You can set up agents, equip them with optional components (LLMs, vector stores, and methods), assign them tasks, and let them collaborate to solve problems by exchanging messages. This multi-agent paradigm is inspired by the Actor framework (but you don't need to know anything about this!). Langroid offers a completely new way of developing LLM applications that is thoughtfully thought out in simplifying the developer experience; it does not use Langchain. We welcome contributions - see the contributions documentation for contribution ideas.

Prompt Joy is a tool to help understand and debug LLM (Large Language Model) prompts. Key features include logging and split testing. Logging can record LLM requests and responses to facilitate checking the output results. Split testing makes it easy to A/B test and find the tips that perform best. It is decoupled from specific LLMs and can be used with LLMs such as OpenAI and Anthropic. It provides APIs for logging and split testing. Built with Node.js+PostgreSQL.

LLime is an enterprise intelligent work assistant based on a large language model. It can provide customized AI assistants for various departments of the enterprise to improve work efficiency. It provides a simple and easy-to-use interface, supports model fine-tuning based on enterprise data, and ensures that the model accurately adapts to enterprise needs. The main functions include code exploration, data analysis, content strategy, etc., which can help developers, managers and marketers make work decisions. The product is subscription-based and priced based on department and number of employees.

LangChain is a library that helps developers build applications that combine large language models (LLMs) with other computational or knowledge sources through compositionality. It provides end-to-end examples of various application scenarios, including question answering, chatbots, agents, and more. LangChain also provides functions such as general interfaces to LLMs, chain calls, data enhancement generation, memory and evaluation. Please visit the official website for pricing information.

GPT4All is a one-stop AI tool that provides more than 300 AI expert conditions and more than 500 finely tuned models, which can be used for a variety of tasks such as writing, coding, data organization, image generation, music generation, etc. It has an easy-to-use user interface, supports light and dark modes, integrates GitHub repositories, supports personalization of different predefined welcome messages, supports generating like and dislike ratings for answers, supports copying, editing and deleting messages, supports local database storage of discussions, supports searching, exporting and deleting multiple discussions, supports image/video generation based on stable diffusion, supports musicgen based music generation, supports multi-generation peer-to-peer network generation through Lollms nodes and petals, supports Docker, conda and manual virtual environment setup.

AutoAgents is an open source LLM-based automatic agent generation experimental application. This program is driven by LLM and can automatically generate multi-role agents based on the goals you set. It can determine the expert roles that need to be added and the specific execution plan based on the problem. Contains agent generator, execution plan generator, result reflection module, etc. LLM is made like a human being and can autonomously assign different roles according to the problem, formulate and execute a plan to solve the problem.

LM Studio is an easy-to-use desktop application for experimenting and running local and open source Large Language Models (LLMs) locally. LM Studio cross-platform desktop application allows you to download and run any ggml-compatible model from Hugging Face and provides a simple yet powerful model configuration and inference interface. The application utilizes your GPU when available.