A high-performance inference system for large language models, designed for production environments.
-
Updated
Jun 12, 2024 - C++
A high-performance inference system for large language models, designed for production environments.
Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Unify Efficient Fine-Tuning of 100+ LLMs
Bot en telegram con IA de llama 2 and llama 3
Fluent CLI is an advanced command-line interface designed to interact seamlessly with multiple workflow systems like FlowiseAI, Langflow, Make, and Zapier. Tailored for developers and IT professionals, Fluent CLI facilitates robust automation, simplifies complex interactions, and enhances productivity through a powerful and command suite
Python scraper based on AI
LlamaIndex is a data framework for your LLM applications
Chat locally using leading open models built by the community, optimized and accelerated by NVIDIA's enterprise-ready inference runtime
Foundation model benchmarking tool. Run any model on Amazon SageMaker and benchmark for performance across instance type and serving stack options.
🤖 The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
V.I.S.O.R., my in-development assistant
On-device LLM Inference Powered by X-Bit Quantization
Add a description, image, and links to the llama3 topic page so that developers can more easily learn about it.
To associate your repository with the llama3 topic, visit your repo's landing page and select "manage topics."