2023-04-19 17:03:12 +00:00
< h1 align = "center" >
< br >
2023-07-31 22:31:40 +00:00
< img height = "300" src = "https://github.com/go-skynet/LocalAI/assets/2420543/0966aa2a-166e-4f99-a3e5-6c915fc997dd" > < br >
2023-04-19 17:03:12 +00:00
LocalAI
< br >
< / h1 >
2023-03-20 20:30:55 +00:00
2023-08-06 22:01:01 +00:00
< p align = "center" >
< a href = "https://github.com/go-skynet/LocalAI/fork" target = "blank" >
< img src = "https://img.shields.io/github/forks/go-skynet/LocalAI?style=for-the-badge" alt = "LocalAI forks" / >
< / a >
< a href = "https://github.com/go-skynet/LocalAI/stargazers" target = "blank" >
< img src = "https://img.shields.io/github/stars/go-skynet/LocalAI?style=for-the-badge" alt = "LocalAI stars" / >
< / a >
< a href = "https://github.com/go-skynet/LocalAI/pulls" target = "blank" >
< img src = "https://img.shields.io/github/issues-pr/go-skynet/LocalAI?style=for-the-badge" alt = "LocalAI pull-requests" / >
< / a >
< a href = 'https://github.com/go-skynet/LocalAI/releases' >
< img src = 'https://img.shields.io/github/release/go-skynet/LocalAI?&label=Latest&style=for-the-badge' >
< / a >
< / p >
2023-04-24 16:10:58 +00:00
2023-08-05 22:04:06 +00:00
> :bulb: Get help - [❓FAQ](https://localai.io/faq/) [💭Discussions](https://github.com/go-skynet/LocalAI/discussions) [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) [:book: Documentation website](https://localai.io/)
2023-08-06 22:01:01 +00:00
>
> [💻 Quickstart](https://localai.io/basics/getting_started/) [📣 News](https://localai.io/basics/news/) [ 🛫 Examples ](https://github.com/go-skynet/LocalAI/tree/master/examples/) [ 🖼️ Models ](https://localai.io/models/)
[](https://github.com/go-skynet/LocalAI/actions/workflows/test.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/release.yaml)[](https://github.com/go-skynet/LocalAI/actions/workflows/image.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/bump_deps.yaml)[](https://artifacthub.io/packages/search?repo=localai)
2023-06-28 17:26:25 +00:00
**LocalAI** is a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.
2023-05-19 17:33:53 +00:00
2023-08-06 22:01:01 +00:00
< p align = "center" > < b > Follow LocalAI < / b > < / p >
< p align = "center" >
< a href = "https://twitter.com/LocalAI_API" target = "blank" >
< img src = "https://img.shields.io/twitter/follow/LocalAI_API?label=Follow: LocalAI_API&style=social" alt = "Follow LocalAI_API" / >
< / a >
< a href = "https://discord.gg/uJAeKSAGDy" target = "blank" >
< img src = "https://dcbadge.vercel.app/api/server/uJAeKSAGDy?style=flat-square&theme=default-inverted" alt = "Join LocalAI Discord Community" / >
< / a >
< p align = "center" > < b > Connect with the Creator < / b > < / p >
< p align = "center" >
< a href = "https://twitter.com/mudler_it" target = "blank" >
< img src = "https://img.shields.io/twitter/follow/mudler_it?label=Follow: mudler_it&style=social" alt = "Follow mudler_it" / >
< / a >
< a href = 'https://github.com/mudler' >
< img alt = "Follow on Github" src = "https://img.shields.io/badge/Follow-mudler-black?logo=github&link=https%3A%2F%2Fgithub.com%2Fmudler" >
< / a >
< / p >
< p align = "center" > < b > Share LocalAI Repository< / b > < / p >
< p align = "center" >
< a href = "https://twitter.com/intent/tweet?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.&url=https://github.com/go-skynet/LocalAI&hashtags=LocalAI,AI" target = "blank" >
< img src = "https://img.shields.io/twitter/follow/_LocalAI?label=Share Repo on Twitter&style=social" alt = "Follow _LocalAI" / > < / a >
< a href = "https://t.me/share/url?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.&url=https://github.com/go-skynet/LocalAI" target = "_blank" > < img src = "https://img.shields.io/twitter/url?label=Telegram&logo=Telegram&style=social&url=https://github.com/go-skynet/LocalAI" alt = "Share on Telegram" / > < / a >
< a href = "https://api.whatsapp.com/send?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.%20https://github.com/go-skynet/LocalAI" > < img src = "https://img.shields.io/twitter/url?label=whatsapp&logo=whatsapp&style=social&url=https://github.com/go-skynet/LocalAI" / > < / a > < a href = "https://www.reddit.com/submit?url=https://github.com/go-skynet/LocalAI&title=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.
" target="blank">
< img src = "https://img.shields.io/twitter/url?label=Reddit&logo=Reddit&style=social&url=https://github.com/go-skynet/LocalAI" alt = "Share on Reddit" / >
< / a > < a href = "mailto:?subject=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.%3A%0Ahttps://github.com/go-skynet/LocalAI" target = "_blank" > < img src = "https://img.shields.io/twitter/url?label=Gmail&logo=Gmail&style=social&url=https://github.com/go-skynet/LocalAI" / > < / a > < a href = "https://www.buymeacoffee.com/mudler" target = "_blank" > < img src = "https://cdn.buymeacoffee.com/buttons/default-orange.png" alt = "Buy Me A Coffee" height = "23" width = "100" style = "border-radius:1px" > < / a >
< / p >
< hr >
2023-05-19 17:33:53 +00:00
In a nutshell:
2023-04-11 22:04:15 +00:00
2023-05-18 13:59:03 +00:00
- Local, OpenAI drop-in alternative REST API. You own your data.
2023-06-28 17:26:25 +00:00
- NO GPU required. NO Internet access is required either
- Optional, GPU Acceleration is available in `llama.cpp` -compatible LLMs. See also the [build section ](https://localai.io/basics/build/index.html ).
2023-08-06 22:01:01 +00:00
- Supports multiple models
2023-06-28 17:26:25 +00:00
- 🏃 Once loaded the first time, it keep models loaded in memory for faster inference
2023-08-06 22:01:01 +00:00
- ⚡ Doesn't shell-out, but uses C++ bindings for a faster inference and better performance.
2023-06-28 17:26:25 +00:00
LocalAI was created by [Ettore Di Giacinto ](https://github.com/mudler/ ) and is a community-driven project, focused on making the AI accessible to anyone. Any contribution, feedback and PR is welcome!
2023-03-30 16:46:11 +00:00
2023-07-27 19:56:05 +00:00
Note that this started just as a [fun weekend project ](https://localai.io/#backstory ) in order to try to create the necessary pieces for a full AI assistant like `ChatGPT` : the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!
2023-08-06 22:01:01 +00:00
## 🚀 Features
2023-05-19 17:33:53 +00:00
2023-08-06 22:01:01 +00:00
- 📖 [Text generation with GPTs ](https://localai.io/features/text-generation/ ) (`llama.cpp`, `gpt4all.cpp` , ... [:book: and more ](https://localai.io/model-compatibility/index.html#model-compatibility-table ))
- 🗣 [Text to Audio ](https://localai.io/features/text-to-audio/ )
- 🔈 [Audio to Text ](https://localai.io/features/audio-to-text/ ) (Audio transcription with `whisper.cpp` )
- 🎨 [Image generation with stable diffusion ](https://localai.io/features/image-generation )
- 🔥 [OpenAI functions ](https://localai.io/features/openai-functions/ ) 🆕
- 🧠 [Embeddings generation for vector databases ](https://localai.io/features/embeddings/ )
- ✍️ [Constrained grammars ](https://localai.io/features/constrained_grammars/ )
- 🖼️ [Download Models directly from Huggingface ](https://localai.io/models/ )
2023-05-14 16:08:42 +00:00
2023-08-06 22:01:01 +00:00
## 🔥🔥 Hot topics / Roadmap
2023-07-02 22:52:26 +00:00
- [x] Support for embeddings
- [x] Support for audio transcription with https://github.com/ggerganov/whisper.cpp
- [X] Support for text-to-audio
- [x] GPU/CUDA support ( https://github.com/go-skynet/LocalAI/issues/69 )
- [X] Enable automatic downloading of models from a curated gallery
- [X] Enable automatic downloading of models from HuggingFace
- [ ] Upstream our golang bindings to llama.cpp (https://github.com/ggerganov/llama.cpp/issues/351)
- [ ] Enable gallery management directly from the webui.
2023-07-23 16:58:24 +00:00
- [x] 🔥 OpenAI functions: https://github.com/go-skynet/LocalAI/issues/588
2023-07-27 19:56:05 +00:00
- [ ] 🔥 GPTQ support: https://github.com/go-skynet/LocalAI/issues/796
2023-07-02 22:52:26 +00:00
2023-08-06 22:01:01 +00:00
## :book: 🎥 Media, Blogs, Social
2023-07-02 22:52:26 +00:00
- [Create a slackbot for teams and OSS projects that answer to documentation ](https://mudler.pm/posts/smart-slackbot-for-teams/ )
- [LocalAI meets k8sgpt ](https://www.youtube.com/watch?v=PKrDNuJ_dfE )
- [Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All ](https://mudler.pm/posts/localai-question-answering/ )
- [Tutorial to use k8sgpt with LocalAI ](https://medium.com/@tyler_97636/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65 )
2023-08-06 22:01:01 +00:00
## 💻 Usage
2023-04-12 23:13:14 +00:00
2023-08-06 22:04:28 +00:00
Check out the [Getting started ](https://localai.io/basics/getting_started/index.html ) section in our documentation.
2023-04-12 23:13:14 +00:00
2023-08-06 22:01:01 +00:00
### 💡 Example: Use GPT4ALL-J model
2023-04-24 21:42:03 +00:00
2023-08-06 22:04:28 +00:00
See the [documentation ](https://localai.io/basics/getting_started/#example-use-gpt4all-j-model-with-docker-compose )
2023-05-03 13:51:54 +00:00
2023-08-06 22:01:01 +00:00
### 🔗 Resources
2023-05-19 17:33:53 +00:00
2023-08-06 22:01:01 +00:00
- [How to build locally ](https://localai.io/basics/build/index.html )
- [How to install in Kubernetes ](https://localai.io/basics/getting_started/index.html#run-localai-in-kubernetes )
- [Projects integrating LocalAI ](https://localai.io/integrations/ )
2023-04-24 16:10:58 +00:00
2023-08-06 22:01:01 +00:00
## ❤️ Sponsors
2023-07-09 12:14:54 +00:00
> Do you find LocalAI useful?
Support the project by becoming [a backer or sponsor ](https://github.com/sponsors/mudler ). Your logo will show up here with a link to your website.
A huge thank you to our generous sponsors who support this project:
|  |
|:-----------------------------------------------:|
| [Spectro Cloud ](https://www.spectrocloud.com/ ) |
| Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs! |
2023-08-06 22:01:01 +00:00
## 🌟 Star history
2023-04-27 08:39:01 +00:00
2023-04-27 04:18:18 +00:00
[](https://star-history.com/#go-skynet/LocalAI& Date)
2023-08-06 22:01:01 +00:00
## 📖 License
2023-03-30 16:46:11 +00:00
2023-05-29 21:09:19 +00:00
LocalAI is a community-driven project created by [Ettore Di Giacinto ](https://github.com/mudler/ ).
2023-04-27 08:39:01 +00:00
2023-08-06 22:01:01 +00:00
MIT - Author Ettore Di Giacinto
2023-05-03 09:45:22 +00:00
2023-08-06 22:01:01 +00:00
## 🙇 Acknowledgements
2023-03-30 16:46:11 +00:00
2023-05-16 17:32:53 +00:00
LocalAI couldn't have been built without the help of great software already available from the community. Thank you!
2023-03-30 16:46:11 +00:00
- [llama.cpp ](https://github.com/ggerganov/llama.cpp )
- https://github.com/tatsu-lab/stanford_alpaca
- https://github.com/cornelk/llama-go for the initial ideas
2023-05-16 17:32:53 +00:00
- https://github.com/antimatter15/alpaca.cpp
- https://github.com/EdVince/Stable-Diffusion-NCNN
- https://github.com/ggerganov/whisper.cpp
- https://github.com/saharNooby/rwkv.cpp
2023-08-06 22:01:01 +00:00
- https://github.com/rhasspy/piper
- https://github.com/cmp-nct/ggllm.cpp
2023-04-28 08:54:39 +00:00
2023-08-06 22:01:01 +00:00
## 🤗 Contributors
2023-04-28 08:54:39 +00:00
2023-08-06 22:01:01 +00:00
This is a community project, a special thanks to our contributors! 🤗
2023-04-28 08:54:39 +00:00
< a href = "https://github.com/go-skynet/LocalAI/graphs/contributors" >
< img src = "https://contrib.rocks/image?repo=go-skynet/LocalAI" / >
< / a >
