LocalAI

# copy your models to models/ cp your-model.bin models/ # (optional) Edit the .env file to set things like context size and threads # vim .env # start with docker-compose docker-compose up -d --pull always # or you can build the images with: # docker-compose up -d --build # Now API is accessible at localhost:8080 curl http://localhost:8080/v1/models # {"object":"list","data":[{"id":"your-model.bin","object":"model"}]} curl http://localhost:8080/v1/completions -H "Content-Type: application/json" -d '{ "model": "your-model.bin", "prompt": "A long time ago in a galaxy far, far away", "temperature": 0.7 }' ``` ### Example: Use GPT4ALL-J model

```bash # Clone LocalAI git clone https://github.com/go-skynet/LocalAI cd LocalAI # (optional) Checkout a specific LocalAI tag # git checkout -b build # Download gpt4all-j to models/ wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j # Use a template from the examples cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/ # (optional) Edit the .env file to set things like context size and threads # vim .env # start with docker-compose docker-compose up -d --pull always # or you can build the images with: # docker-compose up -d --build # Now API is accessible at localhost:8080 curl http://localhost:8080/v1/models # {"object":"list","data":[{"id":"ggml-gpt4all-j","object":"model"}]} curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "ggml-gpt4all-j", "messages": [{"role": "user", "content": "How are you?"}], "temperature": 0.9 }' # {"model":"ggml-gpt4all-j","choices":[{"message":{"role":"assistant","content":"I'm doing well, thanks. How about you?"}}]} ```

### Build locally

In order to build the `LocalAI` container image locally you can use `docker`: ``` # build the image docker build -t localai . docker run localai ``` Or you can build the binary with `make`: ``` make build ```

See the [build section](https://localai.io/basics/build/index.html) in our documentation for detailed instructions. ### Run LocalAI in Kubernetes LocalAI can be installed inside Kubernetes with helm. See [installation instructions](https://localai.io/basics/getting_started/index.html#run-localai-in-kubernetes). ## Supported API endpoints See the [list of the LocalAI features](https://localai.io/features/index.html) for a full tour of the available API endpoints. ## Frequently asked questions See [the FAQ](https://localai.io/faq/index.html) section for a list of common questions. ## Projects already using LocalAI to run local models Feel free to open up a PR to get your project listed! - [Kairos](https://github.com/kairos-io/kairos) - [k8sgpt](https://github.com/k8sgpt-ai/k8sgpt#running-local-models) - [Spark](https://github.com/cedriking/spark) - [autogpt4all](https://github.com/aorumbayev/autogpt4all) - [Mods](https://github.com/charmbracelet/mods) - [Flowise](https://github.com/FlowiseAI/Flowise) - [BMO Chatbot](https://github.com/longy2k/obsidian-bmo-chatbot) - [Mattermost OpenOps](https://openops.mattermost.com) ## Sponsors > Do you find LocalAI useful? Support the project by becoming [a backer or sponsor](https://github.com/sponsors/mudler). Your logo will show up here with a link to your website. A huge thank you to our generous sponsors who support this project: |  | |:-----------------------------------------------:| | [Spectro Cloud](https://www.spectrocloud.com/) | | Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs! | ## Star history [](https://star-history.com/#go-skynet/LocalAI&Date) ## License LocalAI is a community-driven project created by [Ettore Di Giacinto](https://github.com/mudler/). MIT ## Author Ettore Di Giacinto and others ## Acknowledgements LocalAI couldn't have been built without the help of great software already available from the community. Thank you! - [llama.cpp](https://github.com/ggerganov/llama.cpp) - https://github.com/tatsu-lab/stanford_alpaca - https://github.com/cornelk/llama-go for the initial ideas - https://github.com/antimatter15/alpaca.cpp - https://github.com/EdVince/Stable-Diffusion-NCNN - https://github.com/ggerganov/whisper.cpp - https://github.com/saharNooby/rwkv.cpp ## Contributors