Mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-06-07 19:40:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Ettore Di Giacinto 2023-03-26 23:37:26 +02:00 committed by GitHub
parent abee34f60a
commit 1f45ff8cd6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
README.md
View File

@ -5,7 +5,7 @@ llama-cli is a straightforward golang CLI interface for [llama.cpp](https://gith
## Container images ## Container images
The `llama-cli` [container images](https://quay.io/repository/go-skynet/llama-cli?tab=tags&tag=latest) come preloaded with the [alpaca.cpp 7B](https://github.com/antimatter15/alpaca.cpp) model, enabling you to start making predictions immediately! To begin, run: To begin, run:
``` ```
docker run -ti --rm quay.io/go-skynet/llama-cli:v0.3 --instruction "What's an alpaca?" --topk 10000 docker run -ti --rm quay.io/go-skynet/llama-cli:v0.3 --instruction "What's an alpaca?" --topk 10000
@ -115,26 +115,8 @@ You can use the lite images ( for example `quay.io/go-skynet/llama-cli:v0.3-lite
13B and 30B models are known to work: 13B and 30B models are known to work:
### 13B
``` ```
# Download the model image, extract the model # Download the model image, extract the model
id=$(docker create quay.io/go-skynet/models:ggml2-alpaca-13b-v0.2)
docker cp $id:/models/model.bin ./
docker rm -v $id
# Use the model with llama-cli
docker run -v $PWD:/models -p 8080:8080 -ti --rm quay.io/go-skynet/llama-cli:v0.3-lite api --model /models/model.bin
```
### 30B
```
# Download the model image, extract the model
id=$(docker create quay.io/go-skynet/models:ggml2-alpaca-30b-v0.2)
docker cp $id:/models/model.bin ./
docker rm -v $id
# Use the model with llama-cli # Use the model with llama-cli
docker run -v $PWD:/models -p 8080:8080 -ti --rm quay.io/go-skynet/llama-cli:v0.3-lite api --model /models/model.bin docker run -v $PWD:/models -p 8080:8080 -ti --rm quay.io/go-skynet/llama-cli:v0.3-lite api --model /models/model.bin
``` ```