mirror of
https://github.com/mudler/LocalAI.git
synced 2024-06-07 19:40:48 +00:00
docs: enhance and condense few sections (#2178)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
GitHub
parent
01860674c4
commit
a24cd4fda0
@ -1,13 +1,14 @@
|
|||||||
|
|
||||||
+++
|
+++
|
||||||
disableToc = false
|
disableToc = false
|
||||||
title = "Available Container images"
|
title = "Run with container images"
|
||||||
weight = 25
|
weight = 6
|
||||||
|
url = '/basics/container/'
|
||||||
|
ico = "rocket_launch"
|
||||||
+++
|
+++
|
||||||
|
|
||||||
LocalAI provides a variety of images to support different environments. These images are available on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags) and [Docker Hub](https://hub.docker.com/r/localai/localai).
|
LocalAI provides a variety of images to support different environments. These images are available on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags) and [Docker Hub](https://hub.docker.com/r/localai/localai).
|
||||||
|
|
||||||
> _For All-in-One image with a pre-configured set of models and backends, see the [AIO Images]({{%relref "docs/reference/aio-images" %}})._
|
All-in-One images comes with a pre-configured set of models and backends, standard images instead do not have any model pre-configured and installed.
|
||||||
|
|
||||||
For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA images, if you don't have a GPU, use the CPU images. If you have AMD or Mac Silicon, see the [build section]({{%relref "docs/getting-started/build" %}}).
|
For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA images, if you don't have a GPU, use the CPU images. If you have AMD or Mac Silicon, see the [build section]({{%relref "docs/getting-started/build" %}}).
|
||||||
|
|
||||||
@ -22,6 +23,62 @@ For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA
|
|||||||
|
|
||||||
{{% /alert %}}
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## All-in-one images
|
||||||
|
|
||||||
|
All-In-One images are images that come pre-configured with a set of models and backends to fully leverage almost all the LocalAI featureset. These images are available for both CPU and GPU environments. The AIO images are designed to be easy to use and requires no configuration. Models configuration can be found [here](https://github.com/mudler/LocalAI/tree/master/aio) separated by size.
|
||||||
|
|
||||||
|
In the AIO images there are models configured with the names of OpenAI models, however, they are really backed by Open Source models. You can find the table below
|
||||||
|
|
||||||
|
| Category | Model name | Real model (CPU) | Real model (GPU) |
|
||||||
|
| ---- | ---- | ---- | ---- |
|
||||||
|
| Text Generation | `gpt-4` | `phi-2` | `hermes-2-pro-mistral` |
|
||||||
|
| Multimodal Vision | `gpt-4-vision-preview` | `bakllava` | `llava-1.6-mistral` |
|
||||||
|
| Image Generation | `stablediffusion` | `stablediffusion` | `dreamshaper-8` |
|
||||||
|
| Speech to Text | `whisper-1` | `whisper` with `whisper-base` model | <= same |
|
||||||
|
| Text to Speech | `tts-1` | `en-us-amy-low.onnx` from `rhasspy/piper` | <= same |
|
||||||
|
| Embeddings | `text-embedding-ada-002` | `all-MiniLM-L6-v2` in Q4 | `all-MiniLM-L6-v2` |
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
Select the image (CPU or GPU) and start the container with Docker:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# CPU example
|
||||||
|
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu
|
||||||
|
```
|
||||||
|
|
||||||
|
LocalAI will automatically download all the required models, and the API will be available at [localhost:8080](http://localhost:8080/v1/models).
|
||||||
|
|
||||||
|
### Available images
|
||||||
|
|
||||||
|
| Description | Quay | Docker Hub |
|
||||||
|
| --- | --- |-----------------------------------------------|
|
||||||
|
| Latest images for CPU | `quay.io/go-skynet/local-ai:latest-aio-cpu` | `localai/localai:latest-aio-cpu` |
|
||||||
|
| Versioned image (e.g. for CPU) | `quay.io/go-skynet/local-ai:{{< version >}}-aio-cpu` | `localai/localai:{{< version >}}-aio-cpu` |
|
||||||
|
| Latest images for Nvidia GPU (CUDA11) | `quay.io/go-skynet/local-ai:latest-aio-gpu-nvidia-cuda-11` | `localai/localai:latest-aio-gpu-nvidia-cuda-11` |
|
||||||
|
| Latest images for Nvidia GPU (CUDA12) | `quay.io/go-skynet/local-ai:latest-aio-gpu-nvidia-cuda-12` | `localai/localai:latest-aio-gpu-nvidia-cuda-12` |
|
||||||
|
| Latest images for AMD GPU | `quay.io/go-skynet/local-ai:latest-aio-gpu-hipblas` | `localai/localai:latest-aio-gpu-hipblas` |
|
||||||
|
| Latest images for Intel GPU (sycl f16) | `quay.io/go-skynet/local-ai:latest-aio-gpu-intel-f16` | `localai/localai:latest-aio-gpu-intel-f16` |
|
||||||
|
| Latest images for Intel GPU (sycl f32) | `quay.io/go-skynet/local-ai:latest-aio-gpu-intel-f32` | `localai/localai:latest-aio-gpu-intel-f32` |
|
||||||
|
|
||||||
|
### Available environment variables
|
||||||
|
|
||||||
|
The AIO Images are inheriting the same environment variables as the base images and the environment of LocalAI (that you can inspect by calling `--help`). However, it supports additional environment variables available only from the container image
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
| ---------------------| ------- | ----------- |
|
||||||
|
| `PROFILE` | Auto-detected | The size of the model to use. Available: `cpu`, `gpu-8g` |
|
||||||
|
| `MODELS` | Auto-detected | A list of models YAML Configuration file URI/URL (see also [running models]({{%relref "docs/getting-started/run-other-models" %}})) |
|
||||||
|
|
||||||
|
|
||||||
|
## Standard container images
|
||||||
|
|
||||||
|
Standard container images do not have pre-installed models.
|
||||||
|
|
||||||
|
Images are available with and without python dependencies. Note that images with python dependencies are bigger (in order of 17GB).
|
||||||
|
|
||||||
|
Images with `core` in the tag are smaller and do not contain any python dependencies.
|
||||||
|
|
||||||
{{< tabs tabTotal="6" >}}
|
{{< tabs tabTotal="6" >}}
|
||||||
{{% tab tabName="Vanilla / CPU Images" %}}
|
{{% tab tabName="Vanilla / CPU Images" %}}
|
||||||
|
|
||||||
@ -100,4 +157,3 @@ For GPU Acceleration support for Nvidia video graphic cards, use the Nvidia/CUDA
|
|||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [GPU acceleration]({{%relref "docs/features/gpu-acceleration" %}})
|
- [GPU acceleration]({{%relref "docs/features/gpu-acceleration" %}})
|
||||||
- [AIO Images]({{%relref "docs/reference/aio-images" %}})
|
|
||||||
30
docs/content/docs/getting-started/kubernetes.md
Normal file
30
docs/content/docs/getting-started/kubernetes.md
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
+++
|
||||||
|
disableToc = false
|
||||||
|
title = "Run with Kubernetes"
|
||||||
|
weight = 6
|
||||||
|
url = '/basics/kubernetes/'
|
||||||
|
ico = "rocket_launch"
|
||||||
|
+++
|
||||||
|
|
||||||
|
For installing LocalAI in Kubernetes, you can use the `go-skynet` helm chart:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install the helm repository
|
||||||
|
helm repo add go-skynet https://go-skynet.github.io/helm-charts/
|
||||||
|
# Update the repositories
|
||||||
|
helm repo update
|
||||||
|
# Get the values
|
||||||
|
helm show values go-skynet/local-ai > values.yaml
|
||||||
|
|
||||||
|
# Edit the values value if needed
|
||||||
|
# vim values.yaml ...
|
||||||
|
|
||||||
|
# Install the helm chart
|
||||||
|
helm install local-ai go-skynet/local-ai -f values.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
If you prefer to install from manifest file, you can install from the deployment file, and customize as you like:
|
||||||
|
|
||||||
|
```
|
||||||
|
kubectl apply -f https://raw.githubusercontent.com/mudler/LocalAI/master/examples/kubernetes/deployment.yaml
|
||||||
|
```
|
||||||
@ -131,22 +131,7 @@ Note: If you are on Windows, please make sure the project is on the Linux Filesy
|
|||||||
|
|
||||||
{{% tab tabName="Kubernetes" %}}
|
{{% tab tabName="Kubernetes" %}}
|
||||||
|
|
||||||
For installing LocalAI in Kubernetes, you can use the following helm chart:
|
See the [Kubernetes section]({{%relref "docs/getting-started/kubernetes" %}}).
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install the helm repository
|
|
||||||
helm repo add go-skynet https://go-skynet.github.io/helm-charts/
|
|
||||||
# Update the repositories
|
|
||||||
helm repo update
|
|
||||||
# Get the values
|
|
||||||
helm show values go-skynet/local-ai > values.yaml
|
|
||||||
|
|
||||||
# Edit the values value if needed
|
|
||||||
# vim values.yaml ...
|
|
||||||
|
|
||||||
# Install the helm chart
|
|
||||||
helm install local-ai go-skynet/local-ai -f values.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
{{% /tab %}}
|
{{% /tab %}}
|
||||||
{{% tab tabName="From binary" %}}
|
{{% tab tabName="From binary" %}}
|
||||||
|
|||||||
@ -30,7 +30,7 @@ Before you begin, ensure you have a container engine installed if you are not us
|
|||||||
|
|
||||||
> _Do you have already a model file? Skip to [Run models manually]({{%relref "docs/getting-started/manual" %}}) or [Run other models]({{%relref "docs/getting-started/run-other-models" %}}) to use an already-configured model_.
|
> _Do you have already a model file? Skip to [Run models manually]({{%relref "docs/getting-started/manual" %}}) or [Run other models]({{%relref "docs/getting-started/run-other-models" %}}) to use an already-configured model_.
|
||||||
|
|
||||||
LocalAI's All-in-One (AIO) images are pre-configured with a set of models and backends to fully leverage almost all the LocalAI featureset. If you don't need models pre-configured, you can use the standard [images]({{%relref "docs/reference/container-images" %}}).
|
LocalAI's All-in-One (AIO) images are pre-configured with a set of models and backends to fully leverage almost all the LocalAI featureset. If you don't need models pre-configured, you can use the standard [images]({{%relref "docs/getting-started/container-images" %}}).
|
||||||
|
|
||||||
These images are available for both CPU and GPU environments. The AIO images are designed to be easy to use and requires no configuration.
|
These images are available for both CPU and GPU environments. The AIO images are designed to be easy to use and requires no configuration.
|
||||||
|
|
||||||
@ -91,7 +91,7 @@ services:
|
|||||||
# capabilities: [gpu]
|
# capabilities: [gpu]
|
||||||
```
|
```
|
||||||
|
|
||||||
For a list of all the container-images available, see [Container images]({{%relref "docs/reference/container-images" %}}). To learn more about All-in-one images instead, see [All-in-one Images]({{%relref "docs/reference/aio-images" %}}).
|
For a list of all the container-images available, see [Container images]({{%relref "docs/getting-started/container-images" %}}). To learn more about All-in-one images instead, see [All-in-one Images]({{%relref "docs/getting-started/container-images" %}}).
|
||||||
|
|
||||||
{{% alert icon="💡" %}}
|
{{% alert icon="💡" %}}
|
||||||
|
|
||||||
@ -114,9 +114,36 @@ docker run -p 8080:8080 --name local-ai -ti -v localai-models:/build/models loca
|
|||||||
|
|
||||||
{{% /alert %}}
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## From binary
|
||||||
|
|
||||||
|
LocalAI is available as a standalone binary as well. Binaries are compiled for Linux and MacOS and automatically uploaded in the Github releases. Windows is known to work with WSL.
|
||||||
|
|
||||||
|
You can check out the releases in https://github.com/mudler/LocalAI/releases.
|
||||||
|
|
||||||
|
{{< tabs tabTotal="2" >}}
|
||||||
|
{{% tab tabName="Linux" %}}
|
||||||
|
| CPU flagset | Link |
|
||||||
|
| --- | --- |
|
||||||
|
| avx2 | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx2-Linux-x86_64) |
|
||||||
|
| avx512 | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx512-Linux-x86_64) |
|
||||||
|
| avx | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx-Linux-x86_64) |
|
||||||
|
{{% /tab %}}
|
||||||
|
{{% tab tabName="MacOS" %}}
|
||||||
|
| CPU flagset | Link |
|
||||||
|
| --- | --- |
|
||||||
|
| avx2 | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx2-Darwin-arm64) |
|
||||||
|
| avx512 | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx512-Darwin-arm64) |
|
||||||
|
| avx | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-avx-Darwin-arm64) |
|
||||||
|
|
||||||
|
{{% /tab %}}
|
||||||
|
|
||||||
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Try it out
|
## Try it out
|
||||||
|
|
||||||
LocalAI does not ship a webui by default, however you can use 3rd party projects to interact with it (see also [Integrations]({{%relref "docs/integrations" %}}) ). However, you can test out the API endpoints using `curl`, you can find few examples below.
|
Connect to LocalAI, by default the WebUI should be accessible from http://localhost:8080 . You can also use 3rd party projects to interact with LocalAI as you would use OpenAI (see also [Integrations]({{%relref "docs/integrations" %}}) ).
|
||||||
|
|
||||||
|
You can also test out the API endpoints using `curl`, examples below.
|
||||||
|
|
||||||
### Text Generation
|
### Text Generation
|
||||||
|
|
||||||
@ -300,6 +327,6 @@ Explore further resources and community contributions:
|
|||||||
- [Build LocalAI and the container image]({{%relref "docs/getting-started/build" %}})
|
- [Build LocalAI and the container image]({{%relref "docs/getting-started/build" %}})
|
||||||
- [Run models manually]({{%relref "docs/getting-started/manual" %}})
|
- [Run models manually]({{%relref "docs/getting-started/manual" %}})
|
||||||
- [Run other models]({{%relref "docs/getting-started/run-other-models" %}})
|
- [Run other models]({{%relref "docs/getting-started/run-other-models" %}})
|
||||||
- [Container images]({{%relref "docs/reference/container-images" %}})
|
- [Container images]({{%relref "docs/getting-started/container-images" %}})
|
||||||
- [All-in-one Images]({{%relref "docs/reference/aio-images" %}})
|
- [All-in-one Images]({{%relref "docs/getting-started/container-images" %}})
|
||||||
- [Examples](https://github.com/mudler/LocalAI/tree/master/examples#examples)
|
- [Examples](https://github.com/mudler/LocalAI/tree/master/examples#examples)
|
||||||
|
|||||||
@ -1,53 +0,0 @@
|
|||||||
|
|
||||||
+++
|
|
||||||
disableToc = false
|
|
||||||
title = "All-In-One images"
|
|
||||||
weight = 26
|
|
||||||
+++
|
|
||||||
|
|
||||||
All-In-One images are images that come pre-configured with a set of models and backends to fully leverage almost all the LocalAI featureset. These images are available for both CPU and GPU environments. The AIO images are designed to be easy to use and requires no configuration. Models configuration can be found [here](https://github.com/mudler/LocalAI/tree/master/aio) separated by size.
|
|
||||||
|
|
||||||
In the AIO images there are models configured with the names of OpenAI models, however, they are really backed by Open Source models. You can find the table below
|
|
||||||
|
|
||||||
| Category | Model name | Real model (CPU) | Real model (GPU) |
|
|
||||||
| ---- | ---- | ---- | ---- |
|
|
||||||
| Text Generation | `gpt-4` | `phi-2` | `hermes-2-pro-mistral` |
|
|
||||||
| Multimodal Vision | `gpt-4-vision-preview` | `bakllava` | `llava-1.6-mistral` |
|
|
||||||
| Image Generation | `stablediffusion` | `stablediffusion` | `dreamshaper-8` |
|
|
||||||
| Speech to Text | `whisper-1` | `whisper` with `whisper-base` model | <= same |
|
|
||||||
| Text to Speech | `tts-1` | `en-us-amy-low.onnx` from `rhasspy/piper` | <= same |
|
|
||||||
| Embeddings | `text-embedding-ada-002` | `all-MiniLM-L6-v2` in Q4 | `all-MiniLM-L6-v2` |
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Select the image (CPU or GPU) and start the container with Docker:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# CPU example
|
|
||||||
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu
|
|
||||||
```
|
|
||||||
|
|
||||||
LocalAI will automatically download all the required models, and the API will be available at [localhost:8080](http://localhost:8080/v1/models).
|
|
||||||
|
|
||||||
## Available images
|
|
||||||
|
|
||||||
| Description | Quay | Docker Hub |
|
|
||||||
| --- | --- |-----------------------------------------------|
|
|
||||||
| Latest images for CPU | `quay.io/go-skynet/local-ai:latest-aio-cpu` | `localai/localai:latest-aio-cpu` |
|
|
||||||
| Versioned image (e.g. for CPU) | `quay.io/go-skynet/local-ai:{{< version >}}-aio-cpu` | `localai/localai:{{< version >}}-aio-cpu` |
|
|
||||||
| Latest images for Nvidia GPU (CUDA11) | `quay.io/go-skynet/local-ai:latest-aio-gpu-nvidia-cuda-11` | `localai/localai:latest-aio-gpu-nvidia-cuda-11` |
|
|
||||||
| Latest images for Nvidia GPU (CUDA12) | `quay.io/go-skynet/local-ai:latest-aio-gpu-nvidia-cuda-12` | `localai/localai:latest-aio-gpu-nvidia-cuda-12` |
|
|
||||||
| Latest images for AMD GPU | `quay.io/go-skynet/local-ai:latest-aio-gpu-hipblas` | `localai/localai:latest-aio-gpu-hipblas` |
|
|
||||||
| Latest images for Intel GPU (sycl f16) | `quay.io/go-skynet/local-ai:latest-aio-gpu-intel-f16` | `localai/localai:latest-aio-gpu-intel-f16` |
|
|
||||||
| Latest images for Intel GPU (sycl f32) | `quay.io/go-skynet/local-ai:latest-aio-gpu-intel-f32` | `localai/localai:latest-aio-gpu-intel-f32` |
|
|
||||||
|
|
||||||
## Available environment variables
|
|
||||||
|
|
||||||
The AIO Images are inheriting the same environment variables as the base images and the environment of LocalAI (that you can inspect by calling `--help`). However, it supports additional environment variables available only from the container image
|
|
||||||
|
|
||||||
| Variable | Default | Description |
|
|
||||||
| ---------------------| ------- | ----------- |
|
|
||||||
| `PROFILE` | Auto-detected | The size of the model to use. Available: `cpu`, `gpu-8g` |
|
|
||||||
| `MODELS` | Auto-detected | A list of models YAML Configuration file URI/URL (see also [running models]({{%relref "docs/getting-started/run-other-models" %}})) |
|
|
||||||
|
|
||||||
|
|
||||||
Loading…
Reference in New Issue
Block a user