mirror of
https://github.com/mudler/LocalAI.git
synced 2024-06-07 19:40:48 +00:00
30 lines
1.2 KiB
Markdown
30 lines
1.2 KiB
Markdown
|
|
||
|
|
+++
|
||
|
|
disableToc = false
|
||
|
|
title = "🆕 GPT Vision"
|
||
|
|
weight = 2
|
||
|
|
+++
|
||
|
|
|
||
|
|
{{% notice note %}}
|
||
|
|
Available only on `master` builds
|
||
|
|
{{% /notice %}}
|
||
|
|
|
||
|
|
LocalAI supports understanding images by using [LLaVA](https://llava.hliu.cc/), and implements the [GPT Vision API](https://platform.openai.com/docs/guides/vision) from OpenAI.
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
OpenAI docs: https://platform.openai.com/docs/guides/vision
|
||
|
|
|
||
|
|
To let LocalAI understand and reply with what sees in the image, use the `/v1/chat/completions` endpoint, for example with curl:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
|
||
|
|
"model": "llava",
|
||
|
|
"messages": [{"role": "user", "content": [{"type":"text", "text": "What is in the image?"}, {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" }}], "temperature": 0.9}]}'
|
||
|
|
```
|
||
|
|
|
||
|
|
### Setup
|
||
|
|
|
||
|
|
To setup the LLaVa models, follow the full example in the [configuration examples](https://github.com/mudler/LocalAI/blob/master/examples/configurations/README.md#llava).
|