LocalAI/examples/continue/README.md
2023-08-22 18:46:45 +02:00

2.4 KiB

Continue

logo

This document presents an example of integration with continuedev/continue.

Screenshot

For a live demonstration, please click on the link below:

Integration Setup Walkthrough

  1. As outlined in continue's documentation, install the Visual Studio Code extension from the marketplace and open it.

  2. In this example, LocalAI will download the gpt4all model and set it up as "gpt-3.5-turbo". Refer to the docker-compose.yaml file for details.

    # Clone LocalAI
    git clone https://github.com/go-skynet/LocalAI
    
    cd LocalAI/examples/continue
    
    # Start with docker-compose
    docker-compose up --build -d
    
  3. Type /config within Continue's VSCode extension, or edit the file located at ~/.continue/config.py on your system with the following configuration:

    from continuedev.src.continuedev.libs.llm.openai import OpenAI, OpenAIServerInfo
    
    config = ContinueConfig(
       ...
       models=Models(
            default=OpenAI(
               api_key="my-api-key",
               model="gpt-3.5-turbo",
               openai_server_info=OpenAIServerInfo(
                  api_base="http://localhost:8080",
                  model="gpt-3.5-turbo"
               )
            )
       ),
    )
    

This setup enables you to make queries directly to your model running in the Docker container. Note that the api_key does not need to be properly set up; it is included here as a placeholder.

If editing the configuration seems confusing, you may copy and paste the provided default config.py file over the existing one in ~/.continue/config.py after initializing the extension in the VSCode IDE.

Additional Resources