mirror of
https://github.com/mudler/LocalAI.git
synced 2024-06-07 19:40:48 +00:00
e7cbe32601
* fixes #1775 and #1774 Add BitsAndBytes Quantization and fixes embedding on CUDA devices * Manage 4bit and 8 bit quantization Manage different BitsAndBytes options with the quantization: parameter in yaml * fix compilation errors on non CUDA environment * OpenVINO draft First draft of OpenVINO integration in transformer backend * first working implementation * Streaming working * Small fix for regression on CUDA and XPU * use pip version of optimum[openvino] * Update backend/python/transformers/transformers_server.py Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> --------- Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| autogptq | ||
| bark | ||
| common-env/transformers | ||
| coqui | ||
| diffusers | ||
| exllama | ||
| exllama2 | ||
| mamba | ||
| petals | ||
| sentencetransformers | ||
| transformers | ||
| transformers-musicgen | ||
| vall-e-x | ||
| vllm | ||
| README.md | ||
Common commands about conda environment
Create a new empty conda environment
conda create --name <env-name> python=<your version> -y
conda create --name autogptq python=3.11 -y
To activate the environment
As of conda 4.4
conda activate autogptq
The conda version older than 4.4
source activate autogptq
Install the packages to your environment
Sometimes you need to install the packages from the conda-forge channel
By using conda
conda install <your-package-name>
conda install -c conda-forge <your package-name>
Or by using pip
pip install <your-package-name>