Mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-06-07 19:40:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
45d520f913
LocalAI/backend/python/common-env/transformers
History
fakezeta 3882130911
feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823) 
* fixes #1775 and #1774

Add BitsAndBytes Quantization and fixes embedding on CUDA devices

* Manage 4bit and 8 bit quantization

Manage different BitsAndBytes options with the quantization: parameter in yaml

* fix compilation errors on non CUDA environment
2024-03-14 23:06:30 +01:00
..
install.sh feat(intel): add diffusers/transformers support (#1746) 2024-03-07 14:37:45 +01:00
Makefile feat(intel): add diffusers/transformers support (#1746) 2024-03-07 14:37:45 +01:00
transformers-nvidia.yml feat: Add Bitsandbytes quantization for transformer backend enhancement #1775 and fix: Transformer backend error on CUDA #1774 (#1823) 2024-03-14 23:06:30 +01:00
transformers-rocm.yml Update Backend Dependancies (#1797) 2024-03-05 10:10:00 +00:00
transformers.yml Update Backend Dependancies (#1797) 2024-03-05 10:10:00 +00:00