4 Commits

Author	SHA1	Message	Date
Ettore Di Giacinto	a28ab18987	feat(vllm): Allow to set quantization (#1094) ... This particularly useful to set AWQ **Description** Follow up of #1015 **Notes for Reviewers** **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)** - [ ] Yes, I signed my commits. <!-- Thank you for contributing to LocalAI! Contributing Conventions: 1. Include descriptive PR titles with [<component-name>] prepended. 2. Build and test your changes before submitting a PR. 3. Sign your commits By following the community's contribution conventions upfront, the review process will be accelerated and your PR merged more quickly. --> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2023-09-22 15:52:38 +02:00
Ettore Di Giacinto	bdf3f95346	feat(python-grpc): allow to set max workers with PYTHON_GRPC_MAX_WORKERS (#1081) ... **Description** this allows to customize the maximum number of grpc workers for python backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2023-09-19 21:30:39 +02:00
Ettore Di Giacinto	453e9c5da9	fix(vllm): set default top_p with vllm (#1078) ... **Description** This PR fixes vllm when called with a request with an empty top_p Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2023-09-19 18:10:23 +02:00
Ettore Di Giacinto	c0bb5c4bf6	feat(vllm): Initial vllm backend implementation ... Related to: https://github.com/go-skynet/LocalAI/issues/1015 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2023-09-09 17:03:23 +02:00