6 commits

Author	SHA1	Message	Date
oobabooga	ee0592473c	Fix ExLlamaV3_HF leaking memory (attempt)	2025-04-27 21:04:02 -07:00
oobabooga	d4017fbb6d	ExLlamaV3: Add kv cache quantization (#6903)	2025-04-25 21:32:00 -03:00
oobabooga	d4b1e31c49	Use --ctx-size to specify the context size for all loaders ... Old flags are still recognized as alternatives.	2025-04-25 16:59:03 -07:00
oobabooga	b3bf7a885d	Fix ExLlamaV2_HF and ExLlamaV3_HF after ae02ffc605	2025-04-20 11:32:48 -07:00
oobabooga	1c4a2c9a71	Make exllamav3 safer as well	2025-04-18 06:17:58 -07:00
oobabooga	8b8d39ec4e	Add ExLlamaV3 support (#6832)	2025-04-09 00:07:08 -03:00