oobabooga 7618f3fe8c Add -gptq-preload for 4-bit offloading (#460) ... This works in a 4GB card now: ``` python server.py --model llama-7b-hf --gptq-bits 4 --gptq-pre-layer 20 ```		2023-03-20 16:30:56 -03:00
..
callbacks.py	Sort the imports	2023-03-17 11:42:25 -03:00
chat.py	Use markdown in the "HTML" tab	2023-03-17 16:06:11 -03:00
deepspeed_parameters.py	Fix deepspeed (oops)	2023-02-02 10:39:37 -03:00
extensions.py	Exception handling (#454)	2023-03-20 13:36:52 -03:00
GPTQ_loader.py	Add -gptq-preload for 4-bit offloading (#460)	2023-03-20 16:30:56 -03:00
html_generator.py	Use markdown in the "HTML" tab	2023-03-17 16:06:11 -03:00
LoRA.py	Make custom LoRAs work by default #385	2023-03-19 12:11:35 -03:00
models.py	--no-cache and --gpu-memory in MiB for fine VRAM control	2023-03-19 19:21:41 -03:00
RWKV.py	Use str(Path) instead of os.path.abspath(Path)	2023-03-13 00:08:01 -03:00
shared.py	Add -gptq-preload for 4-bit offloading (#460)	2023-03-20 16:30:56 -03:00
text_generation.py	Exception handling (#454)	2023-03-20 13:36:52 -03:00
ui.py	Further reorganize the UI	2023-03-15 13:24:54 -03:00