mirror of
https://github.com/oobabooga/text-generation-webui.git
synced 2025-12-06 07:12:10 +01:00
17 lines
465 B
Markdown
17 lines
465 B
Markdown
|
# ExLlama
|
||
|
|
|
||
|
|
## About
|
||
|
|
|
||
|
|
ExLlama is an extremely optimized GPTQ backend for LLaMA models. It features much lower VRAM usage and much higher speeds due to not relying on unoptimized transformers code.
|
||
|
|
|
||
|
|
# Installation:
|
||
|
|
|
||
|
|
1) Clone the ExLlama repository into your `repositories` folder:
|
||
|
|
|
||
|
|
```
|
||
|
|
cd repositories
|
||
|
|
git clone https://github.com/turboderp/exllama
|
||
|
|
```
|
||
|
|
|
||
|
|
2) Follow the remaining set up instructions in the official README: https://github.com/turboderp/exllama#exllama
|