oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-04-04 14:17:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update cache_4bit documentation (#5649)

Browse source 
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
This commit is contained in:
Bartowski 2024-03-07 11:08:21 -05:00 • committed by GitHub
parent bef08129bc
commit 104573f7d4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 6 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/04 - Model Tab.md
View file

@ -46,6 +46,7 @@ Examples:
* **cfg-cache**: Creates a second cache to hold the CFG negative prompts. You need to set this if and only if you intend to use CFG in the "Parameters" > "Generation" tab. Checking this parameter doubles the cache VRAM usage.
* **no_flash_attn**: Disables flash attention. Otherwise, it is automatically used as long as the library is installed.
* **cache_8bit**: Create a 8-bit precision cache instead of a 16-bit one. This saves VRAM but increases perplexity (I don't know by how much).
* **cache_4bit**: Creates a Q4 cache using grouped quantization.
### ExLlamav2