oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-04-20 22:13:43 +00:00
Code Issues Projects Releases Packages Wiki Activity

Add a "Static KV cache" option for transformers

Browse source
This commit is contained in:
oobabooga 2025-01-04 17:52:57 -08:00
parent 3967520e71
commit 11af199aff
7 changed files with 10 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
modules/shared.py
View file

@ -46,6 +46,7 @@ settings = {
'max_tokens_second': 0,
'max_updates_second': 0,
'prompt_lookup_num_tokens': 0,
'static_cache': False,
'custom_stopping_strings': '',
'custom_token_bans': '',
'auto_max_new_tokens': False,