oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-01-07 17:20:19 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

UI: Set max_updates_second to 12 by default

Browse source 
When the tokens/second at at ~50 and the model is a thinking model,
the markdown rendering for the streaming message becomes a CPU
bottleneck.
This commit is contained in:
oobabooga 2025-04-30 14:53:15 -07:00
parent a4bf339724
commit b46ca01340
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
modules/shared.py
View file

@ -47,7 +47,7 @@ settings = {
'max_new_tokens_max': 4096,
'prompt_lookup_num_tokens': 0,
'max_tokens_second': 0,
'max_updates_second': 0,
'max_updates_second': 12,
'auto_max_new_tokens': True,
'ban_eos_token': False,
'add_bos_token': True,