oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-02-16 04:35:32 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

llama.cpp: Explicitly send cache_prompt = True

Browse source
This commit is contained in:
oobabooga 2025-04-30 15:24:07 -07:00
parent 195a45c6e1
commit a6c3ec2299
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
modules/llama_cpp_server.py
View file

@ -135,6 +135,7 @@ class LlamaServer:
"prompt": token_ids,
"n_predict": max_new_tokens,
"stream": True,
"cache_prompt": True
})
if shared.args.verbose: