oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2025-12-06 07:12:10 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

llama.cpp: Always pass --flash-attn on

Browse source
This commit is contained in:
oobabooga 2025-09-02 12:12:17 -07:00
parent 2395c647d4
commit f3829b268a
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
modules/llama_cpp_server.py
View file

@ -316,6 +316,7 @@ class LlamaServer:
"--batch-size", str(shared.args.batch_size),
"--port", str(self.port),
"--no-webui",
"--flash-attn", "on",
]
if shared.args.threads > 0: