oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-04-07 15:43:49 +00:00
Code Issues Projects Releases Packages Wiki Activity

API: Remove deprecated "settings" parameter from model load endpoint

Browse source
This commit is contained in:
oobabooga 2026-04-04 11:00:14 -07:00
parent 9183dc444e
commit 2eef90a323
3 changed files with 5 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

20
modules/api/script.py
View file

@ -475,10 +475,8 @@ async def handle_list_models():
@app.post("/v1/internal/model/load", dependencies=check_admin_key)
async def handle_load_model(request_data: LoadModelRequest):
'''
This endpoint is experimental and may change in the future.
The "args" parameter can be used to modify flags like "--load-in-4bit"
or "--n-gpu-layers" before loading a model. Example:
The "args" parameter can be used to modify loader flags before loading
a model. Example:
```
"args": {
@ -487,18 +485,8 @@ async def handle_load_model(request_data: LoadModelRequest):
}
```
Note that those settings will remain after loading the model. So you
may need to change them back to load a second model.
The "settings" parameter is also a dict but with keys for the
shared.settings object. It can be used to modify the default instruction
template like this:
```
"settings": {
"instruction_template": "Alpaca"
}
```
Loader args are reset to their startup defaults between loads, so
settings from a previous load do not leak into the next one.
'''
try: