oobabooga/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2026-01-15 21:20:23 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
5163 commits 2 branches 92 tags 66 MiB
Commit graph

1988 commits

Author SHA1 Message Date
oobabooga e0f5905a97 Code formatting 2025-08-19 06:34:05 -07:00
oobabooga 5b06284a8a UI: Keep ExLlamav3_HF selected if already selected for EXL3 models 2025-08-19 06:23:21 -07:00
oobabooga cbba58bef9 UI: Fix code blocks having an extra empty line 2025-08-18 15:50:09 -07:00
oobabooga 7d23a55901 Fix model unloading when switching loaders (closes #7203) 2025-08-18 09:05:47 -07:00
oobabooga 64eba9576c mtmd: Fix a bug when "include past attachments" is unchecked 2025-08-17 14:08:40 -07:00
oobabooga dbabe67e77 ExLlamaV3: Enable the --enable-tp option, add a --tp-backend option 2025-08-17 13:19:11 -07:00
oobabooga d771ca4a13 Fix web search (attempt) 2025-08-14 12:05:14 -07:00
altoiddealer 57f6e9af5a
Set multimodal status during Model Loading (#7199) 2025-08-13 16:47:27 -03:00
oobabooga 41b95e9ec3 Lint 2025-08-12 13:37:37 -07:00
oobabooga 7301452b41 UI: Minor info message change 2025-08-12 13:23:24 -07:00
oobabooga 8d7b88106a Revert "mtmd: Fail early if images are provided but the model doesn't support them (llama.cpp)" 
This reverts commit d8fcc71616.
 2025-08-12 13:20:16 -07:00
oobabooga 2238302b49 ExLlamaV3: Add speculative decoding 2025-08-12 08:50:45 -07:00
oobabooga d8fcc71616 mtmd: Fail early if images are provided but the model doesn't support them (llama.cpp) 2025-08-11 18:02:33 -07:00
oobabooga e6447cd24a mtmd: Update the llama-server request 2025-08-11 17:42:35 -07:00
oobabooga 0e3def449a llama.cpp: --swa-full to llama-server when streaming-llm is checked 2025-08-11 15:17:25 -07:00
oobabooga 0e88a621fd UI: Better organize the right sidebar 2025-08-11 15:16:03 -07:00
oobabooga a78ca6ffcd Remove a comment 2025-08-11 12:33:38 -07:00
oobabooga 999471256c Lint 2025-08-11 12:32:17 -07:00
oobabooga b62c8845f3 mtmd: Fix /chat/completions for llama.cpp 2025-08-11 12:01:59 -07:00
oobabooga 38c0b4a1ad Default ctx-size to 8192 when not found in the metadata 2025-08-11 07:39:53 -07:00
oobabooga 52d1cbbbe9 Fix an import 2025-08-11 07:38:39 -07:00
oobabooga 4809ddfeb8 Exllamav3: small sampler fixes 2025-08-11 07:35:22 -07:00
oobabooga 4d8dbbab64 API: Fix sampler_priority usage for ExLlamaV3 2025-08-11 07:26:11 -07:00
oobabooga 0ea62d88f6 mtmd: Fix "continue" when an image is present 2025-08-09 21:47:02 -07:00
oobabooga 2f90ac9880 Move the new image_utils.py file to modules/ 2025-08-09 21:41:38 -07:00
oobabooga c6b4d1e87f Fix the exllamav2 loader ignoring add_bos 2025-08-09 21:34:35 -07:00
oobabooga d86b0ec010
Add multimodal support (llama.cpp) (#7027) 2025-08-10 01:27:25 -03:00
oobabooga a289a92b94 Fix exllamav3 token count 2025-08-09 17:10:58 -07:00
oobabooga d489eb589a Attempt at fixing new exllamav3 loader undefined behavior when switching conversations 2025-08-09 14:11:31 -07:00
oobabooga a6d6bee88c Change a comment 2025-08-09 07:51:03 -07:00
oobabooga 2fe79a93cc mtmd: Handle another case after 3f5ec9644f 2025-08-09 07:50:24 -07:00
oobabooga 59c6138e98 Remove a log message 2025-08-09 07:32:15 -07:00
oobabooga f396b82a4f mtmd: Better way to detect if an EXL3 model is multimodal 2025-08-09 07:31:36 -07:00
oobabooga fa9be444fa Use ExLlamav3 instead of ExLlamav3_HF by default for EXL3 models 2025-08-09 07:26:59 -07:00
oobabooga 3f5ec9644f mtmd: Place the image <__media__> at the top of the prompt 2025-08-09 07:06:07 -07:00
oobabooga 1168004067 Minor change 2025-08-09 07:01:55 -07:00
oobabooga 9e260332cc Remove some unnecessary code 2025-08-08 21:22:47 -07:00
oobabooga 544c3a7c9f Polish the new exllamav3 loader 2025-08-08 21:15:53 -07:00
oobabooga 8fcadff8d3 mtmd: Use the base64 attachment for the UI preview instead of the file 2025-08-08 20:13:54 -07:00
oobabooga 6e9de75727 Support loading chat templates from chat_template.json files 2025-08-08 19:35:09 -07:00
Katehuuh 88127f46c1
Add multimodal support (ExLlamaV3) (#7174) 2025-08-08 23:31:16 -03:00
oobabooga b391ac8eb1 Fix getting the ctx-size for EXL3/EXL2/Transformers models 2025-08-08 18:11:45 -07:00
oobabooga 3e24f455c8 Fix continue for GPT-OSS (hopefully the final fix) 2025-08-06 10:18:42 -07:00
oobabooga 0c1403f2c7 Handle GPT-OSS as a special case when continuing 2025-08-06 08:05:37 -07:00
oobabooga 6ce4b353c4 Fix the GPT-OSS template 2025-08-06 07:12:39 -07:00
oobabooga 7c82d65a9d Handle GPT-OSS as a special template case 2025-08-05 18:05:09 -07:00
oobabooga fbea21a1f1 Only use enable_thinking if the template supports it 2025-08-05 17:33:27 -07:00
oobabooga bfbbfc2361 Ignore add_generation_prompt in GPT-OSS 2025-08-05 17:33:01 -07:00
oobabooga 20adc3c967 Start over new template handling (to avoid overcomplicating) 2025-08-05 16:58:45 -07:00
oobabooga 80f6abb07e Begin fixing 'Continue' with GPT-OSS 2025-08-05 16:01:19 -07:00
oobabooga e5b8d4d072 Fix a typo 2025-08-05 15:52:56 -07:00
oobabooga 701048cf33 Try to avoid breaking jinja2 parsing for older models 2025-08-05 15:51:24 -07:00
oobabooga 7d98ca6195 Make web search functional with thinking models 2025-08-05 15:44:33 -07:00
oobabooga 0e42575c57 Fix thinking block parsing for GPT-OSS under llama.cpp 2025-08-05 15:36:20 -07:00
oobabooga 498778b8ac Add a new 'Reasoning effort' UI element 2025-08-05 15:19:11 -07:00
oobabooga 6bb8212731 Fix thinking block rendering for GPT-OSS 2025-08-05 15:06:22 -07:00
oobabooga 5c5a4dfc14 Fix impersonate 2025-08-05 13:04:10 -07:00
oobabooga ecd16d6bf9 Automatically set skip_special_tokens to False for channel-based templates 2025-08-05 12:57:49 -07:00
oobabooga 178c3e75cc Handle templates with channels separately 2025-08-05 12:52:17 -07:00
oobabooga 9f28f53cfc Better parsing of the gpt-oss template 2025-08-05 11:56:00 -07:00
oobabooga 3b28dc1821 Don't pass torch_dtype to transformers loader, let it be autodetected 2025-08-05 11:35:53 -07:00
oobabooga 3039aeffeb Fix parsing the gpt-oss-20b template 2025-08-05 11:35:17 -07:00
oobabooga 5989043537 Transformers: Support standalone .jinja chat templates (for GPT-OSS) 2025-08-05 11:22:18 -07:00
oobabooga f08bb9a201 Handle edge case in chat history loading (closes #7155) 2025-07-24 10:34:59 -07:00
oobabooga d746484521 Handle both int and str types in grammar char processing 2025-07-23 11:52:51 -07:00
oobabooga 0c667de7a7 UI: Add a None option for the speculative decoding model (closes #7145) 2025-07-19 12:14:41 -07:00
oobabooga 845432b9b4 Remove the obsolete modules/relative_imports.py file 2025-07-14 21:03:18 -07:00
oobabooga 1d1b20bd77 Remove the --torch-compile option (it doesn't do anything currently) 2025-07-11 10:51:23 -07:00
oobabooga 273888f218 Revert "Use eager attention by default instead of sdpa" 
This reverts commit bd4881c4dc.
 2025-07-10 18:56:46 -07:00
oobabooga 635e6efd18 Ignore add_bos_token in instruct prompts, let the jinja2 template decide 2025-07-10 07:14:01 -07:00
oobabooga e015355e4a Update README 2025-07-09 20:03:53 -07:00
oobabooga bd4881c4dc Use eager attention by default instead of sdpa 2025-07-09 19:57:37 -07:00
oobabooga b69f435311 Fix latest transformers being super slow 2025-07-09 19:56:50 -07:00
oobabooga 6c2bdda0f0 Transformers loader: replace use_flash_attention_2/use_eager_attention with a unified attn_implementation 
Closes #7107
 2025-07-09 18:39:37 -07:00
oobabooga 07e6f004c5 Rename a button in the Session tab for clarity 2025-07-07 11:28:47 -07:00
Alidr79 e5767d4fc5
Update ui_model_menu.py blocking the --multi-user access in backend (#7098) 2025-07-06 21:48:53 -03:00
oobabooga 60123a67ac Better log message when extension requirements are not found 2025-07-06 17:44:41 -07:00
oobabooga e6bc7742fb Support installing user extensions in user_data/extensions/ 2025-07-06 17:30:23 -07:00
Philipp Claßen 959d4ddb91
Fix for chat sidebars toggle buttons disappearing (#7106) 2025-07-06 20:51:42 -03:00
oobabooga de4ccffff8 Fix the duckduckgo search 2025-07-06 16:24:57 -07:00
oobabooga 92ec8dda03 Fix chat history getting lost if the UI is inactive for a long time (closes #7109) 2025-07-04 06:04:04 -07:00
zombiegreedo 877c651c04
Handle either missing <think> start or </think> end tags (#7102) 2025-07-03 23:05:46 -03:00
oobabooga c3faecfd27 Minor change 2025-06-22 17:51:09 -07:00
oobabooga 1b19dd77a4 Move 'Enable thinking' to the Chat tab 2025-06-22 17:29:17 -07:00
oobabooga 02f604479d Remove the pre-jinja2 custom stopping string handling (closes #7094) 2025-06-21 14:03:35 -07:00
oobabooga 58282f7107 Replace 'Generate' with 'Send' in the Chat tab 2025-06-20 06:59:48 -07:00
oobabooga acd57b6a85 Minor UI change 2025-06-19 15:39:43 -07:00
oobabooga f08db63fbc Change some comments 2025-06-19 15:26:45 -07:00
oobabooga a1b606a6ac Fix obtaining the maximum number of GPU layers for DeepSeek-R1-0528-GGUF 2025-06-19 12:30:57 -07:00
oobabooga 3344510553 Force dark theme on the Gradio login page 2025-06-19 12:11:34 -07:00
oobabooga 645463b9f0 Add fallback values for theme colors 2025-06-19 11:28:12 -07:00
oobabooga 9c6913ad61 Show file sizes on "Get file list" 2025-06-18 21:35:07 -07:00
oobabooga 0cb82483ef Lint 2025-06-18 18:26:59 -07:00
oobabooga 6cc7bbf009 Better autosave behavior for notebook tab when there are 2 columns 2025-06-18 15:54:32 -07:00
oobabooga 197b327374 Minor log message change 2025-06-18 13:36:54 -07:00
oobabooga 2f45d75309 Increase the area of the notebook textbox 2025-06-18 13:22:06 -07:00
oobabooga 7cb2b1bfdb Fix some events 2025-06-18 10:27:38 -07:00
oobabooga 22cc9e0115 Remove 'Send to Default' 2025-06-18 10:21:48 -07:00
oobabooga 678f40297b Clear the default tab output when switching prompts 2025-06-17 17:40:48 -07:00
oobabooga da148232eb Better filenames for new prompts in the Notebook tab 2025-06-17 15:10:44 -07:00
oobabooga fc23345c6d Send the default input to the notebook textbox when switching 2 columns to 1 (instead of the output) 2025-06-17 15:03:14 -07:00
oobabooga aa44e542cb Revert "Safer usage of mkdir across the project" 
This reverts commit 0d1597616f.
 2025-06-17 07:11:59 -07:00
oobabooga 0d1597616f Safer usage of mkdir across the project 2025-06-17 07:09:33 -07:00
oobabooga 66e991841a Fix the character pfp not appearing when switching from instruct to chat modes 2025-06-16 18:45:44 -07:00
oobabooga be3d371290 Close the big profile picture when switching to instruct mode 2025-06-16 18:42:17 -07:00
oobabooga 26eda537f0 Add auto-save for notebook textbox while typing 2025-06-16 17:48:23 -07:00
oobabooga 88c0204357 Disable start_with when generating the websearch query 2025-06-16 14:53:05 -07:00
oobabooga faae4dc1b0
Autosave generated text in the Notebook tab (#7079) 2025-06-16 17:36:05 -03:00
oobabooga de24b3bb31
Merge the Default and Notebook tabs into a single Notebook tab (#7078) 2025-06-16 13:19:29 -03:00
oobabooga cac225b589 Small style improvements 2025-06-16 07:26:39 -07:00
oobabooga 7ba3d4425f Remove the 'Send to negative prompt' button 2025-06-16 07:23:09 -07:00
oobabooga 34bf93ef47 Move 'Custom system message' to the Parameters tab 2025-06-16 07:22:14 -07:00
oobabooga c9c3b716fb Move character settings to a new 'Character' main tab 2025-06-16 07:21:25 -07:00
oobabooga f77f1504f5 Improve the style of the Character and User tabs 2025-06-16 06:12:37 -07:00
oobabooga bc2b0f54e9 Only save extensions settings on manual save 2025-06-15 15:53:16 -07:00
oobabooga 609c3ac893 Optimize the end of generation with llama.cpp 2025-06-15 08:03:27 -07:00
oobabooga db7d717df7 Remove images and links from websearch results 
This reduces noise a lot
 2025-06-14 20:00:25 -07:00
oobabooga e263dbf852 Improve user input truncation 2025-06-14 19:43:51 -07:00
oobabooga 09606a38d3 Truncate web search results to at most 8192 tokens 2025-06-14 19:37:32 -07:00
oobabooga 8e9c0287aa UI: Fix edge case where gpu-layers slider maximum is incorrectly limited 2025-06-14 10:12:11 -07:00
oobabooga d2da40b0e4 Remember the last selected chat for each mode/character 2025-06-14 08:25:00 -07:00
oobabooga 879fa3d8c4 Improve the wpp style & simplify the code 2025-06-14 07:14:22 -07:00
oobabooga 9a2353f97b Better log message when the user input gets truncated 2025-06-13 05:44:02 -07:00
Miriam f4f621b215
ensure estimated vram is updated when switching between different models (#7071) 2025-06-13 02:56:33 -03:00
oobabooga f337767f36 Add error handling for non-llama.cpp models in portable mode 2025-06-12 22:17:39 -07:00
oobabooga 2dee3a66ff Add an option to include/exclude attachments from previous messages in the chat prompt 2025-06-12 21:37:18 -07:00
oobabooga 004fd8316c Minor changes 2025-06-11 07:49:51 -07:00
oobabooga 570d5b8936 Only save extensions on manual save 2025-06-11 07:39:49 -07:00
oobabooga 27140f3563 Revert "Don't save active extensions through the UI" 
This reverts commit df98f4b331.
 2025-06-11 07:25:27 -07:00
LawnMauer bc921c66e5
Load js and css sources in UTF-8 (#7059) 2025-06-10 22:16:50 -03:00
oobabooga 75da90190f Fix character dropdown sometimes disappearing in the Parameters tab 2025-06-10 17:34:54 -07:00
oobabooga 1c1fd3be46 Remove some log messages 2025-06-10 14:29:28 -07:00
oobabooga 3f9eb3aad1 Fix the preset dropdown when the default preset file is not present 2025-06-10 14:22:37 -07:00
oobabooga 18bd78f1f0 Make the llama.cpp prompt processing messages shorter 2025-06-10 14:03:25 -07:00
oobabooga 889153952f Lint 2025-06-10 09:02:52 -07:00
oobabooga c92eba0b0a Reorganize the Parameters tab (left: preset parameters, right: everything else) 2025-06-09 22:05:20 -07:00
oobabooga efd9c9707b Fix random seeds being saved to settings.yaml 2025-06-09 20:57:25 -07:00
oobabooga df98f4b331 Don't save active extensions through the UI 
Prevents command-line activated extensions from becoming permanently active due to autosave.
 2025-06-09 20:28:16 -07:00
Mykeehu ec73121020
Fix continue/start reply with when using translation extensions (#6944) 
---------

Co-authored-by: oobabooga <oobabooga4@gmail.com>
 2025-06-10 00:17:05 -03:00
Miriam 1443612e72
check .attention.head_count if .attention.head_count_kv doesn't exist (#7048) 2025-06-09 23:22:01 -03:00
oobabooga 263b5d5557 Use html2text to extract the text of web searches without losing formatting 2025-06-09 17:55:26 -07:00
oobabooga f5a5d0c0cb Add the URL of web attachments to the prompt 2025-06-09 17:32:25 -07:00
oobabooga eefbf96f6a Don't save truncation_length to user_data/settings.yaml 2025-06-08 22:14:56 -07:00
oobabooga f9a007c6a8 Properly filter out failed web search downloads from attachments 2025-06-08 19:25:23 -07:00
oobabooga f3388c2ab4 Fix selecting next chat when deleting with active search 2025-06-08 18:53:04 -07:00
oobabooga 4a369e070a Add buttons for easily deleting past chats 2025-06-08 18:47:48 -07:00
oobabooga 0b8d2d65a2 Minor style improvement 2025-06-08 18:11:27 -07:00
oobabooga f81b1540ca Small style improvements 2025-06-08 15:19:25 -07:00
oobabooga eb0ab9db1d Fix light/dark theme persistence across page reloads 2025-06-08 15:04:05 -07:00
oobabooga 1f1435997a Don't show the new 'Restore character' button in the Chat tab 2025-06-08 09:37:54 -07:00