highperfocused/open-webui
1
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2025-03-27 02:02:31 +01:00
Code Issues Packages Projects Releases Wiki Activity

Core fix for num_predict not working.

Browse Source 
1) max_tokens was being looked for in openai_payload, but is present in openai_payload['options'], so is never found.
2) After copying the value for max_tokens to num_predict, delete max_tokens from the dictionary. This is to prevent Ollama throwing a warning about invalid option (max_tokens)
This commit is contained in:
ferret99gt 2025-02-19 08:52:34 -05:00
parent adde37394b
commit fea169a9c0
1 changed files with 5 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
backend/open_webui/utils/payload.py
View File

@ -178,10 +178,11 @@ def convert_payload_openai_to_ollama(openai_payload: dict) -> dict:
if openai_payload.get("options"):
ollama_payload["options"] = openai_payload["options"]
ollama_options = openai_payload["options"]
# Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
if "max_tokens" in openai_payload:
ollama_options["num_predict"] = openai_payload["max_tokens"]
# Re-Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
if "max_tokens" in ollama_options:
ollama_options["num_predict"] = ollama_options["max_tokens"]
del ollama_options["max_tokens"] # To prevent Ollama warning of invalid option provided
# Add options to payload if any have been set
if ollama_options: