fix(llm): honor GRAPHIFY_MAX_OUTPUT_TOKENS for OpenAI-compatible backends (#973)
Backends routed through _call_openai_compat (gemini, openai, kimi,
deepseek, ollama) silently ignored the documented env override when
their backend config dict carried a hardcoded max_completion_tokens.
The dispatcher used:
cfg.get("max_completion_tokens", max_out)
which always returned the config-dict value when the key was present,
shadowing the env-var-resolved max_out.
For gemini specifically, the hardcoded cap of 16384 truncated
extracted-graph JSON mid-response on multi-document chunks (~17 specs
of 100-1500 lines each pushing the output past 16k tokens). Symptom:
cascading 'LLM returned invalid JSON, skipping chunk: Unterminated
string at column 4XXXX' followed by bisect-retry storms that bill
input tokens without producing graph nodes.
Fix: route the same _resolve_max_tokens(...) call that the Claude and
Bedrock paths already use, so the override applies uniformly across
backends.
Verified with gemini-2.5-pro over a 20-doc / 76k-input-token chunk:
output of 36008 tokens emitted without truncation, producing 193
nodes / 223 edges / 23 communities in a single chunk.
This commit is contained in:
+1
-1
@@ -582,7 +582,7 @@ def extract_files_direct(
|
||||
user_msg,
|
||||
temperature=cfg.get("temperature", 0),
|
||||
reasoning_effort=cfg.get("reasoning_effort"),
|
||||
max_completion_tokens=cfg.get("max_completion_tokens", max_out),
|
||||
max_completion_tokens=_resolve_max_tokens(cfg.get("max_completion_tokens", 8192)),
|
||||
backend=backend,
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user