Streaming introduced a correctness regression: if the event stream ended
without a terminal frame (idle timeout, dead publishing worker), run_draft
synthesised status="failed" with an empty dify_run_id. A run that had in fact
SUCCEEDED would then be reported as a failed build and sent into the repair
loop. The blocking path would have waited and returned the truth.
The stream is the fast path, not the authority. Every non-ping frame is
stamped with workflow_run_id by convert_stream_full_response, so the run id
is known from the first frame onwards -- long before the terminal frame that
may never arrive. run_draft now captures it from any frame and, when no
terminal frame arrives, reads the run's real status from the workflow-run row
instead of inventing one.
Note that data["id"] is the node-execution id on node frames, not the run id,
so it is only read as a run id on workflow-level frames.
When even the row cannot settle it -- no run id, no row, or a row still
marked running because the worker died mid-run -- the result is reported as
status="running" with an explicit Run.error, which is what Run.status already
documents for "not over / not known". A truncated stream is now either the
truth or a visible unknown, never a fabricated failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
run_draft blocked on AppGenerateService.generate(streaming=False) and then
replayed every finished node-execution row through on_event, so the user
watched a spinner for the whole run and then saw every node light up at once.
It now consumes the streaming generator and fires on_event per frame.
Two things the plan assumed that turned out not to hold:
- The stream does NOT yield dicts. convert_stream_full_response yields dicts,
but that runs in the Celery worker; on our side
BaseAppGenerator.convert_to_event_stream (not overridden) re-wraps every
event as "data: {json}\n\n" and every keep-alive as "event: ping\n\n". An
isinstance(chunk, dict) guard would have dropped every frame. New
stream_chunk_as_mapping accepts both shapes and rejects keep-alives.
- The streaming workflow_paused frame carries workflow_run_id where the
blocking response carried id, so map_run_result would KeyError on it.
Adapted at the call site via run_result_data_from_terminal_chunk;
map_run_result's contract is unchanged for the fix/edit flows.
Session lifetime: _dispatch_generate is a plain function, so all of its
Session work is eager, and the generator it returns only reads a Redis
subscription while the run executes in a Celery worker with its own session.
The consumption loop therefore sits outside the `with` block rather than
pinning a DB connection for the length of a workflow run; a test pins this.
The node-execution read stays at the end -- map_run_result still needs those
rows for per_node[].outputs. Only the progress streams.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Charles YaoandClaude Opus 5