Integrates the FE team's live-progress + conversation-streaming rework
(durable progress events, repository layer, snapshot-consistent reads,
idempotent message retries, EDIT_PUBLISH->EDIT_COMPLETE) and reconciles its
interrupted-step recovery model with the reconciler already on
feat/dify-builder.
Combined recovery model:
- interrupted (crash mid working-step, lock released, nothing durably
recorded) -> Retry (recovery_continue, re-runs the handler) + Start over
- FAILED (handler raised, recorded by runner.fail) -> Restart only
- a Retry that itself raises lands in FAILED -> Restart (no wedge)
Kept from the PR: EDIT_PUBLISH working / EDIT_COMPLETE terminal, snapshot
refresh in apply_recovery_action, paused reset. Kept from the reconciler:
Retry on interrupted, state-aware access tier (EDIT_PUBLISH added to the
publish tier), apply_repair idempotency filter.
680 dify_builder unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A recovery_continue (Retry) re-runs the interrupted working handler, which
re-invokes the state's privileged op -- publish (RELEASE) at build.publish/
fix.publish, a draft test run (TEST_AND_RUN) at the test/verify states. Before
this, recovery_continue was classified as base EDIT, so a Retry re-ran the
privileged op below its own authorization tier without re-checking. Make
_app_access_for_action state-aware for recovery_continue so it enforces the
same tier the state's normal action requires. recovery_restart only resets +
re-enters the entry state, so it stays EDIT.
Also: an interrupted working step now always surfaces the Retry/Start-over
offer, overriding a stale drift recovery_class a user may have left set at an
earlier waiting gate (drift recovery only applies at a waiting gate).
Addresses the two findings from the final reconciliation review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a session crashes mid working-step (e.g. build.publish; the async advance
raised and released the lock), it was a locked dead-end: working states surface
no UI actions and the composer locked with no way forward.
- Surface a RecoveryRef offer (can_continue=Retry, can_restart=Start over,
recovery_class="") on an interrupted working-state view -- built directly so a
real drift class still wins.
- Allow recovery_continue / recovery_restart at an interrupted working state in
_internal_action_allowed. recovery_continue returns the working state
(non-settled) so the runner re-runs the stalled handler; recovery_restart
resets working fields and re-enters the entry state -- both reuse existing
recovery machinery, no new action kinds / no runner change.
Adds unit tests (offer surfaced, allowance) + regression tests (recovery_continue
re-runs handle_publish and advances; recovery_restart resets to entry state;
repeat-failure re-surfaces the offer with no wedge).
This reconciler was reviewed as a standalone plan on the pre-merge base; here it
is re-applied onto feat/dify-builder after PR #41677 merged in the reworked
service.py + _internal_action_allowed gate (which makes the allowance necessary).
The apply_repair idempotency + delete-guard commits (safe fix.apply Retry) land
alongside it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
apply_repair's already-present filter (added for interrupted-step Retry
idempotency) compared create_node/connect intents against before_graph
alone, with no awareness of delete_node intents in the same batch. A
from-scratch build sends delete_node(placeholder_start) + create_node(same
id) in one apply_repair call to replace the canvas's default placeholder
start with the generator's own start node -- the filter dropped the
create_node as "already present" while the delete_node still ran, deleting
the start node with no re-create, so the following connect raised
ValueError: node not found. Mirror handlers_build.py's existing M2 guard:
subtract in-batch delete_node targets from the present-node/edge sets
before filtering, so a delete+recreate of the same id in one batch survives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drops create_node intents whose id already exists and connect intents whose
edge already exists (referenced against the original draft graph), so a
fix.apply Retry re-applying the same intents is a clean no-op instead of a
duplicate/ValueError. graph_ops validation (raise on a genuinely fresh
duplicate id) is preserved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
progress_bus.subscribe() eagerly enters the pub/sub subscription (opening a
Redis connection + starting a listener thread) BEFORE the SSE generator
exists, so the subscribe completes before the Celery advance can publish.
But the only close() lived in the generator's finally -- which Python does
not run if the generator is never started. A streaming Response dropped
before its first frame (client aborted, or an after_request/teardown error)
therefore leaked the connection + thread until process restart.
stream_advance_frames now returns a small _ClosingFrameStream wrapper whose
close() closes the subscription directly (idempotent); the WSGI server calls
close() on the response iterable even on early disconnect, so cleanup fires
regardless of iteration. The inner generator keeps its own finally for the
iterated/disconnect paths, so this is strictly additive -- no happy-path
change. All seven call sites get it unchanged.
Adds a regression test (never-iterated stream -> close() -> subscription closed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two paths ended a streaming advance without a terminal `state`/`error`
frame, leaving the SSE client stalled to the deadline with a stale held
version -> its next action 409s:
1. advance_session published the terminal `state` frame outside the
try/finally and unguarded. If the post-advance view projection
(get_session_view -> read_graph) or the publish itself raised after a
SUCCESSFUL advance, the exception escaped and no terminal frame was
sent. Guard it and fall back to a terminal `error` frame.
2. submit_action_stream emitted a terminal `state` frame on settle only
for `message`. `update_model` also settles synchronously while bumping
the version, so it now emits one too -- otherwise the FE keeps the old
version and 409s on its next action.
Adds regression tests for both (projection-failure -> error frame;
update_model -> state frame).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This branch removed the Dify Builder contract generator (contract_gen.py
and the gen-dify-builder-contract npm script), but autofix.yml still
invoked that script. On every api/** change the "Generate frontend
contracts" step failed (command not found under bash -e), turning the
job red and skipping the autofix-ci commit step -- so even the good
gen-api-contract output never landed. Remove the stale invocation and
the now-redundant contract_gen.py path-filter entries (already covered
by api/**).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>