Commit Graph
962 Commits
Author SHA1 Message Date
Charles YaoandClaude Opus 4.8 f623dc3fe1 feat(dify_builder): recover an interrupted working step via Retry/Start over
When a session crashes mid working-step (e.g. build.publish; the async advance
raised and released the lock), it was a locked dead-end: working states surface
no UI actions and the composer locked with no way forward.

- Surface a RecoveryRef offer (can_continue=Retry, can_restart=Start over,
  recovery_class="") on an interrupted working-state view -- built directly so a
  real drift class still wins.
- Allow recovery_continue / recovery_restart at an interrupted working state in
  _internal_action_allowed. recovery_continue returns the working state
  (non-settled) so the runner re-runs the stalled handler; recovery_restart
  resets working fields and re-enters the entry state -- both reuse existing
  recovery machinery, no new action kinds / no runner change.

Adds unit tests (offer surfaced, allowance) + regression tests (recovery_continue
re-runs handle_publish and advances; recovery_restart resets to entry state;
repeat-failure re-surfaces the offer with no wedge).

This reconciler was reviewed as a standalone plan on the pre-merge base; here it
is re-applied onto feat/dify-builder after PR #41677 merged in the reworked
service.py + _internal_action_allowed gate (which makes the allowance necessary).
The apply_repair idempotency + delete-guard commits (safe fix.apply Retry) land
alongside it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-04 00:07:14 -07:00
Charles YaoandClaude Opus 4.8 3dcb06db01 fix(dify-builder): exclude in-batch delete_node ids from apply_repair idempotency filter
apply_repair's already-present filter (added for interrupted-step Retry
idempotency) compared create_node/connect intents against before_graph
alone, with no awareness of delete_node intents in the same batch. A
from-scratch build sends delete_node(placeholder_start) + create_node(same
id) in one apply_repair call to replace the canvas's default placeholder
start with the generator's own start node -- the filter dropped the
create_node as "already present" while the delete_node still ran, deleting
the start node with no re-create, so the following connect raised
ValueError: node not found. Mirror handlers_build.py's existing M2 guard:
subtract in-batch delete_node targets from the present-node/edge sets
before filtering, so a delete+recreate of the same id in one batch survives.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-04 00:01:42 -07:00
Charles YaoandClaude Opus 4.8 61b35a36b7 fix(dify_builder): make apply_repair idempotent for already-present create/connect
Drops create_node intents whose id already exists and connect intents whose
edge already exists (referenced against the original draft graph), so a
fix.apply Retry re-applying the same intents is a clean no-op instead of a
duplicate/ValueError. graph_ops validation (raise on a genuinely fresh
duplicate id) is preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-04 00:01:42 -07:00
twwu 537d2a3178 Merge branch 'main' into feat/dify-builder 2026-09-04 09:44:01 +08:00
Byron.wangandautofix-ci[bot] 692010f0fc refactor(api): decouple console OAuth flow from legacy account services (#41188)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-03 08:47:14 +00:00
Byron.wang b015aa416f refactor(api): extract account login service and consolidate adapters (#41184) 2026-09-03 06:20:21 +00:00
非法操作 ba1d5ba172 refactor(api): centralize remote file handling (#41627) 2026-09-03 06:00:13 +00:00
zyssyz123 688107c3e5 fix(api): bound suggested question generation latency (#41705) 2026-09-03 05:02:09 +00:00
twwu a257ac4dee Merge branch 'main' into feat/dify-builder 2026-09-03 11:12:22 +08:00
Byron.wangandautofix-ci[bot] be7d11ca1d refactor(api): extract forgot password application service (#41111)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-03 01:38:59 +00:00
Charles YaoandClaude Opus 4.8 2b4df02305 fix(dify_builder): close SSE subscription even when the response is never iterated
progress_bus.subscribe() eagerly enters the pub/sub subscription (opening a
Redis connection + starting a listener thread) BEFORE the SSE generator
exists, so the subscribe completes before the Celery advance can publish.
But the only close() lived in the generator's finally -- which Python does
not run if the generator is never started. A streaming Response dropped
before its first frame (client aborted, or an after_request/teardown error)
therefore leaked the connection + thread until process restart.

stream_advance_frames now returns a small _ClosingFrameStream wrapper whose
close() closes the subscription directly (idempotent); the WSGI server calls
close() on the response iterable even on early disconnect, so cleanup fires
regardless of iteration. The inner generator keeps its own finally for the
iterated/disconnect paths, so this is strictly additive -- no happy-path
change. All seven call sites get it unchanged.

Adds a regression test (never-iterated stream -> close() -> subscription closed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 08:48:23 +08:00
Charles YaoandClaude Opus 4.8 de8a5745ea fix(dify_builder): always emit a terminal SSE frame so held version can't go stale
Two paths ended a streaming advance without a terminal `state`/`error`
frame, leaving the SSE client stalled to the deadline with a stale held
version -> its next action 409s:

1. advance_session published the terminal `state` frame outside the
   try/finally and unguarded. If the post-advance view projection
   (get_session_view -> read_graph) or the publish itself raised after a
   SUCCESSFUL advance, the exception escaped and no terminal frame was
   sent. Guard it and fall back to a terminal `error` frame.

2. submit_action_stream emitted a terminal `state` frame on settle only
   for `message`. `update_model` also settles synchronously while bumping
   the version, so it now emits one too -- otherwise the FE keeps the old
   version and 409s on its next action.

Adds regression tests for both (projection-failure -> error frame;
update_model -> state frame).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-03 08:48:23 +08:00
twwu 3eacff14a6 feat: add typed SSE interactions to Dify Builder 2026-09-03 08:48:23 +08:00
Charles YaoandClaude Opus 4.8 1eb5bb1dd1 fix(dify_builder): publish adapter unpacked publish_workflow's return as a tuple
WorkflowService.publish_workflow returns a single Workflow (-> Workflow), but
dify_port.publish still unpacked it as (workflow, retirement_candidates) -- a
stale signature. Every other caller was updated when the return type changed
(via the main merge); only the Dify Builder port was missed, so every publish
crashed with "TypeError: cannot unpack non-iterable Workflow object", surfaced
to the user as a generic "step failed" SSE frame.

Assign the single return value (retirement_candidates was unused anyway). The
existing test_dify_port publish test mocked publish_workflow as a 2-tuple, which
masked the bug -- corrected it to the real -> Workflow signature so it now guards
the regression.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-02 16:11:49 -07:00
Charles YaoandClaude Opus 4.8 e7e14ab888 fix(dify_builder): stop empty/mode-mismatched builds silently emptying the canvas
A build for a workflow app whose goal reads like a chatbot produced an empty
canvas while the assistant reported "Workflow built on the canvas.": the plan
was chatbot-shaped, build_nodes fed it to the generator (mode="workflow"), the
LLM emitted a chatflow graph (answer node / no end node), the generator
rejected it as MISSING_TERMINAL -- a class it does NOT retry -- and build_nodes
swallowed the error into []. handle_plan_approval then treated [] as a
successful build: it deleted the placeholder start and advanced to
build.execution, leaving nothing on the canvas.

Two parts:

1. build_nodes now prepends an explicit workflow-topology directive (one
   start + >=1 end node, no answer nodes) to the generator instruction, and
   retries ONCE on an error/empty result with the specific error fed back as
   a corrective nudge -- covering the terminal-node failure the shared
   generator skips.

2. handle_plan_approval refuses to fake success: when build_nodes yields zero
   create_node intents it surfaces an honest error and stays in
   build.plan_approval (retryable) without deleting the placeholder, instead
   of emptying the canvas and claiming a build. A loop-back re-approve still
   returns the fixed create intents (only filtered as already-present), so
   zero create intents unambiguously means a genuine generation failure.

Adds regression tests for both. Note: part 1 improves LLM reliability
(verifiable live); part 2 is the deterministic guarantee against the
false-success empty canvas.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-02 14:39:56 -07:00
YungLeandautofix-ci[bot] 3cd1fc098c feat: allow knowledge base API keys to be scoped to a single dataset (#37569)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-02 10:30:42 +00:00
非法操作 59ee8878f9 refactor(api): require active workspace in request context (#41280) 2026-09-02 06:54:47 +00:00
Wu Tianweiandfatelei 82d86dcf98 feat: app access point permission (#41631)
Co-authored-by: fatelei <fatelei@gmail.com>
2026-09-02 06:34:33 +00:00
Byron.wangandautofix-ci[bot] c4ea406586 refactor(api): decouple system features and web adapters (#40772)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-01 11:50:52 +00:00
twwu 3152ba3faa Merge branch 'main' into feat/dify-builder 2026-09-01 17:09:54 +08:00
Charles YaoandClaude Opus 4.8 52d241d754 feat(dify-builder): streaming create_fix/build/edit_session with subscribe-before-dispatch
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-31 22:18:42 -07:00
Charles YaoandClaude Opus 4.8 6888282e95 feat(dify-builder): subscribe-before-dispatch streaming submit_action/message
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-31 22:18:42 -07:00
Charles YaoandClaude Opus 4.8 fa4921eff0 feat(dify-builder): stream_advance_frames terminal-aware SSE generator
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-31 22:18:42 -07:00
Charles YaoandClaude Opus 4.8 390eea61fb feat(dify-builder): terminal state frame carries the full SessionView
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-31 22:18:42 -07:00
twwu 22be55af5f feat: redesign app builder fix flow 2026-08-31 18:18:30 +08:00
zyssyz123andautofix-ci[bot] 762019a160 fix(workflow): block manual runs for trigger workflows (#41519)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-31 09:30:52 +00:00
Asuka Minato 1952cff091 test: migrate model and agent sessions to SQLite (#40087) 2026-08-31 05:57:40 +00:00
Asuka Minato f8f71fdc68 test: migrate audit findings sessions and ORM models to SQLite (#40551) 2026-08-31 05:55:38 +00:00
twwu cb53f02905 Merge branch 'main' into feat/dify-builder 2026-08-31 13:31:50 +08:00
d25e2b51fc refactor(api): decouple onboarding and notification services (#40759)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: hjlarry <hjlarry@163.com>
2026-08-31 04:37:07 +00:00
Wu Tianwei 7167d7d662 fix(rbac): grant agent access to all legacy roles (#41511) 2026-08-31 03:33:09 +00:00
Byron.wangandhjlarry 9e07c2dac6 refactor(api): extract email registration application service (#41108)
Co-authored-by: hjlarry <hjlarry@163.com>
2026-08-31 03:22:01 +00:00
Xiyuan Chen 750810ea81 fix(api): keep workspace owner consistent on ownership transfer (#41267) 2026-08-31 02:51:19 +00:00
Charles YaoandClaude Opus 4.8 ba7692e2e9 feat(dify-builder): log build_nodes empty/failed generation and starved resource sources
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-30 10:03:48 -07:00
Charles YaoandClaude Opus 4.8 6f27a709ba feat(dify-builder): log llm-agent model-resolution failure once per advance
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-30 10:03:48 -07:00
Charles YaoandClaude Opus 4.8 133e8fdc0c refactor(dify-builder): remove unused propose_build_repair from agent Protocol
It was stubbed to [] in LlmBuilderAgent, canned in PlaceholderAgent, and never
called by any handler — the Build failure path reuses Fix cognition (diagnose +
propose_repair). Removed from the DifyBuilderAgent Protocol, both implementations,
and all test doubles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-30 10:03:48 -07:00
Charles YaoandClaude Opus 4.8 86341638df fix(dify-builder): keep generator node ids in graph_translate (no start rename)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-30 10:03:47 -07:00
twwu 433046e0de Merge branch 'main' into feat/dify-builder 2026-08-30 14:25:22 +08:00
Charles YaoandClaude Opus 4.8 bbbf2ff457 feat(dify-builder): real schema-shaped generate_mock_inputs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 18:01:16 -07:00
Charles YaoandClaude Opus 4.8 cd5fb6bbf0 fix(dify-builder): ground model on all model-bearing nodes, carry branch handles, clean test lint
Final consolidated fix wave from whole-branch review of the Real Build +
Edit Cognition slice:

- _ground now overwrites the model dict on ANY create_node whose config
  carries a model dict (llm, question-classifier, parameter-extractor),
  not just "llm", so a drifted/hallucinated provider/name on those other
  node types no longer flows through ungrounded.
- to_intents now carries a generator edge's sourceHandle/targetHandle
  into the connect intent's args (source_handle/target_handle) so
  branching nodes (if-else, question-classifier) wire onto the correct
  port instead of collapsing onto the default handle; edges without
  handles still produce bare {from_node, to_node} args.
- Cleared the 17 ruff errors (PT018/E302/E306/E501) across the slice's
  test files by splitting composite asserts, adding missing blank
  lines, and wrapping long lines -- no assertions weakened, no noqa.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 14:22:19 -07:00
Charles YaoandClaude Opus 4.8 27be3de30a feat(dify-builder): wire real Edit cognition into LlmBuilderAgent
analyze_impact/propose_edit_plan/build_edit_intents now call
services.dify_builder.agent.edit.* via _model_or_none() instead of
delegating to the canned PlaceholderAgent. 9 of the Protocol's 14
Build/Edit/Fix methods now use real LLM cognition; only
generate_mock_inputs and propose_build_repair (returns []) remain
deferred pending the live-test step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 14:00:58 -07:00
Charles YaoandClaude Opus 4.8 f801481fdc fix(dify-builder): guard analyze_impact target_node_ids; drop ARG noqa in edit tests
analyze_impact could raise on a non-list target_node_ids from the LLM (no
isinstance guard, unlike fields/values) -- violated the never-crash
contract. Guard it the same way. Also collapse the fake models' invoke_llm
signatures to **_kwargs (underscore-prefixed, no noqa) per the ARG lint
constraint, and pin two more malformed-shape degrade paths
(build_edit_intents non-list intents, propose_edit_plan non-list plan).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 13:54:08 -07:00
Charles YaoandClaude Opus 4.8 f5458e16b3 feat(dify-builder): real Edit cognition (impact/plan/intents via Fix pattern)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 13:41:59 -07:00
Charles YaoandClaude Opus 4.8 cbb1487d6a fix(dify-builder): sync cards manifest to attached cards, cover remaining Build wirings
handle_test_and_repair's AssistantTurnItem.cards was unconditional even
though error/change_set cards are now only appended when
propose_build_repair returns intents -- the manifest claimed cards that
were never attached in the neutral (empty-repair) path. Make cards
conditional to match what's actually appended, and assert it in both
branches.

Remove three noqa comments added in the prior commit that suppressed
ARG001/ARG005 on new test code (the global constraint disallows noqa
for non-enabled rules); switch the genuinely-unused monkeypatch
args/lambda params to underscore-prefixed names instead, and drop a
build_nodes monkeypatch that exercised nothing. Add
test_remaining_build_methods_thread_model_and_tenant_args to cover
propose_plan_v1/discover_resources/bind_resources/build_nodes/
learn_from_build's model/tenant_id/model_config threading, matching
the existing analyze_goal coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 13:14:08 -07:00
Charles YaoandClaude Opus 4.8 8683e6517c feat(dify-builder): wire real Build cognition + neutral empty test-repair
LlmBuilderAgent's 6 Build methods (analyze_goal, propose_plan_v1,
discover_resources, bind_resources, build_nodes, learn_from_build) now
delegate to services.dify_builder.agent.build instead of the canned
PlaceholderAgent. propose_build_repair returns [] -- there's no real
failure signal to diagnose until the live test step is de-canned.

handle_test_and_repair now conditions the error card + apply_repair on
a non-empty propose_build_repair result: placeholder mode keeps today's
"found and fixed" arc, while real cognition (which never reports a
failure yet) emits a neutral "no issues found" review summary instead
of a fake error/fix on a hardcoded node id that real build_nodes
wouldn't have generated anyway.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 12:25:14 -07:00
Charles YaoandClaude Opus 4.8 02edd73204 fix(dify-builder): stop fabricating llm completion_params, dedupe dataset_ids grounding
Grounding only overwrites provider+name on an llm node's model dict now,
preserving whatever mode/completion_params the generator produced instead
of synthesizing a temperature when none was present. knowledge-retrieval
grounding now assigns each node its own dataset_ids list copy instead of
aliasing one shared list across every KB node in the graph.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 12:15:08 -07:00
Charles YaoandClaude Opus 4.8 dc85ea51a2 feat(dify-builder): build_nodes via WorkflowGenerator with model+dataset grounding
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 12:04:58 -07:00
Charles YaoandClaude Opus 4.8 0ba1f932a9 fix(dify-builder): address review findings for build.py cognition
- Fix S110: replace bare except-pass with explicit degrade behavior
- Fix E501: wrap long function signatures to <=120 chars
- Fix I001: sort imports in test file (resources to top)
- Fix PT018: split composite assertions into separate statements
- Add _BoomInstance for provider-error testing
- Add 5 degrade tests (analyze_goal, propose_plan_v1 x2, discover_resources, learn_from_build)
- Remove unused _KIND_BY_PREFIX dict
- Add noqa comments for test mock arguments

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 11:50:40 -07:00
Charles YaoandClaude Opus 4.8 2e77396d39 feat(dify-builder): real Build light cognition (analyze/plan/discover/bind/learn)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 11:40:05 -07:00
Charles YaoandClaude Opus 4.8 bcd0f59100 fix(dify-builder): prevent duplicate start node ids on multiple starts
Only map the first start-type node to canonical 'start'; additional starts
retain their original ids to prevent collisions and ambiguous intents.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-29 11:36:02 -07:00