Commit Graph
1676 Commits
Author SHA1 Message Date
Charles YaoandClaude Opus 5 a3c2e7fca5 feat(dify-builder): tell the canvas which run to show
Streaming the test run gives the user a live progress bar but no results:
NodeEvent carries node_id/title/status/error and nothing else, so a client
watching the stream learns which node is running, never what it produced.
The results already exist -- the run is a normal InvokeFrom.DEBUGGER draft
run, its node executions are in the database, and the editor's canvas
already knows how to render exactly that view. It was simply never told
which run to open.

So put the Dify run id where a client can reach it:

- NodeEvent gains run_id, filled from the streaming frame. Every non-ping
  frame is stamped with workflow_run_id, so this arrives with the FIRST
  node event -- while the run is still going, which is the point.
- The node SSE frame publishes it alongside the existing fields.
- mark_test_success / mark_test_error carry dify_run_id in both the Build
  and Edit flows, so the terminal state names its run too.

This is what makes per-node results land on the graph, where a node's
inputs and outputs already have a home, instead of only as a JSON blob on
a conversation card.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 17:16:41 -07:00
Charles YaoandClaude Opus 5 69ccf2ad15 fix(dify-builder): wire resource readiness into recommender/gap prompts
Three minor whole-branch-review fixups:

- ResourceOption.readiness ("ready"/"missing_config") was computed but never
  shown to the model: discover_resources's recommender prompt listed ids and
  labels only, so it could recommend an unauthorized tool; _GAP_SYSTEM showed
  readiness in the listing (assess_capability_gap) but never defined what
  missing_config means. Both prompts now show readiness and state that
  missing_config is installed-but-not-authorized, i.e. not usable -- pinned
  with tests asserting the literal "missing_config" in the constructed
  prompts.
- strings.py: dropped three orphaned catalogue entries ("Here's the initial
  plan.", "Ask each time", "Prefer audited") -- the conflict-policy concept
  they backed was removed end to end; nothing in the handlers emits them.
- core/workflow/generator/tool_catalogue.py: provider.need_credentials was
  read OUTSIDE the per-provider try/except that already guards
  get_tools()/tool parsing. This module is shared with the cmd+K /create and
  /refine flows, so a provider whose property raised would break those too,
  not just the Builder. Moved the read inside the try/except so a bad
  provider is skipped like any other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 16:41:52 -07:00
Charles YaoandClaude Opus 5 f1bf2fb318 feat(dify-builder): say when the workspace cannot cover the plan
Add assess_capability_gap: one plain sentence naming a plan step nothing
installed can perform, or "" when the plan is covered. Grounded only in
installed resources -- there is no marketplace lookup, so it names a missing
CAPABILITY, never a product to install. A missing_config tool (installed but
unauthorized, from the prior task's real readiness) doesn't count as
covering a step either, since the model is shown each option's readiness
alongside its label and kind.

Wired through the full agent seam: the DifyBuilderAgent protocol, the real
LlmBuilderAgent (delegates to build.assess_capability_gap), and the
PlaceholderAgent test double (returns ""). handlers_build.py calls it
after discover_resources and, when non-empty, appends a NoticeItem beside
the existing empty-resources notice -- the two are independent and can
both fire.

Like its sibling cognition functions, it degrades to "" (never raises) on
no model, an LLM error, or an unparseable reply: a gap that can't be
assessed isn't a gap worth inventing. The sentence is LLM-generated prose
already in the user's language (json_language_instruction), not a
catalogue entry, so no static string was added.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 16:13:02 -07:00
Charles YaoandClaude Opus 5 0a228b7c23 feat(dify-builder): report whether a recommended tool is actually authorized
The Builder recommended every installed tool as ready to use without
ever checking whether its provider actually has credentials configured.
A credentialed tool with no tool_builtin_providers row got bound into a
plan and then failed at test time with an authorization error the user
can't act on.

tool_catalogue.py now carries provider_controller.need_credentials onto
each ToolCatalogueEntry as needs_credentials. resources.py resolves
which providers are actually authorized for the tenant (mirroring
ToolTransformService.builtin_provider_to_user_provider's need_credentials
/ db_provider rule exactly, so the Builder and the console's tools page
can never disagree) and sets ResourceRef.readiness to "missing_config"
for a credentialed tool with no authorized provider row, "ready"
otherwise. Datasets and models are unaffected. If the authorization
query itself fails, readiness fails open to "ready" rather than
mass-labeling every credentialed tool unauthorized on a transient error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 16:04:48 -07:00
Charles YaoandClaude Opus 5 574c9d1208 fix(dify-builder): never report a truncated test run as a failed one
Streaming introduced a correctness regression: if the event stream ended
without a terminal frame (idle timeout, dead publishing worker), run_draft
synthesised status="failed" with an empty dify_run_id. A run that had in fact
SUCCEEDED would then be reported as a failed build and sent into the repair
loop. The blocking path would have waited and returned the truth.

The stream is the fast path, not the authority. Every non-ping frame is
stamped with workflow_run_id by convert_stream_full_response, so the run id
is known from the first frame onwards -- long before the terminal frame that
may never arrive. run_draft now captures it from any frame and, when no
terminal frame arrives, reads the run's real status from the workflow-run row
instead of inventing one.

Note that data["id"] is the node-execution id on node frames, not the run id,
so it is only read as a run id on workflow-level frames.

When even the row cannot settle it -- no run id, no row, or a row still
marked running because the worker died mid-run -- the result is reported as
status="running" with an explicit Run.error, which is what Run.status already
documents for "not over / not known". A truncated stream is now either the
truth or a visible unknown, never a fabricated failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 15:49:50 -07:00
Charles YaoandClaude Opus 5 e9599355a2 feat(dify-builder): stream the test run instead of replaying it
run_draft blocked on AppGenerateService.generate(streaming=False) and then
replayed every finished node-execution row through on_event, so the user
watched a spinner for the whole run and then saw every node light up at once.
It now consumes the streaming generator and fires on_event per frame.

Two things the plan assumed that turned out not to hold:

- The stream does NOT yield dicts. convert_stream_full_response yields dicts,
  but that runs in the Celery worker; on our side
  BaseAppGenerator.convert_to_event_stream (not overridden) re-wraps every
  event as "data: {json}\n\n" and every keep-alive as "event: ping\n\n". An
  isinstance(chunk, dict) guard would have dropped every frame. New
  stream_chunk_as_mapping accepts both shapes and rejects keep-alives.

- The streaming workflow_paused frame carries workflow_run_id where the
  blocking response carried id, so map_run_result would KeyError on it.
  Adapted at the call site via run_result_data_from_terminal_chunk;
  map_run_result's contract is unchanged for the fix/edit flows.

Session lifetime: _dispatch_generate is a plain function, so all of its
Session work is eager, and the generator it returns only reads a Redis
subscription while the run executes in a Celery worker with its own session.
The consumption loop therefore sits outside the `with` block rather than
pinning a DB connection for the length of a workflow run; a test pins this.

The node-execution read stays at the end -- map_run_result still needs those
rows for per_node[].outputs. Only the progress streams.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 15:42:25 -07:00
Charles YaoandClaude Opus 5 a96ed2d387 Builder: fix the continue_adjusting dead end at build.initial_plan
Code review finding: build.initial_plan was left in _WAITING with zero
_ACTIONS_FOR entries but a handler still gated on find_resources. Since a
waiting state settles the runner's advance loop (is_waiting(next_state) ->
settled), continue_adjusting from build.review landed a session there with
no button the FE could render and no action the handler would accept --
the loop-back could never advance.

Reclassify build.initial_plan as working (state.py _WORKING, not _WAITING):
it is now a genuine pass-through step, matching build.test_and_repair /
build.publish / build.governance_feedback. handle_initial_plan drops its
find_resources gate and runs resource discovery unconditionally on entry;
the runner drives it with the action already consumed and falls straight
through to build.resource_recommendation within the same advance() call.
Removed the now-stale BUILD_INITIAL_PLAN entry from service.py's
_BACKEND_ACTIONS_FOR (no action to gate on any more) and the compensating
exemption it required in test_every_backend_waiting_action_has_a_ui_path
(the invariant now holds with no special case).

Added a state-classification test (test_build_initial_plan_is_working_not_
waiting) and an end-to-end regression driving the Runner from build.review
through continue_adjusting with no find_resources action ever sent
(test_continue_adjusting_reaches_resources_without_find_resources_action),
which is the scenario this fix exists to unwedge. Fixed the two existing
idempotency flow tests that asserted the old (broken) two-step loop-back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 14:50:45 -07:00
Charles YaoandClaude Opus 5 806cc4e517 Builder: collapse the decision-free find_resources gate at build.initial_plan
submit_requirements now tail-calls a shared _discover_and_offer_resources
helper straight into build.resource_recommendation, instead of stopping at
build.initial_plan to show a plan card the user must click through with no
real decision to make. The resource-discovery body (progress reporting,
discover_resources, ResourceSelectCard, empty-options notice, assistant
turn) is extracted verbatim into that helper so the straight-through path
and the continue_adjusting/retry_after_revert loop-back (which still
re-enters at build.initial_plan and still requires find_resources) share
one implementation instead of drifting apart.

build.initial_plan survives as the loop-back target but now offers no UI
action (_ACTIONS_FOR entry removed); its handler still accepts
find_resources at the backend for that loop-back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 14:37:01 -07:00
Charles YaoandClaude Opus 5 e8b14b50ff refactor(dify-builder): drop the resource conflict policy
The resource-recommendation step made the user pick a "conflict
policy" that never decided anything. Remove it end to end: the
ConflictPolicyOption card option, ResourceSelectCard's
conflict_policy_options field, the third bind_resources() parameter
across the port/placeholder/LLM-agent implementations, and the dead
conflict_policy field on the (unused) ResourcesPayload contract type.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 14:10:34 -07:00
twwu 3686aa891b Merge branch 'main' into feat/dify-builder 2026-09-20 16:12:20 +08:00
Xiyuan Chen a24135ae83 fix(account): make case-fallback email lookup work in both directions (#42541) 2026-09-20 02:01:55 +00:00
twwu cce9470161 Merge branch 'main' into feat/dify-builder 2026-09-20 09:45:05 +08:00
Ingram Z fae098c6c0 fix(retention): harden V2 archive deletion recovery (#41358) 2026-09-19 07:54:35 +00:00
dbeebfc15d refactor: reduce ambiguity in mcp queries (#42395)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-09-18 09:17:36 +00:00
林玮 (Jade Lin)andJoel f495581a05 fix: preserve default model display after plugin removal (#42498)
Co-authored-by: Joel <iamjoel007@gmail.com>
2026-09-18 09:17:35 +00:00
林玮 (Jade Lin)andJoel db6a4f5cd3 feat(agent): support exporting published versions (#42485)
Co-authored-by: Joel <iamjoel007@gmail.com>
2026-09-18 08:59:54 +00:00
Xiyuan Chenandautofix-ci[bot] bc0636c4db refactor(openapi): route /openapi/v1 auth by subject (#41420)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-18 08:42:53 +00:00
twwu bb7f7e4640 Merge branch 'main' into feat/dify-builder 2026-09-18 16:06:20 +08:00
Xiyuan Chen 3d436a7e31 refactor(openapi): delete the dead bearer→handler auth adapter (#41311) 2026-09-18 06:27:49 +00:00
Mesrop Harutyunyanandautofix-ci[bot] 104f3b239b refactor(api): merge duplicated model builders in unit tests (#41549)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-18 05:10:25 +00:00
william 26831445d0 refactor(api): update tag name with a single UPDATE statement (#42449) 2026-09-18 03:22:44 +00:00
Charles YaoandClaude Opus 5 d3087cbe34 fix(dify-builder): stop recommending the whole workspace when nothing matches
discover_resources initialised its selection to the entire tenant inventory and
only replaced it when the model returned a non-empty pick, so three paths -- no
model, a model that picked nothing, a failed call -- recommended everything. On
a workspace with 57 resources that is not merely noisy: the client pre-checks
every recommendation and submits them all by default, and bind_resources then
appends all 57 labels to the last plan item, which is the text fed to the graph
generator.

Recommending nothing is the honest answer when there is no signal. The card is
still emitted so the confirm gate keeps its active interaction and a later pass
cannot inherit a stale one, and a notice says why rather than leaving a blank
card. The conflict-policy control no longer renders over an empty list.

The test that should have caught this used a one-resource catalog, where "all"
and "none" are indistinguishable; it now uses 57.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:09:18 -07:00
Charles YaoandClaude Opus 5 d20286482f fix(dify-builder): let a testdata submit through a draft that moved while typing
The testdata gate is the one place a human types for tens of seconds, and it
opens right after Builder has written the canvas -- long enough for the editor's
own autosave to move the app revision underneath them. Both revision gates then
refused the submit, and because a conflict leaves the session with only
check_recovery as an action, active_interaction drops to None and the client
freezes every input: the form stayed on screen but could no longer be typed into.

A testdata submit carries run inputs, not graph edits, so neither gate applies
to it. Exempting only the draft-changed check would not have been enough --
the stale-app-revision check above it fires first, on the base_app_revision the
client has held since the form was issued.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 20:09:18 -07:00
twwu f0c2c644d2 Merge branch 'main' into feat/dify-builder 2026-09-18 10:24:19 +08:00
Asuka Minato a504dfee27 ci: Update pyrefly dependency version to 1.3.0 (#42445) 2026-09-18 02:16:47 +00:00
Charles YaoandClaude Opus 5 65a47018b4 feat(dify-builder): say the app's name once when the build lands
Spec N4 has the build-complete card headline read "<App name> is ready", with
the annotation 名字只在回复句里念一次;记录卡上不出现 -- said once, in that
sentence, and never repeated on the completion receipt. We said it nowhere: the
card was titled "Build plan" and the engine had no app name to say.

DifyBuilderContext now carries app_name, seeded at session creation through a
get_app_name_fn seam alongside the existing revision/authorize ones. Env.rename_app
returns the name the database actually took rather than None, so the refine
records what landed instead of what it proposed -- the service normalizes after
the engine hands off, and the two can differ.

The headline is interpolated, so it gets a strings.py template or it would
survive localization untranslated. That pattern matches almost anything ending
in "is ready", so it sits last and every more specific template gets first
refusal.

The receipt already omitted the name; there is now a test pinning that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 18:09:23 -07:00
Charles YaoandClaude Opus 5 aa2b4fe4ab feat(dify-builder): name apps from their prompt, and turn gate actions into card options
Two App Builder spec changes that share the Builder session contract.

Naming (spec N1-N3): an app created from the homepage prompt is no longer named
by the user. core/dify_builder/naming.py cuts the prompt's opening clause at
create time -- instant and infallible, since it runs inside POST /apps -- and
CreateAppPayload.name becomes optional alongside a new prompt field. The Builder
then refines that name once during goal analysis, where a model and the detected
reply language are already in hand, so the judgement calls that are
English-shaped in pure Python (dropping an imperative lead-in, spotting a
contentless prompt) work in any language. Plan approval freezes the name.

Card options: SessionView gains a Decision carrying the gate's choice as options
inside the card plus a fixed confirm/cancel pair, replacing a button row whose
shape changed at every state. Option ids are the old action ids untouched, so
resolve_action_kind and the per-state legality check are unchanged, and
SessionView.actions keeps being populated -- the deployed web client goes on
working until it reads decision instead.

The regenerated console OpenAPI doc also picks up ErrorCard.diagnostics, missed
when cc9ca8b1dd landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:55:38 -07:00
Charles YaoandClaude Opus 5 cc9ca8b1dd feat(dify-builder): carry generation diagnostics into the exported debug log
api pods lose their logs on restart, so a failed build's real cause
("Workflow generator: structural validation failed: ...") could only be read
from a pod log that may already be gone. Session 01a08fd0 hit exactly that.

Carry the generator's own diagnostics back with the result instead:
BuildNodesResult and ErrorCard gain a ``diagnostics`` list, and build_nodes
records one entry per failed generation attempt -- initial AND the corrective
retry, so an error that changes between attempts stays visible -- plus the
all-rejected and exception paths.

Each entry holds a server-side UTC ``at`` (so the surviving pod-log window can
be searched around it), ``source``, ``attempt``, the generator's ``message``
(the same text the server logs), its ``codes``, and the full structured
``errors`` including ``node_id``. That is strictly more than the log line
carries: for an UNRESOLVED_REFERENCE the export now names the offending node.

The list rides in the ErrorCard's item payload, which is what the streamed
conversation item -- and therefore the exported debug log -- preserves. It is
not rendered in the UI. Retries that eventually succeed keep their breadcrumbs.

Deliberately not mirroring the "structural validation failed" prefix: build.py
cannot tell structural failures from the rest without duplicating the
generator's code taxonomy, which would go stale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:16:10 -07:00
Dorian ad9a5be560 refactor(models): pass session into MCPToolProvider.load_user (#42432) 2026-09-17 13:37:27 +00:00
林玮 (Jade Lin) 223039128f feat(agent): support package imports from external URLs (#42425) 2026-09-17 12:54:40 +00:00
zyssyz123 d21518f8fe fix(agent): keep composer reads free of draft mutations (#42427) 2026-09-17 12:02:45 +00:00
wangxiaoleiandautofix-ci[bot] bfd7ac015f feat: skill support duplicate and export use the latest draft (#42424)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-17 09:52:25 +00:00
twwu 782688563d Merge branch 'main' into feat/dify-builder 2026-09-17 17:01:34 +08:00
林玮 (Jade Lin)andautofix-ci[bot] 3d7f6f4496 feat(agent): import roster packages through App routes (#42189)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-17 07:28:18 +00:00
wangxiaolei 7dad80b1f9 fix: fix upload skill zip has an empty directory (#42407) 2026-09-17 07:17:45 +00:00
林玮 (Jade Lin) 93895fa7fb feat(agent): add roster package export foundation (#42066) 2026-09-17 07:05:02 +00:00
wangxiaolei 0825e6ca51 fix: fix some skill issues (#42389) 2026-09-17 05:54:38 +00:00
JessYanandClaude Opus 5 28e2682986 refactor(models): pass session into App.app_model_config and App.workflow (#42190)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 05:43:18 +00:00
twwu 7d18af6483 Merge branch 'main' into feat/dify-builder 2026-09-17 09:57:23 +08:00
Lukas Buck 38f9d85d5a fix(api): report has_more from the page the query served (#42323) 2026-09-16 07:45:48 +00:00
Xx.Zhao 6a0039812d fix(agent): validate workspace skill references on publish (#42299) 2026-09-16 03:27:00 +00:00
Sean Sunandlyfuci a4d74f7c82 refactor(api): pass session explicitly into RBACResourceService (#37403) (#42248)
Co-authored-by: lyfuci <lyfuci11@gmail.com>
2026-09-15 12:45:34 +00:00
Bruce-Yiiandautofix-ci[bot] 79effdd498 fix(api): preserve literal NA in annotation CSV imports (#42221)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-12 04:55:53 +00:00
KingPointerandautofix-ci[bot] 085a79c748 chore(models): add Mapped annotations to EndUser and ApiToken (#42146)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-11 17:07:04 +00:00
twwu cecf2a1b9a fix(dify-builder): handle unavailable models during session creation 2026-09-11 18:28:52 +08:00
CoralGarden52 5b58f51460 perf(api): batch indexing status segment counts (#42191) 2026-09-11 07:43:16 +00:00
Charles YaoandClaude Opus 4.8 e59389776d fix(dify-builder): surface the specific reason in build-fail error card
When build-generate-graph failed at plan-approval, the error card always
showed one hardcoded generic message regardless of the actual cause
(UNRESOLVED_REFERENCE, non-object JSON, credit_balance_exhausted, ...).

Root cause: build_nodes() returned only list[MutationIntent]; on failure it
returned [], so the reason was logged internally but never reached the
handler, which had nothing to show but a fallback string.

Thread the reason out: build_nodes now returns BuildNodesResult{intents,
error}. _generation_error_text() extracts the best reason (top-level error
-> joined errors[].detail/message/code -> generic fallback), and every
failure path populates .error (generator error, all-intents-rejected, and
the except path via str(exc) so provider errors surface verbatim).
handle_plan_approval builds the ErrorCard body from build_result.error when
present, keeping the generic sentence only as a genuine fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 21:00:47 -07:00
twwu 8c41cbbd4c Merge branch 'main' into feat/dify-builder 2026-09-11 11:18:28 +08:00
林玮 (Jade Lin) cd61738f33 feat(api): normalize additional email providers (#42112) 2026-09-11 02:41:35 +00:00
Charles YaoandClaude Opus 4.8 2a574c7f74 fix(dify-builder): ground built workflow nodes to the user-selected model
Build wrote the Builder's session/cognition model (e.g. deepseek) into every
generated node's model block, ignoring the model the user picked at resource
confirmation (e.g. gpt). _ground always used the generation model, and the
selected model resource only reached the plan as label text (bind_resources) —
so a workflow got a model with no runtime credentials -> ModelNotExistError ->
instant test failure.

Thread the selected resource_ids from resource_selection through
agent.build_nodes into build.build_nodes, resolve the selected MODEL resource
(id is `{provider}/{name}`) via _selected_workflow_model, and ground node model
blocks to it — falling back to the session/generator model when the user
selected no model. The generator still runs on the session model (cognition);
only the built workflow's runtime model follows the user's choice.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 19:18:36 -07:00