Commit Graph
2917 Commits
Author SHA1 Message Date
twwu a931c81cdc Merge branch 'main' into feat/dify-builder 2026-09-17 10:53:03 +08:00
twwu 7d18af6483 Merge branch 'main' into feat/dify-builder 2026-09-17 09:57:23 +08:00
Pranav Agarwal b721e438ba fix(api): use inclusive <= for completion-conversation end-time filter (#42321) 2026-09-17 01:43:46 +00:00
Asuka Minato 9c6c48b50b test: keep API token cache coverage on real Redis (#40712) 2026-09-16 13:41:22 +00:00
Asuka Minato 8b9f3016e6 test: remove duplicate integration-only coverage (#40711) 2026-09-16 13:41:21 +00:00
Asuka Minato 9e4f0fa23a test: isolate parameter extractor model fixtures (#40710) 2026-09-16 13:41:21 +00:00
Lukas Buck 38f9d85d5a fix(api): report has_more from the page the query served (#42323) 2026-09-16 07:45:48 +00:00
Xx.Zhao 6a0039812d fix(agent): validate workspace skill references on publish (#42299) 2026-09-16 03:27:00 +00:00
Sean Sunandlyfuci a4d74f7c82 refactor(api): pass session explicitly into RBACResourceService (#37403) (#42248)
Co-authored-by: lyfuci <lyfuci11@gmail.com>
2026-09-15 12:45:34 +00:00
Mesrop Harutyunyan cb4d8ab5ea perf(api): batch dataset list detail fields (#42279) 2026-09-15 02:33:42 +00:00
human a5bc77eaa7 fix(api): delete offload objects outside the DB transaction (#42258) 2026-09-14 10:10:05 +00:00
Weike Zhang 318964e4dc fix(api): compute annotation list has_more with the effective page size (#41876) 2026-09-14 03:58:22 +00:00
CoralGarden52 5389f70719 fix(api): avoid per-row MAX(position) queries in batch segment import (#42233) 2026-09-14 02:11:23 +00:00
CoralGarden52 43ac0fce5c fix(rag): cap QA indexing preview results (#42205) 2026-09-13 13:25:14 +00:00
Bruce-Yiiandautofix-ci[bot] 79effdd498 fix(api): preserve literal NA in annotation CSV imports (#42221)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-12 04:55:53 +00:00
Bruce-Yii 7f7afc576c fix: ignore blank keyword rows in moderation limit (#42212) 2026-09-12 02:32:19 +00:00
KingPointerandautofix-ci[bot] 085a79c748 chore(models): add Mapped annotations to EndUser and ApiToken (#42146)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-11 17:07:04 +00:00
twwu cecf2a1b9a fix(dify-builder): handle unavailable models during session creation 2026-09-11 18:28:52 +08:00
CoralGarden52 c6c891430c fix(rag): preserve CSV cell text in batch segment import (#42159) 2026-09-11 08:17:13 +00:00
Sourav Rajviandautofix-ci[bot] aa6dceeece fix(web): send model delete params as query string on DELETE (#42162)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-11 08:09:48 +00:00
Harsh Kashyap 4cc2f632b8 fix: parse Redis Sentinel and Cluster nodes safely (#42111) 2026-09-11 08:06:32 +00:00
CoralGarden52 5b58f51460 perf(api): batch indexing status segment counts (#42191) 2026-09-11 07:43:16 +00:00
Charles YaoandClaude Opus 4.8 e59389776d fix(dify-builder): surface the specific reason in build-fail error card
When build-generate-graph failed at plan-approval, the error card always
showed one hardcoded generic message regardless of the actual cause
(UNRESOLVED_REFERENCE, non-object JSON, credit_balance_exhausted, ...).

Root cause: build_nodes() returned only list[MutationIntent]; on failure it
returned [], so the reason was logged internally but never reached the
handler, which had nothing to show but a fallback string.

Thread the reason out: build_nodes now returns BuildNodesResult{intents,
error}. _generation_error_text() extracts the best reason (top-level error
-> joined errors[].detail/message/code -> generic fallback), and every
failure path populates .error (generator error, all-intents-rejected, and
the except path via str(exc) so provider errors surface verbatim).
handle_plan_approval builds the ErrorCard body from build_result.error when
present, keeping the generic sentence only as a genuine fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 21:00:47 -07:00
Charles YaoandClaude Opus 5 96c8cc4383 fix(mcp): stop returning opaque 500s from MCP connect/auth (#42177)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 03:46:23 +00:00
Charles YaoandClaude Opus 4.8 1e6d776697 fix(dify-builder): surface model-config test failures instead of repair-thrashing
A verify/affected-path run that fails because the workflow references a model
that isn't configured/available (ModelNotExistError, "Model X does not exist",
provider/credentials errors) was routed into the config-repair path -> diagnose
+ propose_repair kept proposing graph mutations (if-else / check nodes) that
can't fix a missing model, thrashing across rounds.

Add model_config_error_text() (classifier parallel to is_input_failure; pairs an
existence/credential signal with a model/provider/credential token so unrelated
"<node> does not exist" faults aren't swallowed). In both build and edit failure
branches, when it matches, surface a clear "model not configured" diagnosis with
NO staged repair and route to the await-repair gate: the user configures the
model (global update_model) or changes it, then re-runs. Catalog the two new
user-facing strings so the reply-language layer localizes them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 20:37:56 -07:00
Charles YaoandClaude Opus 4.8 537a38157c fix(workflow-generator): repair options-less select start variables
Dify has no native boolean start-variable type, so a bool requirement the
Builder proposes gets emitted as a `select` — and the LLM often leaves its
`options` empty. An options-less select renders as an empty, unfillable dropdown
in the Builder's test-input form, so the user can never submit test data
(affects every workflow whose requirements include a bool field).

Add _normalize_start_select_variables to the generator postprocess (beside
_normalize_start_file_variables): a `select` start variable with no usable
(non-empty string) options is converted to a `text-input` the user can type
into; a select that declares real options is normalized to those strings and
kept. Idempotent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 20:21:18 -07:00
twwu 8c41cbbd4c Merge branch 'main' into feat/dify-builder 2026-09-11 11:18:28 +08:00
林玮 (Jade Lin) cd61738f33 feat(api): normalize additional email providers (#42112) 2026-09-11 02:41:35 +00:00
Charles YaoandClaude Opus 4.8 2a574c7f74 fix(dify-builder): ground built workflow nodes to the user-selected model
Build wrote the Builder's session/cognition model (e.g. deepseek) into every
generated node's model block, ignoring the model the user picked at resource
confirmation (e.g. gpt). _ground always used the generation model, and the
selected model resource only reached the plan as label text (bind_resources) —
so a workflow got a model with no runtime credentials -> ModelNotExistError ->
instant test failure.

Thread the selected resource_ids from resource_selection through
agent.build_nodes into build.build_nodes, resolve the selected MODEL resource
(id is `{provider}/{name}`) via _selected_workflow_model, and ground node model
blocks to it — falling back to the session/generator model when the user
selected no model. The generator still runs on the session model (cognition);
only the built workflow's runtime model follows the user's choice.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 19:18:36 -07:00
Harsh Kashyap a1bb317892 fix: strip keyword rows in keyword moderation (#42048) 2026-09-11 01:02:22 +00:00
Charles YaoandClaude Opus 4.8 968e442c10 fix(workflow-generator): repair HTTP-request response refs to body
Builds failed with an empty graph when the generator LLM wired downstream
references to an HTTP-request node's output as `{#node.response#}`. HTTP-request
nodes expose only {body, status_code, headers, files} — never `response` — and
the reference-repair pass only rewrites to a node's SOLE output, so with four
outputs the ref stayed unresolved: structural validation failed
(runner.py:646 UNRESOLVED_REFERENCE), generate_workflow_graph returned no graph,
and build_nodes produced an empty build ("Couldn't build the workflow").

Add a targeted, unambiguous output-name alias (http-request `response` -> `body`,
the response body) applied in _reconcile_variable_references before the
sole-output fallback, guarded so it only rewrites when `body` is actually
declared. Extensible via _OUTPUT_ALIASES for future known-wrong names.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 17:49:15 -07:00
Charles YaoandClaude Opus 4.8 70c92b97d9 fix(dify-builder): make M2 translation parsing model-agnostic
Harden the Localizer's batch translation against models that format JSON
differently (the Builder model is user-changeable):
- correlate by numeric INDEX instead of echoing source strings as keys, so a
  model that normalizes/truncates long keys no longer misses every lookup
- accept the reply as an index-keyed object OR a positional array, optionally
  wrapped one level (_translations_by_index)
- cache only real translations; a failed/partial reply is left UNCACHED so the
  next turn retries (self-heals) instead of sticking on the English original

TDD: added parser shape-variation tests + a no-cache-failure/self-heal test;
the fake now speaks the index protocol.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 17:31:38 -07:00
Charles YaoandClaude Opus 4.8 28fc7d0fac fix(dify-builder): unwrap the model's translation envelope in the Localizer
Live e2e revealed M2 never translated the static catalog strings: the model
returns the batch-translate mapping wrapped under a "strings" key (echoing the
request envelope) — {"strings": {src: translation}} — so _fill_cache's
top-level table.get(src) missed every source and cached the English original.
Unwrap one level when the sources aren't top-level keys, and tighten the prompt
to ask for a flat object. The unit-test fake now mirrors the real wrapped shape
so this can't regress (a flat fake hid the bug).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 17:03:29 -07:00
Charles YaoandClaude Opus 4.8 0597a8b512 feat(dify-builder): localize the update_model "Model changed to X" notice
Follow-up to the reply-language feature. The update_model action commits its
"Model changed to <name>" notice directly via compare_and_advance, bypassing
Runner._commit and thus the Env.localize_items hook, so that one notice stayed
English. Localize it in the service (when the session has a detected
reply_language) with a "Model changed to {name}" catalog template that
translates the frame and re-inserts the model name verbatim.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 16:38:48 -07:00
Charles YaoandClaude Opus 4.8 9b4e095e71 refactor(dify-builder): promote model_or_none to the agent protocol
Follow-up to the reply-language feature. `_model_or_none` was private to
LlmBuilderAgent, so the advance-task wiring reached it via a fragile
`getattr(agent, "_model_or_none", lambda: None)`. Make it part of the
DifyBuilderAgent protocol (typed `object | None` to keep the pure port free of
any model-runtime import) and implement it on every conforming agent
(LlmBuilderAgent renames the private method public; PlaceholderAgent and the
test doubles return None). The Localizer wiring now uses `agent.model_or_none`
directly. Also drops the useless-return ruff nit in the wiring test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 16:34:29 -07:00
Charles YaoandClaude Opus 4.8 ad06a30961 fix(dify-builder): restore reply_language persistence, share translation cache, harden localize contract
Final-review fix wave for the "reply in the user's input language" feature:

- serde.py: context_from_dict omitted reply_language from the explicit field
  allowlist, so it round-tripped to "" on every read despite context_to_dict
  (asdict) serializing it -- the localization feature silently no-op'd after
  the first persisted write. Added it to the constructor call; TDD'd with a
  failing-then-passing round-trip test in test_serde.py.
- localize.py: Localizer._cache was per-instance, but a fresh Localizer is
  built per advance-task invocation, so the static string catalog was
  re-translated every turn instead of once per language. Moved the cache to a
  module-level _TRANSLATION_CACHE dict (safe to share: keys are
  tenant/session-independent catalog-string+language pairs); added an autouse
  fixture in test_localize.py to clear it between tests for isolation.
- localize.py: detect_language and _fill_cache called self._model_provider()
  outside their try/except, violating the Env callback contract that these
  must never raise. Moved both calls inside their try/except so a raising
  provider degrades to the existing safe fallback instead of propagating into
  advance/_commit.
- strings.py: documented that Template.translate_fields is advisory only --
  the Localizer re-inserts all captured groups verbatim and does not consult
  it. No behavior change.

uv run --project api pytest tests/unit_tests/core/dify_builder tests/unit_tests/services/dify_builder -q
-> 722 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 16:17:54 -07:00
Charles YaoandClaude Opus 4.8 bd8951fbd6 feat(dify-builder): wire Localizer into the advance task (M2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:57:55 -07:00
Charles YaoandClaude Opus 4.8 a9775b523f fix(dify-builder): strengthen localize test to prove persisted mutation; document callback contract (fix round 1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:49:15 -07:00
Charles YaoandClaude Opus 4.8 e7c7c7684a feat(dify-builder): engine hooks for language detection + item localization (M2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:39:04 -07:00
Charles YaoandClaude Opus 4.8 7e70c7e189 fix(dify-builder): validate detect_language reply shape before trusting it (M2 fix round 1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:33:11 -07:00
Charles YaoandClaude Opus 4.8 32870a731a feat(dify-builder): Localizer — detect language + cached catalog translation (M2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:24:23 -07:00
Charles YaoandClaude Opus 4.8 c54274e92f feat(dify-builder): static-string catalog for reply localization (M2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:10:40 -07:00
Charles YaoandClaude Opus 4.8 9a87125231 feat(dify-builder): instruct agent to reply in the user's language (M1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 15:02:42 -07:00
Charles YaoandClaude Opus 4.8 97b8354173 fix(dify-builder): capture verify-launch errors instead of swallowing them
The build/edit test handlers wrapped run_draft in a bare, non-logging
`except Exception` that turned any launch-time failure into a blank failed
run (dify_run_id="", per_node=[], no error). Diagnose then had nothing to
act on and the repair loop ran blind — the prod symptom where a stale test
input (ValueError "<var> is required in input form") produced an
undiagnosable await_repair loop.

Now the launch error is logged, captured, and persisted so diagnose/routing
can act on it:
- Run.error field + dify_builder_runs.error column (migration a9cc8d552db4)
- handlers_build/edit: logger.exception + launch_error_text(exc) -> Run.error
- is_input_failure() also inspects Run.error (+ "in input form" signal), so a
  stale-input launch failure routes to the testdata gate, not blind repair
- diagnose/_degraded_diagnosis surface the launch error when no node executed

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-10 14:24:51 -07:00
Soroush Ahmadiandsoroush5 a425a127f5 fix(api): require exact match for CSRF whitelist paths (#42118)
Co-authored-by: soroush5 <soroush5@users.noreply.github.com>
2026-09-10 09:49:29 +00:00
wangxiaolei 66f1e6568e fix: fix sample_size not work (#42131) 2026-09-10 09:43:50 +00:00
twwu 485dede372 Merge branch 'main' into feat/dify-builder 2026-09-10 17:35:09 +08:00
QuantumGhost 798fd5e4c0 fix(api): restore Human Input completion events (#42082) 2026-09-10 06:20:44 +00:00
zyssyz123 c3f603c4dc fix(agent): pin chatflow participant versions across config updates (#42105) 2026-09-10 06:10:16 +00:00
Asuka Minato 31dfa21f36 test: migrate RAG retrieval sessions and ORM models to SQLite (#40512) 2026-09-10 04:15:40 +00:00