mirror of
https://github.com/multica-ai/multica.git
synced 2026-08-05 17:40:11 +02:00
* fix(daemon): keep the runtime brief byte-stable across triggers (MUL-5377) Claude Code loads the runtime brief (CLAUDE.md / AGENTS.md) into messages[0], ahead of the entire conversation. A cache breakpoint is all-or-nothing, so a single differing byte there invalidates the prompt cache for the whole history on every `--resume`. InjectRuntimeConfig rewrites that file on every run and interpolated nine per-run values into it, so in practice the cache was thrown away on the first comment that landed on any issue. Measured on one issue over three runs: run 1 (cold) spent 89.9k cache-write tokens building 105k of context; runs 2 and 3 each spent ~425k re-creating a prefix they should have read. 842k of 946.5k cache-write tokens (89%) went into re-creation, with only tools[]+system[] surviving each resume (a constant 18,085 tokens both times). Fix: the brief now carries only what is stable for the lifetime of a resumed session, and per-run state travels in the per-turn user message, which is appended after the cached prefix. - Merge kindCommentTriggered + kindAssignmentTriggered into kindIssue, and stop reading TriggerCommentID in classifyTask. The brief can no longer diverge by trigger type structurally, rather than by convention. - Replace writeWorkflowComment/writeWorkflowAssignment with one writeWorkflowIssue that routes on the per-turn message. The mode-specific status rules live inside their own mode block, so "own the status arc" and "do not touch the status" can never be read as unarbitrated peers. - Move Task Initiator, Session Continuity Notice and Connected Apps out of the brief into BuildPrompt via BuildTaskInitiatorBlock / SessionContinuityNotice / BuildConnectedAppsBlock. - Drop TriggerCommentID, TriggerThreadID, NewCommentsSince, NewCommentCount, PriorSessionResumed and CommentReplyTargets from the brief; BuildPrompt already emitted all six from the same helpers, so this is de-duplication. - Set PriorSessionResumeUnavailable on `task` as well as `taskCtx` in both local resume gates, or the notice would silently vanish on exactly the failure path it exists to disclose. Tests: TestInjectRuntimeConfigByteIdenticalAcrossTriggers renders the brief across nine per-run variants (trigger type, differing comment/thread ids, resume delta, resume-unavailable, cross-thread fan-out, member/agent initiator, connected apps) for two providers and requires bytes.Equal, with a non-vacuity guard so it cannot pass on a function that ignores its input. Daemon-side tests assert the moved sections still reach the agent through the per-turn prompt. Co-authored-by: multica-agent <github@multica.ai> * fix(daemon): route the issue workflow on an explicit turn-mode marker Review follow-up on the mode router. The brief said Reply mode applies when the per-turn message "opens with a [NEW COMMENT] block", but buildCommentPrompt writes two paragraphs before that block and only emits it when TriggerCommentContent is non-empty. Two ways to get it wrong: - The message never literally opens with the block, so the router's own wording did not match the prompt it describes. - A comment-triggered run with an empty comment body — or an older server that does not send one — emitted no block at all. An agent following the brief would fall through to Ownership mode and change the issue status on a turn whose rule is "do NOT change the issue status". BuildPrompt now emits an unconditional `**Turn mode: Reply.**` / `**Turn mode: Ownership.**` line from the same branches it uses to pick a code path, and the brief routes on that marker. Brief and prompt can no longer disagree about the mode, because the value that selects the path also states it. The router also names a safe fallback (treat an unlabelled turn as Reply mode and leave the status alone). Tests: TestTurnModeMarkerAlwaysPresent covers comment-triggered with and without comment content, plus both assignment shapes; TestTurnModeMarkerAbsentOnIssuelessKinds keeps the marker off chat / quick-create / autopilot; TestBriefModeRouterMatchesPromptMarkers fails if the brief ever describes a marker the prompt does not emit. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai>
306 lines
11 KiB
Go
306 lines
11 KiB
Go
package execenv
|
|
|
|
import (
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
// TestBuildCommentReplyInstructionsCodexLinux pins that the Linux/macOS
|
|
// reply template now mandates `--content-file` (post-#4182). The previous
|
|
// `--content-stdin` + HEREDOC mandate (#1795 / #1851 / MUL-2904) was kept
|
|
// for years to defend against backtick / `$()` substitution in the body,
|
|
// but the heredoc/flag boundary turned out to be fragile in its own right:
|
|
// when a model wrapped extra flags around the heredoc on `multica issue
|
|
// create`, the flags got swallowed into stdin and silently dropped (OXY-78,
|
|
// OXY-76). The file path defeats both classes — the body never reaches the
|
|
// shell, and all flags live on one shell-token line.
|
|
//
|
|
// Not parallel: mutates the package-level runtimeGOOS.
|
|
func TestBuildCommentReplyInstructionsCodexLinux(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
runtimeGOOS = "linux"
|
|
|
|
issueID := "11111111-1111-1111-1111-111111111111"
|
|
triggerID := "22222222-2222-2222-2222-222222222222"
|
|
|
|
got := BuildCommentReplyInstructions("codex", issueID, triggerID)
|
|
|
|
for _, want := range []string{
|
|
"multica issue comment add " + issueID + " --parent " + triggerID + " --content-file ./reply.md",
|
|
"Write the reply body to a UTF-8 file",
|
|
"`--content-file`",
|
|
"#4182",
|
|
"rm ./reply.md",
|
|
"Do NOT write literal `\\n` escapes to simulate line breaks",
|
|
"do NOT reuse --parent values from previous turns",
|
|
} {
|
|
if !strings.Contains(got, want) {
|
|
t.Fatalf("codex/linux reply instructions missing %q\n---\n%s", want, got)
|
|
}
|
|
}
|
|
|
|
for _, banned := range []string{
|
|
"--content \"...\"",
|
|
"<<'COMMENT'",
|
|
"cat <<",
|
|
"--parent " + triggerID + " --content-stdin",
|
|
} {
|
|
if strings.Contains(got, banned) {
|
|
t.Fatalf("codex/linux reply instructions should not contain %q\n---\n%s", banned, got)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestBuildCommentReplyInstructionsNonCodexLinux pins that EVERY provider on
|
|
// Linux/macOS — not just Codex — gets the `--content-file` template. Two
|
|
// shell-driven failure classes motivate the uniform file path:
|
|
// - MUL-2904 / OKK-497: an agent inlined a backtick-wrapped table name into
|
|
// `--content`; the shell ran it as a command substitution, silently deleted
|
|
// it, the stored comment no longer matched the model's intent, and the
|
|
// model retried forever.
|
|
// - GitHub #4182 (OXY-78 / OXY-76): an agent wrapped extra flags around an
|
|
// `--content-stdin` HEREDOC; the bash heredoc/flag boundary swallowed
|
|
// `--assignee` / `--project` into stdin or dropped them as failed
|
|
// standalone shell statements, while the create still exited 0 with nulls.
|
|
//
|
|
// Both classes are shell-driven, so the guardrail is uniform across providers
|
|
// and across hosts.
|
|
//
|
|
// Not parallel: mutates the package-level runtimeGOOS.
|
|
func TestBuildCommentReplyInstructionsNonCodexLinux(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
|
|
issueID := "11111111-1111-1111-1111-111111111111"
|
|
triggerID := "22222222-2222-2222-2222-222222222222"
|
|
|
|
for _, host := range []string{"linux", "darwin"} {
|
|
for _, provider := range []string{"claude", "opencode", "openclaw", "hermes", "kimi", "kiro", "cursor"} {
|
|
name := provider + "/" + host
|
|
t.Run(name, func(t *testing.T) {
|
|
runtimeGOOS = host
|
|
got := BuildCommentReplyInstructions(provider, issueID, triggerID)
|
|
|
|
for _, want := range []string{
|
|
"multica issue comment add " + issueID + " --parent " + triggerID + " --content-file ./reply.md",
|
|
"Write the reply body to a UTF-8 file",
|
|
"`--content-file`",
|
|
"#4182",
|
|
"rm ./reply.md",
|
|
"do NOT reuse --parent values from previous turns",
|
|
"If you decide to reply",
|
|
} {
|
|
if !strings.Contains(got, want) {
|
|
t.Errorf("%s reply instructions missing %q\n---\n%s", name, want, got)
|
|
}
|
|
}
|
|
|
|
// The two regressions: agent-authored comments must never be
|
|
// steered at inline `--content "..."` (MUL-2904) and never at
|
|
// `--content-stdin` HEREDOC on multi-flag commands (#4182).
|
|
for _, banned := range []string{
|
|
"--content \"...\"",
|
|
"<<'COMMENT'",
|
|
"cat <<",
|
|
"--parent " + triggerID + " --content-stdin",
|
|
} {
|
|
if strings.Contains(got, banned) {
|
|
t.Errorf("%s reply instructions still contains %q\n---\n%s", name, banned, got)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestBuildCommentReplyInstructionsWindowsUsesContentFile pins that on
|
|
// Windows every provider — Codex AND non-Codex — gets the
|
|
// `--content-file` template. The bug is shell-layer, not provider-layer:
|
|
// any agent on Windows piping HEREDOC through PowerShell loses non-ASCII
|
|
// bytes (PS 5.1's `$OutputEncoding` defaults to ASCIIEncoding). Issues
|
|
// #2198 (Chinese, Codex), #2236 (Chinese, Codex), #2376 (Cyrillic,
|
|
// non-Codex agent name) all match this signature.
|
|
//
|
|
// Not parallel: mutates the package-level runtimeGOOS.
|
|
func TestBuildCommentReplyInstructionsWindowsUsesContentFile(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
runtimeGOOS = "windows"
|
|
|
|
issueID := "11111111-1111-1111-1111-111111111111"
|
|
triggerID := "22222222-2222-2222-2222-222222222222"
|
|
|
|
for _, provider := range []string{"codex", "claude", "opencode", "openclaw", "hermes", "kimi", "kiro", "cursor"} {
|
|
t.Run(provider+"/windows", func(t *testing.T) {
|
|
got := BuildCommentReplyInstructions(provider, issueID, triggerID)
|
|
for _, want := range []string{
|
|
"multica issue comment add " + issueID + " --parent " + triggerID + " --content-file",
|
|
"On Windows, write the reply body to a UTF-8 file",
|
|
"Do NOT pipe via `--content-stdin`",
|
|
"silently drops non-ASCII",
|
|
"$OutputEncoding",
|
|
} {
|
|
if !strings.Contains(got, want) {
|
|
t.Errorf("%s reply instructions missing %q\n---\n%s", provider, want, got)
|
|
}
|
|
}
|
|
for _, banned := range []string{
|
|
"<<'COMMENT'",
|
|
"--parent " + triggerID + " --content-stdin",
|
|
"cat <<",
|
|
} {
|
|
if strings.Contains(got, banned) {
|
|
t.Errorf("%s/windows reply instructions should not contain %q\n---\n%s", provider, banned, got)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestBuildCommentReplyInstructionsEmptyWhenNoTrigger(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
for _, provider := range []string{"codex", "claude", "opencode"} {
|
|
if got := BuildCommentReplyInstructions(provider, "issue-id", ""); got != "" {
|
|
t.Fatalf("expected empty string when triggerCommentID is empty for %s, got %q", provider, got)
|
|
}
|
|
}
|
|
}
|
|
|
|
// The brief must never carry this turn's trigger comment id; it points the
|
|
// agent at the per-turn user message instead (MUL-5377).
|
|
func TestInjectRuntimeConfigKeepsTriggerCommentOutOfBrief(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
runtimeGOOS = "linux"
|
|
|
|
dir := t.TempDir()
|
|
issueID := "11111111-1111-1111-1111-111111111111"
|
|
triggerID := "22222222-2222-2222-2222-222222222222"
|
|
|
|
if _, err := InjectRuntimeConfig(dir, "claude", TaskContextForEnv{
|
|
IssueID: issueID,
|
|
TriggerCommentID: triggerID,
|
|
}); err != nil {
|
|
t.Fatalf("InjectRuntimeConfig failed: %v", err)
|
|
}
|
|
content, err := os.ReadFile(filepath.Join(dir, "CLAUDE.md"))
|
|
if err != nil {
|
|
t.Fatalf("read CLAUDE.md: %v", err)
|
|
}
|
|
s := string(content)
|
|
|
|
if strings.Contains(s, triggerID) {
|
|
t.Errorf("CLAUDE.md must not carry the trigger comment id (MUL-5377)\n---\n%s", s)
|
|
}
|
|
for _, want := range []string{
|
|
"Mode router",
|
|
"`Turn mode: Reply.`",
|
|
"`Turn mode: Ownership.`",
|
|
"Use the `--parent` value the per-turn user message gives you for this turn",
|
|
"do NOT reuse a `--parent` from an earlier turn in this session",
|
|
} {
|
|
if !strings.Contains(s, want) {
|
|
t.Errorf("CLAUDE.md missing %q\n---\n%s", want, s)
|
|
}
|
|
}
|
|
}
|
|
|
|
// Windows reply instructions are file-first, never stdin. The instructions
|
|
// now ship in the per-turn prompt, so pin the helper directly (MUL-5377).
|
|
func TestWindowsCommentReplyInstructionsHaveNoStdin(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
runtimeGOOS = "windows"
|
|
|
|
issueID := "11111111-1111-1111-1111-111111111111"
|
|
triggerID := "22222222-2222-2222-2222-222222222222"
|
|
|
|
for _, provider := range []string{"claude", "codex", "opencode"} {
|
|
t.Run(provider, func(t *testing.T) {
|
|
s := BuildCommentReplyInstructions(provider, issueID, triggerID)
|
|
for _, want := range []string{
|
|
"multica issue comment add " + issueID + " --parent " + triggerID + " --content-file",
|
|
"--content-file",
|
|
} {
|
|
if !strings.Contains(s, want) {
|
|
t.Errorf("%s reply instructions missing %q\n---\n%s", provider, want, s)
|
|
}
|
|
}
|
|
for _, banned := range []string{
|
|
"--parent " + triggerID + " --content-stdin",
|
|
"always use `--content-stdin` with a HEREDOC, even for short single-line replies",
|
|
} {
|
|
if strings.Contains(s, banned) {
|
|
t.Errorf("%s reply instructions must not prescribe stdin on Windows: %q\n---\n%s", provider, banned, s)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestInjectRuntimeConfigWindowsAssignmentBriefStaysFileOnly pins the PR #3654
|
|
// review fix: on Windows, the ASSIGNMENT-triggered brief must never *recommend*
|
|
// `--content-stdin`. Unlike the comment-trigger path, the assignment workflow
|
|
// has no BuildCommentReplyInstructions override, so an agent that follows the
|
|
// "post your final results" step literally would pipe its final comment through
|
|
// PowerShell and drop non-ASCII bytes (#2198 / #2236 / #2376). The OS-aware
|
|
// ## Comment Formatting section (file-only on Windows) is the single source of
|
|
// truth; the Available Commands entry and step 6 must defer to it, not re-offer
|
|
// stdin. The flag synopsis may still *list* `--content-stdin` as available.
|
|
//
|
|
// Not parallel: mutates the package-level runtimeGOOS.
|
|
func TestInjectRuntimeConfigWindowsAssignmentBriefStaysFileOnly(t *testing.T) {
|
|
saved := runtimeGOOS
|
|
t.Cleanup(func() { runtimeGOOS = saved })
|
|
runtimeGOOS = "windows"
|
|
|
|
// Assignment-triggered: IssueID set, no TriggerCommentID.
|
|
ctx := TaskContextForEnv{IssueID: "issue-1"}
|
|
|
|
for _, provider := range []string{"claude", "codex", "opencode"} {
|
|
t.Run(provider, func(t *testing.T) {
|
|
dir := t.TempDir()
|
|
if _, err := InjectRuntimeConfig(dir, provider, ctx); err != nil {
|
|
t.Fatalf("InjectRuntimeConfig failed: %v", err)
|
|
}
|
|
fileName := "CLAUDE.md"
|
|
if provider != "claude" {
|
|
fileName = "AGENTS.md"
|
|
}
|
|
data, err := os.ReadFile(filepath.Join(dir, fileName))
|
|
if err != nil {
|
|
t.Fatalf("read %s: %v", fileName, err)
|
|
}
|
|
s := string(data)
|
|
|
|
// The Windows Comment Formatting section is file-only.
|
|
for _, want := range []string{
|
|
"## Comment Formatting",
|
|
"On Windows, **always write the comment body to a UTF-8 file",
|
|
"do NOT pipe via `--content-stdin`",
|
|
} {
|
|
if !strings.Contains(s, want) {
|
|
t.Errorf("%s missing Windows file-only guidance %q\n---\n%s", fileName, want, s)
|
|
}
|
|
}
|
|
|
|
// No prose may RECOMMEND stdin on Windows. The flag synopsis may
|
|
// still list `--content-stdin`; only the prescriptive "file or
|
|
// stdin" phrasings are banned.
|
|
for _, banned := range []string{
|
|
"or `--content-stdin`",
|
|
"using `--content-file` or `--content-stdin`",
|
|
"use `--content-file <path>` or `--content-stdin`",
|
|
} {
|
|
if strings.Contains(s, banned) {
|
|
t.Errorf("%s recommends stdin on Windows: %q\n---\n%s", fileName, banned, s)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|