Files
multica/server/pkg/agent/codebuddy.go
Bohan Jiang a5a42846e6 fix(daemon): retry with a fresh session only when the resume was actually rejected (MUL-4966) (#5715)
* fix(daemon): gate fresh-session retry on tools executed, not session id (MUL-4966)

Switching provider accounts leaves the stored session id pointing at a
conversation the new account does not own. The daemon still passes it to
--resume, the provider rejects it, and the task dies before doing any work.

The existing fresh-session fallback was supposed to catch this but was
gated on `result.SessionID == ""`, which is not a lifecycle fact:

- Too narrow: a backend that echoes the requested id back when it rejects
  a resume keeps SessionID non-empty, so the fallback never fired — the
  reported bug.
- Too broad: a provider 401 before the first stream message also leaves
  SessionID empty, so an unrecoverable auth failure burned a second full
  run.

Gate on `tools == 0` instead. That states the property that actually makes
a retry safe — the agent executed no tool, so it mutated nothing, so
re-running cannot double-post a comment (comment creation has no
idempotency key and a duplicate re-fires its @mention triggers), reopen a
PR, or re-plan on top of its own half-finished work in the reused workdir.
Auth failures are excluded, mirroring retryableReasons in service/task.go.

The predicate is extracted to shouldRetryWithFreshSession so the tests
exercise production logic; both existing fallback tests re-implemented the
condition inline and would not have caught a regression in it.

Co-authored-by: multica-agent <github@multica.ai>

* fix(daemon): gate fresh-session retry on a positive resume-rejected signal (MUL-4966)

Review of the previous commit was right: `tools == 0` plus "not an auth
error" answers whether re-running is *safe*, not whether a new session can
*fix* the failure. Those are orthogonal, and answering the second by
exclusion inverts the burden of proof — the failures a fresh session cures
are a small enumerable set, while the ones it cannot are open-ended.

Concretely, the previous predicate fresh-retried on provider_network,
429/529, quota, 5xx and unclassified startup failures. provider_network is
the sharpest conflict: internal/service/task.go marks it resume-safe
(MUL-4910) specifically so the platform retry inherits the session and
continues the truncated conversation. Resetting the session first made that
contract unsatisfiable, silently discarding conversation context on a
transient blip — and rate limits got an immediate no-backoff re-run.

Replace the inference with positive evidence: agent.Result gains an explicit
ResumeRejected field, set only when a backend has proof the resume itself
was refused. claude/codebuddy/qwen derive it from resumeWasRejected, which
promotes the predicate resolveSessionID was already computing and encoding
as the side effect of blanking SessionID — using an empty string to carry
that meaning is what made the original bug possible. SessionID keeps being
dropped for a rejected resume (a dead pointer must not be persisted), but it
is no longer the signal the daemon reads to decide *why* a run failed.

The six ACP backends that recover from "session not found" set the flag at
the same points they already clear the id, so their existing recovery is not
caught by the narrower gate. codex needs nothing: thread/resume already falls
back to thread/start in-process, and deliberately does not on transport
errors.

Matching now includes the account-switch guardrail reported in #5704
(Claude Code 2.1.207, zh-CN): "400 此 session 已绑定另外的ai账号,请执行
/new 开启新 session". The en-US wording of the same guardrail has not been
captured yet, so those variants are marked inferred in the source; a miss
degrades to a terminal failure carrying the provider's raw text rather than
a mis-routed run.

Tests: backend-level fixtures drive ResumeRejected from real stream-json for
both the account-binding 400 and a network drop, and the predicate now
covers network/rate-limit/quota/5xx/auth/unclassified as explicit
non-retries.

Co-authored-by: multica-agent <github@multica.ai>

* fix(daemon): restore fresh-session recovery for backends with no rejection signal (MUL-4966)

Final review caught qwen regressing: its verified rejection string
("No saved session found with ID ...", already captured in
testdata/qwen-code-0.20.0-resume-not-found.stderr.txt) was not in the phrase
list, and qwen reports no session id on that path, so the new inclusion gate
turned a working auto-recovery into a terminal failure.

Auditing the other 17 resume-capable backends showed qwen was not alone.
antigravity, copilot, cursor, deveco and opencode all recovered from a
refused resume purely by reporting an empty SessionID, and none of them has
any rejection detection to convert into ResumeRejected — copilot's own
comment documents the hole (session.error before session.start), and
antigravity's helper returns "" when "the CLI exited before dispatching".
Making ResumeRejected the sole gate silently removed recovery from all five.

Fixing that by guessing rejection phrases for five more CLIs is the wrong
trade: no real output has been captured for any of them, and a false
positive discards a recoverable session pointer. So the gate is now two
tiers. Positive evidence (ResumeRejected) decides on its own where a backend
can produce it. Where none is available, an empty SessionID still gates the
retry — it proves no session was established, which is exactly what the
pre-change behaviour relied on — minus the classes a fresh session provably
cannot cure (network, rate limit, quota, provider 5xx, auth). That keeps the
resume-safe contract in internal/service/task.go intact while restoring what
these five backends had.

Also renames claudeResumeRejectedPhrases to resumeRejectedPhrases: it is
matched by claude, codebuddy and qwen, so a qwen-only string living under a
claude-prefixed name would be actively misleading.

Tests: qwen's existing missing-resume fixture now asserts ResumeRejected
(verified failing without the phrase), and the predicate covers the
no-signal tiers — retry when nothing was established, no retry once a
session exists or the failure classifies as uncurable.

Co-authored-by: multica-agent <github@multica.ai>

* fix(daemon): scope the no-signal fallback to backends that cannot detect rejections (MUL-4966)

Final review caught the compatibility path applying to every backend, not
just the five it was justified for. shouldRetryWithFreshSession only saw
(Result, priorSessionID, tools), so a false ResumeRejected could not be told
apart from a backend that has no way to answer — and claude/codebuddy/qwen/ACP
startup failures with no session id still fell through to the exclusion
branch. That contradicted both the stated intent and the function's own doc
comment ("where a backend can produce it, it is the whole answer").

Make the capability explicit. agent.ResumeRejectionUndetectable names the
five backends that scrape SessionID out of stream output and have no
rejection detection at all; the daemon takes provider and consults it, so a
capable backend reporting false is now taken at its word. Membership is
opt-in, so a new backend fails closed instead of silently inheriting a
guess-based retry.

Also completes the exclusion set: missing config, unavailable model, missing
executable, unsupported runtime version and (defensively) agent timeout all
have defined non-session remedies and were reaching `default: true`. What is
left through stays narrow — unknown, process failure, unparseable output,
context overflow — because a real rejection from these five most likely
surfaces as a non-zero exit or unparseable output, none of them reporting one
explicitly.

Tests: one identical result asserted across all five undetectable backends
(retries), twelve capable ones (no retry), and an unregistered provider
(fails closed), plus table cases for each newly excluded reason. Classifier
inputs were verified to map to the intended reasons rather than passing by
accident.

Also updates the ResumeRejected doc comment, which still said the daemon
gates on it alone.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
2026-07-21 19:07:46 +08:00

559 lines
16 KiB
Go

package agent
import (
"bufio"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"os/exec"
"strings"
"sync"
"time"
)
// codebuddyBackend implements Backend by spawning the CodeBuddy CLI
// (a Claude Code fork) with --output-format stream-json.
// It mirrors claude.go's execution model: concurrent stdin/stdout to
// avoid pipe deadlocks, open stdin for control_request auto-approval,
// and runContext for zero-timeout = no-deadline semantics.
type codebuddyBackend struct {
cfg Config
}
// codebuddyBlockedArgs are flags hardcoded by the daemon that must not be
// overridden by user-configured custom_args. Overriding these would break
// the daemon↔codebuddy communication protocol.
var codebuddyBlockedArgs = map[string]blockedArgMode{
"-p": blockedStandalone, // non-interactive mode
"--output-format": blockedWithValue, // stream-json protocol
"--input-format": blockedWithValue, // stream-json protocol
"--permission-mode": blockedWithValue, // bypassPermissions for autonomous operation
"--mcp-config": blockedWithValue, // set by daemon from agent.mcp_config
// `--effort` is owned by the per-agent thinking_level picker so a
// user-supplied custom_arg cannot silently outvote it.
"--effort": blockedWithValue,
}
func buildCodebuddyArgs(opts ExecOptions, logger *slog.Logger) []string {
args := []string{
"-p",
"--output-format", "stream-json",
"--input-format", "stream-json",
"--verbose",
"--strict-mcp-config",
"--permission-mode", "bypassPermissions",
"--disallowedTools", "AskUserQuestion",
}
if opts.Model != "" {
args = append(args, "--model", opts.Model)
}
if opts.ThinkingLevel != "" {
args = append(args, "--effort", opts.ThinkingLevel)
}
if opts.MaxTurns > 0 {
args = append(args, "--max-turns", fmt.Sprintf("%d", opts.MaxTurns))
}
if opts.SystemPrompt != "" {
args = append(args, "--append-system-prompt", opts.SystemPrompt)
}
if opts.ResumeSessionID != "" {
args = append(args, "--resume", opts.ResumeSessionID)
}
args = append(args, filterCustomArgs(opts.ExtraArgs, codebuddyBlockedArgs, logger)...)
args = append(args, filterCustomArgs(opts.CustomArgs, codebuddyBlockedArgs, logger)...)
return args
}
func (b *codebuddyBackend) Execute(ctx context.Context, prompt string, opts ExecOptions) (*Session, error) {
execPath := b.cfg.ExecutablePath
if execPath == "" {
execPath = "codebuddy"
}
if _, err := exec.LookPath(execPath); err != nil {
return nil, fmt.Errorf("codebuddy executable not found at %q: %w", execPath, err)
}
timeout := opts.Timeout
runCtx, cancel := runContext(ctx, timeout)
args := buildCodebuddyArgs(opts, b.cfg.Logger)
// If the caller provided an MCP config, write it to a temp file and pass
// --mcp-config <path> so the agent uses a controlled set of MCP servers.
var mcpConfigPath string
var mcpFileCleanup func()
if len(opts.McpConfig) > 0 {
path, err := writeMcpConfigToTemp(opts.McpConfig)
if err != nil {
cancel()
return nil, err
}
mcpConfigPath = path
mcpFileCleanup = func() { cleanupMcpConfigTemp(mcpConfigPath) }
args = append(args, "--mcp-config", mcpConfigPath)
}
// Clean up the temp file if we return before the goroutine takes ownership.
defer func() {
if mcpFileCleanup != nil {
mcpFileCleanup()
}
}()
cmd := exec.CommandContext(runCtx, execPath, args...)
hideAgentWindow(cmd)
b.cfg.Logger.Info("agent command", "exec", execPath, "args", args)
cmd.WaitDelay = 10 * time.Second
if opts.Cwd != "" {
cmd.Dir = opts.Cwd
}
cmd.Env = buildEnv(b.cfg.Env)
stdout, err := cmd.StdoutPipe()
if err != nil {
cancel()
return nil, fmt.Errorf("codebuddy stdout pipe: %w", err)
}
stdin, err := cmd.StdinPipe()
if err != nil {
cancel()
return nil, fmt.Errorf("codebuddy stdin pipe: %w", err)
}
var closeStdinOnce sync.Once
closeStdin := func() { closeStdinOnce.Do(func() { _ = stdin.Close() }) }
stderrBuf := newStderrTail(newLogWriter(b.cfg.Logger, "[codebuddy:stderr] "), agentStderrTailBytes)
cmd.Stderr = stderrBuf
if err := cmd.Start(); err != nil {
closeStdin()
cancel()
return nil, fmt.Errorf("start codebuddy: %w", err)
}
b.cfg.Logger.Info("codebuddy started", "pid", cmd.Process.Pid, "cwd", opts.Cwd, "model", opts.Model)
// cmd.Start() succeeded — transfer temp file ownership to the goroutine.
mcpFileCleanup = nil
msgCh := make(chan Message, 256)
resCh := make(chan Result, 1)
// Write the prompt in a dedicated goroutine to prevent deadlock.
// CodeBuddy (like Claude Code) emits a startup banner to stdout before
// reading stdin; a synchronous write would block once the pipe buffer
// fills. Keep stdin open after writing so control_request events can
// be answered mid-run.
writeDone := make(chan error, 1)
go func() {
err := writeCodebuddyInput(stdin, prompt)
if err != nil {
closeStdin()
}
writeDone <- err
}()
go func() {
defer cancel()
defer close(msgCh)
defer close(resCh)
if mcpConfigPath != "" {
defer cleanupMcpConfigTemp(mcpConfigPath)
}
startTime := time.Now()
var lastAssistantText string
var finalResultText string
sawResult := false
resultIsError := false
var sessionID string
usage := make(map[string]TokenUsage)
eventCount := 0
invalidEventCount := 0
assistantEventCount := 0
toolUseCount := 0
// Close stdout when the context is cancelled so scanner.Scan() unblocks.
go func() {
<-runCtx.Done()
closeStdin()
_ = stdout.Close()
}()
scanner := bufio.NewScanner(stdout)
scanner.Buffer(make([]byte, 0, 1024*1024), 10*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
var msg codebuddySDKMessage
if err := json.Unmarshal([]byte(line), &msg); err != nil {
invalidEventCount++
continue
}
eventCount++
switch msg.Type {
case "assistant":
assistantEventCount++
assistantText, tools := b.handleAssistant(msg, msgCh, usage)
toolUseCount += tools
if tools == 0 {
lastAssistantText = assistantText
} else {
// A turn that invokes a tool is intermediate even when it also
// contains narration. Do not use it as an empty-result fallback.
lastAssistantText = ""
}
case "user":
b.handleUser(msg, msgCh)
case "system":
if msg.SessionID != "" {
sessionID = msg.SessionID
}
trySend(msgCh, Message{Type: MessageStatus, Status: "running", SessionID: sessionID})
case "result":
sawResult = true
finalResultText = msg.ResultText
resultIsError = msg.IsError
sessionID = msg.SessionID
if resultUsage := codebuddyResultUsage(msg, opts.Model); len(resultUsage) > 0 {
usage = resultUsage
}
closeStdin()
case "log":
if msg.Log != nil {
trySend(msgCh, Message{
Type: MessageLog,
Level: msg.Log.Level,
Content: msg.Log.Message,
})
}
case "control_request":
b.handleControlRequest(msg, stdin)
}
}
scanErr := scanner.Err()
if scanErr != nil {
// Stop a malformed/oversized writer from blocking cmd.Wait after the
// scanner has stopped consuming stdout.
_ = stdout.Close()
}
closeStdin()
// Wait for process exit.
exitErr := cmd.Wait()
duration := time.Since(startTime)
// writeDone is buffered (cap 1) and the writer always sends — by the
// time cmd has exited, the prompt write has either succeeded, hit a
// broken pipe, or been unblocked by the kill that ended cmd.
writeErr := <-writeDone
finalStatus, finalOutput, finalError := finalizeStreamResult(
"codebuddy",
timeout,
runCtx.Err(),
writeErr,
exitErr,
sessionID,
streamTerminalState{
lastAssistantText: lastAssistantText,
finalResultText: finalResultText,
sawResult: sawResult,
resultIsError: resultIsError,
scanErr: scanErr,
},
"",
)
if finalError != "" {
finalError = withAgentStderr(finalError, "codebuddy", stderrBuf.Tail())
}
logStreamProtocolObservation(b.cfg.Logger, streamProtocolObservation{
provider: "codebuddy",
cliVersion: b.cfg.CLIVersion,
model: opts.Model,
exitCode: streamProcessExitCode(exitErr),
eventCount: eventCount,
invalidEventCount: invalidEventCount,
assistantEventCount: assistantEventCount,
toolUseCount: toolUseCount,
sawResult: sawResult,
resultIsError: resultIsError,
resultBytes: len(finalResultText),
lastAssistantBytes: len(lastAssistantText),
scannerError: scanErr != nil,
anthropicBaseURLConfigured: strings.TrimSpace(b.cfg.Env["ANTHROPIC_BASE_URL"]) != "",
})
b.cfg.Logger.Info("codebuddy finished", "pid", cmd.Process.Pid, "status", finalStatus, "duration", duration.Round(time.Millisecond).String())
resumeRejected := resumeWasRejected(opts.ResumeSessionID, sessionID, finalStatus == "failed", finalError)
reportedSessionID := resolveSessionID(opts.ResumeSessionID, sessionID, finalStatus == "failed", finalError)
if resumeRejected {
b.cfg.Logger.Info("codebuddy resume was rejected; dropping session id and signalling fresh-session retry",
"requested_resume", opts.ResumeSessionID,
"emitted_session", sessionID,
)
}
resCh <- Result{
Status: finalStatus,
Output: finalOutput,
Error: finalError,
DurationMs: duration.Milliseconds(),
SessionID: reportedSessionID,
Usage: usage,
ResumeRejected: resumeRejected,
}
}()
return &Session{Messages: msgCh, Result: resCh}, nil
}
func (b *codebuddyBackend) handleAssistant(msg codebuddySDKMessage, ch chan<- Message, usage map[string]TokenUsage) (string, int) {
var content codebuddyMessageContent
if err := json.Unmarshal(msg.Message, &content); err != nil {
return "", 0
}
var assistantText strings.Builder
toolUseCount := 0
// Accumulate token usage per model.
if content.Usage != nil && content.Model != "" {
u := usage[content.Model]
u.InputTokens += content.Usage.InputTokens
u.OutputTokens += content.Usage.OutputTokens
u.CacheReadTokens += content.Usage.CacheReadInputTokens
u.CacheWriteTokens += content.Usage.CacheCreationInputTokens
usage[content.Model] = u
}
for _, block := range content.Content {
switch block.Type {
case "text":
if block.Text != "" {
assistantText.WriteString(block.Text)
trySend(ch, Message{Type: MessageText, Content: block.Text})
}
case "thinking":
if block.Text != "" {
trySend(ch, Message{Type: MessageThinking, Content: block.Text})
}
case "tool_use":
toolUseCount++
var input map[string]any
if block.Input != nil {
_ = json.Unmarshal(block.Input, &input)
}
trySend(ch, Message{
Type: MessageToolUse,
Tool: block.Name,
CallID: block.ID,
Input: input,
})
}
}
return assistantText.String(), toolUseCount
}
func (b *codebuddyBackend) handleUser(msg codebuddySDKMessage, ch chan<- Message) {
var content codebuddyMessageContent
if err := json.Unmarshal(msg.Message, &content); err != nil {
return
}
for _, block := range content.Content {
if block.Type == "tool_result" {
resultStr := ""
if block.Content != nil {
resultStr = string(block.Content)
}
trySend(ch, Message{
Type: MessageToolResult,
CallID: block.ToolUseID,
Output: resultStr,
})
}
}
}
func (b *codebuddyBackend) handleControlRequest(msg codebuddySDKMessage, stdin interface{ Write([]byte) (int, error) }) {
// Auto-approve all tool uses in autonomous/daemon mode.
var req codebuddyControlRequestPayload
if err := json.Unmarshal(msg.Request, &req); err != nil {
return
}
var inputMap map[string]any
if req.Input != nil {
_ = json.Unmarshal(req.Input, &inputMap)
}
if inputMap == nil {
inputMap = map[string]any{}
}
response := map[string]any{
"type": "control_response",
"response": map[string]any{
"subtype": "success",
"request_id": msg.RequestID,
"response": map[string]any{
"behavior": "allow",
"updatedInput": inputMap,
},
},
}
data, err := json.Marshal(response)
if err != nil {
b.cfg.Logger.Warn("codebuddy: failed to marshal control response", "error", err)
return
}
data = append(data, '\n')
if _, err := stdin.Write(data); err != nil {
b.cfg.Logger.Warn("codebuddy: failed to write control response", "error", err)
}
}
func writeCodebuddyInput(w io.Writer, prompt string) error {
payload := map[string]any{
"type": "user",
"message": map[string]any{
"role": "user",
"content": []map[string]string{
{
"type": "text",
"text": prompt,
},
},
},
}
data, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("marshal codebuddy input: %w", err)
}
data = append(data, '\n')
if _, err := w.Write(data); err != nil {
return err
}
return nil
}
// ── Codebuddy SDK JSON types ──
type codebuddySDKMessage struct {
Type string `json:"type"`
Message json.RawMessage `json:"message,omitempty"`
Subtype string `json:"subtype,omitempty"`
SessionID string `json:"session_id,omitempty"`
Model string `json:"model,omitempty"`
// result fields
ResultText string `json:"result,omitempty"`
IsError bool `json:"is_error,omitempty"`
DurationMs float64 `json:"duration_ms,omitempty"`
NumTurns int `json:"num_turns,omitempty"`
Usage *codebuddyUsage `json:"usage,omitempty"`
ModelUsage map[string]codebuddyResultModelUsage `json:"modelUsage,omitempty"`
// log fields
Log *codebuddyLogEntry `json:"log,omitempty"`
// control request fields
RequestID string `json:"request_id,omitempty"`
Request json.RawMessage `json:"request,omitempty"`
}
type codebuddyLogEntry struct {
Level string `json:"level"`
Message string `json:"message"`
}
type codebuddyMessageContent struct {
Role string `json:"role"`
Model string `json:"model"`
Content []codebuddyContentBlock `json:"content"`
Usage *codebuddyUsage `json:"usage,omitempty"`
}
type codebuddyUsage struct {
InputTokens int64 `json:"input_tokens"`
OutputTokens int64 `json:"output_tokens"`
CacheReadInputTokens int64 `json:"cache_read_input_tokens"`
CacheCreationInputTokens int64 `json:"cache_creation_input_tokens"`
}
type codebuddyResultModelUsage struct {
InputTokens int64 `json:"inputTokens"`
OutputTokens int64 `json:"outputTokens"`
CacheReadInputTokens int64 `json:"cacheReadInputTokens"`
CacheCreationInputTokens int64 `json:"cacheCreationInputTokens"`
}
type codebuddyContentBlock struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
ToolUseID string `json:"tool_use_id,omitempty"`
Content json.RawMessage `json:"content,omitempty"`
}
type codebuddyControlRequestPayload struct {
Subtype string `json:"subtype"`
ToolName string `json:"tool_name,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
}
func codebuddyResultUsage(msg codebuddySDKMessage, fallbackModel string) map[string]TokenUsage {
if len(msg.ModelUsage) > 0 {
usage := make(map[string]TokenUsage, len(msg.ModelUsage))
for model, u := range msg.ModelUsage {
if model == "" || !codebuddyUsageHasTokens(u.InputTokens, u.OutputTokens, u.CacheReadInputTokens, u.CacheCreationInputTokens) {
continue
}
usage[model] = TokenUsage{
InputTokens: u.InputTokens,
OutputTokens: u.OutputTokens,
CacheReadTokens: u.CacheReadInputTokens,
CacheWriteTokens: u.CacheCreationInputTokens,
}
}
if len(usage) > 0 {
return usage
}
}
model := msg.Model
if model == "" {
model = fallbackModel
}
if msg.Usage == nil || model == "" || !codebuddyUsageHasTokens(
msg.Usage.InputTokens,
msg.Usage.OutputTokens,
msg.Usage.CacheReadInputTokens,
msg.Usage.CacheCreationInputTokens,
) {
return nil
}
return map[string]TokenUsage{
model: {
InputTokens: msg.Usage.InputTokens,
OutputTokens: msg.Usage.OutputTokens,
CacheReadTokens: msg.Usage.CacheReadInputTokens,
CacheWriteTokens: msg.Usage.CacheCreationInputTokens,
},
}
}
func codebuddyUsageHasTokens(input, output, cacheRead, cacheWrite int64) bool {
return input > 0 || output > 0 || cacheRead > 0 || cacheWrite > 0
}