fix(server): remove the last language seeds from the LLM prompts (MUL-5689) (#6348)

* fix(server): drop the Chinese seed from the chat title prompt (MUL-5689)

chatTitleSystemPrompt carried literal Chinese in two places: a rule that
spelled out "Chinese input → Chinese title", and a formatting example
listing "标题:" as a prefix not to use. It is the same defect as the
quick-actions label rule — a prompt that names a language, even only as a
formatting example, reads as permission to answer in it.

Chat titles are generated from the user's opening message alone, so they
have neither the ALREADY SUGGESTED feedback loop nor an agent reply to be
pulled by. Removing the seed is the whole fix; the language rule stays,
now stated without naming a language.

Nothing is lost by dropping the "标题:" example: chatTitleLabelPrefixes
already strips 标题/题目/主题 (and the English forms) from the model's
output, which is where that guarantee actually lives.

Co-authored-by: multica-agent <github@multica.ai>

* fix(server): correct two inaccuracies in the quick-actions language rule

Both from review on #6345, non-blocking there and deferred to keep that
merge unblocked.

"these instructions" was self-referential: the LANGUAGE RULE is itself an
instruction, so a model reading "ignore these instructions when choosing
the language" could read the rule as disowning itself. It means the
system prompt, so it now says so.

The system prompt claimed the user message "ends with" the LANGUAGE RULE.
It does not — the task line follows it. Reworded to "contains a LANGUAGE
RULE line near the end", which is what the renderer actually produces.
The rule's position is unchanged: still after the conversation and the
replayed labels, which is the part that matters.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
This commit is contained in:
Bohan Jiang
2026-08-04 14:21:25 +08:00
committed by GitHub
parent 1251cbca0b
commit 757b09c801
3 changed files with 16 additions and 9 deletions

View File

@@ -23,16 +23,23 @@ const chatTitleGenTimeout = 20 * time.Second
// chatTitleSystemPrompt instructs the model to condense the opening of a
// conversation into a short, language-matched title. The rules mirror the
// acceptance criteria in MUL-4295: no quotes, no trailing punctuation, no
// "标题:" / "Title:" prefix, follow the conversation's language. sanitizeChatTitle
// re-applies these rules defensively in case the model ignores them.
// label prefix, follow the conversation's language. sanitizeChatTitle re-applies
// these rules defensively in case the model ignores them.
//
// This text names no language and contains no CJK, deliberately. A prompt that
// spells out a specific language — even only as a formatting example — reads as
// permission to answer in it, which is how quick actions ended up emitting
// Chinese pills for English conversations (MUL-5689). The label prefixes this
// used to enumerate are still stripped for real by chatTitleLabelPrefixes,
// which is where that guarantee belongs.
const chatTitleSystemPrompt = `You write a very short title that summarizes the topic of a chat conversation, given the user's opening message.
Rules:
- Output ONLY the title text — nothing else, no explanation.
- Keep it short: a few words, ideally under 8, never a full sentence.
- Write the title in the SAME language as the user's message (Chinese input → Chinese title, English input → English title).
- Write the title in the SAME language as the user's message, and in no other.
- Do NOT wrap the title in quotes or brackets.
- Do NOT prefix it with "Title:", "标题:", or similar.
- Do NOT prefix it with a label such as "Title:", in any language.
- Do NOT end with a period or any trailing punctuation.`
// maybeGenerateChatTitleAsync kicks off best-effort LLM title generation for a

View File

@@ -137,8 +137,8 @@ Field rules:
- "primary": true on exactly one suggestion, the single most likely next step.
false on all others.
Language: the user message ends with a LANGUAGE RULE line. It is authoritative;
follow it exactly.
Language: the user message contains a LANGUAGE RULE line near the end. It is
authoritative; follow it exactly.
Output JSON only, exactly this shape:
{"actions":[{"label":"...","prompt":"...","primary":true}]}
@@ -156,7 +156,7 @@ No prose, no markdown, no code fences.`
//
// Everything else is named and excluded explicitly, because each one has been
// observed to pull the output the wrong way (MUL-5689): the agent may reply in
// another language, these instructions are English, and ALREADY SUGGESTED
// another language, the system prompt is English, and ALREADY SUGGESTED
// replays the previous turn's labels — which is what made one bad pass stick,
// each Chinese label seeding the next round.
//
@@ -168,7 +168,7 @@ No prose, no markdown, no code fences.`
// to infer), it reads an earlier Korean turn over the latest Japanese one, and
// a kana sentence carrying enough English identifiers classifies as Latin, at
// which point a "never emit CJK" clause forbids the user's own script.
const chatQuickActionsLanguageRule = `LANGUAGE RULE: Write every "label" and "prompt" in the same language as the most recent [user] message above. Ignore the agent's reply, older messages, these instructions, and ALREADY SUGGESTED when choosing the language. If there is no [user] message, use the latest [agent] message.`
const chatQuickActionsLanguageRule = `LANGUAGE RULE: Write every "label" and "prompt" in the same language as the most recent [user] message above. Ignore the agent's reply, older messages, the system instructions, and ALREADY SUGGESTED when choosing the language. If there is no [user] message, use the latest [agent] message.`
// GenerateChatQuickActionsForTask runs one suggestion pass for a completed chat
// turn and attaches the result to that turn's assistant row, broadcasting

View File

@@ -210,7 +210,7 @@ func TestRenderChatQuickActionsContextClosesWithTheLanguageRule(t *testing.T) {
if !strings.Contains(out, "same language as the most recent [user] message") {
t.Fatalf("rule must anchor on the most recent user turn:\n%s", out)
}
for _, disowned := range []string{"agent's reply", "older messages", "these instructions", "ALREADY SUGGESTED"} {
for _, disowned := range []string{"agent's reply", "older messages", "the system instructions", "ALREADY SUGGESTED"} {
if !strings.Contains(chatQuickActionsLanguageRule, disowned) {
t.Fatalf("rule must explicitly exclude %q: %s", disowned, chatQuickActionsLanguageRule)
}