Files
multica/packages/ui/markdown/file-cards.ts
Naiyuan Qing ebd0248be2 MUL-4016: fix mention tokenizer stacktrace backtracking (#4889)
* MUL-4016: fix mention tokenizer stacktrace backtracking

Co-authored-by: multica-agent <github@multica.ai>

* fix(editor): de-ambiguate escaped-label regexes to kill ReDoS (MUL-4016)

The mention/slash/file-card label regexes used `(?:\\.|[^\]])` where both
alternatives can consume a backslash. On an unterminated match, each `\x`
run is enumerated 2^n ways — pasting a Java stacktrace (`\~\[...\]`) or a
crafted ~50-char string freezes the main thread for seconds (GitHub #4881).

Exclude backslash from the char class (`[^\]\\]`) so a backslash can only be
consumed by `\\.`. The alternatives become disjoint and matching is linear;
legal escaped-bracket labels like `David\[TF\]` still parse unchanged.

Fixed in all four sites that shared the pattern:
- mention-extension.ts tokenize()
- slash-command-extension.ts start() + tokenize()
- file-card.tsx FILE_CARD_MARKDOWN_RE
- packages/ui/markdown/file-cards.ts NEW_FILE_CARD_RE (runs on every
  read-only comment/description render, not just the editor)

Adds adversarial regression tests (repeated `\a` + missing closing bracket)
that fail in ~10-40s against the old regexes and pass in <1ms after the fix.
Builds on #4889's marker-first mention start().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>

* fix(editor): escape backslash in mention/slash labels for round-trip (MUL-4016)

Follow-up to the de-ambiguation fix, addressing Howard's PR review.

The linear tokenizer now treats "\" as an escape lead (\\.), so a label whose
serialized form contains a bare "\" adjacent to the closing "]" no longer parses
back — the "\]" is consumed as an escaped bracket and swallows the boundary. The
old ambiguous regex tolerated this by chance; the de-ambiguation exposes it.

mention/slash renderMarkdown escaped only [ and ], not \. Switch both to the
shared escapeMarkdownLabel() (escapes [ ] \ ( )) and mirror it on parse with
replace(/\\([[\]\\()])/g, "$1"), matching what file-card already does. This also
converges the three tokenizers on one escape contract. file-card was already
correct and is unchanged.

Adds parameterized round-trip tests for labels containing "\" / "\]" / parens
(e.g. "A\\", "ends\\", "a\\]b"); these fail on the old serializer and pass now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 08:43:51 +08:00

130 lines
4.6 KiB
TypeScript

/**
* File card preprocessing for markdown content.
*
* Converts file-card syntax into HTML divs that can be rendered by
* react-markdown with a custom `div` component.
*
* Two syntaxes are supported:
* 1. `!file[name](url)` — new unambiguous syntax (no hostname check needed)
* 2. `[name](cdnUrl)` — legacy syntax, matched by CDN hostname on own line
*
* Output: `<div data-type="fileCard" data-href="url" data-filename="name"></div>`
*
* All functions are pure — no global state, no imports from core/ or views/.
*/
const IMAGE_EXTS = /\.(png|jpe?g|gif|webp|svg|ico|bmp|tiff?)$/i
// Keep in sync with UUID_RE in packages/core/types/attachment-url.ts.
const ATTACHMENT_UUID_SOURCE =
'[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}'
const ATTACHMENT_DOWNLOAD_URL_SOURCE = `/api/attachments/${ATTACHMENT_UUID_SOURCE}/download`
const ATTACHMENT_DOWNLOAD_URL_RE = new RegExp(
`^${ATTACHMENT_DOWNLOAD_URL_SOURCE}$`,
)
/**
* URL alternation accepted inside `!file[name](url)` markdown.
*
* Restricted to:
* - `/uploads/...` site-relative paths (LocalStorage backend with no LOCAL_UPLOAD_BASE_URL)
* - `/api/attachments/<UUID>/download` site-relative attachment downloads
* - `http(s)://...` absolute URLs (S3 / CloudFront / hosted)
*
* Anything else — `javascript:`, `data:`, protocol-relative `//host/x`, other
* APIs `/api/…`, path-traversal `/../…` — is rejected so a stored file-card
* cannot be turned into an out-of-band navigation.
*/
export const FILE_CARD_URL_PATTERN = new RegExp(
`/uploads/[^)]*|https?:\\/\\/[^)]+|${ATTACHMENT_DOWNLOAD_URL_SOURCE}`,
)
/** Prefix test applied by renderers to validate `data-href` before opening it. */
export function isAllowedFileCardHref(href: string): boolean {
return (
/^(https?:\/\/|\/uploads\/)/i.test(href) ||
ATTACHMENT_DOWNLOAD_URL_RE.test(href)
)
}
/**
* New syntax: !file[name](url) — unambiguous, no hostname matching needed.
* Backslash is excluded from the label char class so "\x" runs can only be
* consumed by \\. — overlapping alternatives backtrack in 2^n ways (ReDoS,
* GitHub #4881). This runs on every comment/description render.
*/
const NEW_FILE_CARD_RE = new RegExp(
`^!file\\[((?:\\\\.|[^\\]\\\\])*)\\]\\((${FILE_CARD_URL_PATTERN.source})\\)$`,
)
/** Legacy syntax: [name](cdnUrl) on its own line — matched by CDN hostname. */
const FILE_LINK_LINE = /^\[([^\]]+)\]\((https?:\/\/[^)]+)\)$/
function escapeAttr(s: string): string {
return s.replace(/&/g, '&amp;').replace(/"/g, '&quot;').replace(/</g, '&lt;')
}
function toFileCardHtml(filename: string, url: string): string {
return `<div data-type="fileCard" data-href="${escapeAttr(url)}" data-filename="${escapeAttr(filename)}"></div>`
}
/**
* Check if a URL points to our upload CDN.
*
* Uses exact hostname match against `cdnDomain` (e.g. "multica-static.copilothub.ai"),
* and also matches any `.amazonaws.com` subdomain as a fallback for direct S3 URLs.
*/
export function isCdnUrl(url: string, cdnDomain: string): boolean {
try {
const u = new URL(url)
return u.hostname === cdnDomain || u.hostname.endsWith('.amazonaws.com')
} catch {
return false
}
}
/**
* Check if a CDN URL is a non-image file that should render as a file card.
* Image URLs (png, jpg, etc.) are excluded — they render as inline images.
*/
export function isFileCardUrl(url: string, cdnDomain: string): boolean {
try {
return isCdnUrl(url, cdnDomain) && !IMAGE_EXTS.test(new URL(url).pathname)
} catch {
return false
}
}
/**
* Preprocess markdown to convert file-card syntax into HTML divs.
*
* Handles both `!file[name](url)` (new syntax) and legacy `[name](cdnUrl)`
* lines. Only standalone lines are matched — inline links are left untouched.
*
* @param markdown Raw markdown string
* @param cdnDomain CDN hostname for legacy link detection (e.g. "multica-static.copilothub.ai")
*/
export function preprocessFileCards(markdown: string, cdnDomain: string): string {
return markdown
.split('\n')
.map((line) => {
const trimmed = line.trim()
// New syntax: !file[name](url) — always a file card, no hostname check needed.
const newMatch = trimmed.match(NEW_FILE_CARD_RE)
if (newMatch) {
const filename = newMatch[1]!.replace(/\\([[\]\\()])/g, '$1')
return toFileCardHtml(filename, newMatch[2]!)
}
// Legacy: [name](cdnUrl) on its own line — CDN hostname matching.
const match = trimmed.match(FILE_LINK_LINE)
if (!match) return line
const filename = match[1]!
const url = match[2]!
if (!isFileCardUrl(url, cdnDomain)) return line
return toFileCardHtml(filename, url)
})
.join('\n')
}