highperfocused/multica
1
0
Fork 0
mirror of https://github.com/multica-ai/multica.git synced 2026-08-12 19:06:06 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
fix/migration-265-duplicate-prefix
multica/docker-compose.selfhost.yml
Xichang(Seacen) Zhao a4d4444f80 feat(wecom): add the MULTICA_WECOM_TRACE operator switch (and stop it printing binding tokens) (#6602) 
* feat(wecom): add the MULTICA_WECOM_TRACE operator switch

The wecom adapter logs failures and nothing else. A bad envelope warns, a
non-zero server ack warns, an event and a non-text receipt log at Debug —
but an ordinary inbound message and every single outbound frame produce no
log line at all. So a run that goes wrong QUIETLY leaves nothing behind on
the server: the reply that went to the room instead of the person, the
command that was silently dropped, the receipt that went out twice.

What that costs in practice: verifying any of those needs a person to
describe what they saw on their phone. That is slow, it is lossy, and for
anything about ordering or timing it does not work at all — a user cannot
tell you which of two frames the server took first, or that the frame we
thought we sent was never written.

MULTICA_WECOM_TRACE=1 records every frame the adapter reads and writes:
direction, cmd, req_id, the chat it was addressed to, whether that chat is
a room or a person, the sender, and the server's errcode. Off by default,
and off is the right state outside a debugging session.

Why not slog.Debug, which is how slack and lark get their per-frame lines
(slack_channel.go:162, lark/ws_connector.go:339): logger.parseLevel
defaults LOG_LEVEL to *debug*, so a Debug call is on in every deployment
that has not set LOG_LEVEL. This records a bounded prefix of message text,
which must not be on by default, so the switch has to be its own.

Because it records message text, it must not record credentials:

  - The aibot_subscribe body carries the installation's decrypted smart-bot
    secret. traceOut walks named fields instead of dumping the frame, so
    the secret is never read.

  - A binding token is a bearer credential, and the binding prompt fits
    inside the preview. sendBindingPrompt builds the prompt copy plus
    appURL + "/wecom/bind?token=" + a 43-character token; with a normal
    MULTICA_APP_URL the token's last character lands at rune 107-112,
    inside the 120-rune cap. An unredacted preview therefore printed the
    whole live token. RedeemAndBind only checks that the redeemer belongs
    to the token's workspace and the bind page redeems on load as whoever
    is signed in, so anyone who could read the log could bind that sender's
    WeCom identity to their own Multica account — the same hijack
    replier.go:150 already refuses to post the link into a room to prevent.
    tracePreview now redacts token= / binding_token= / access_token= /
    code= query parameters before the cut.

The preview cap counts runes rather than bytes: a byte cut on Chinese text
truncates at a third of the intended length and can split a character into
invalid UTF-8, in the deployments this switch exists for.

Co-authored-by: multica-agent <github@multica.ai>

* fix(wecom): record outbound frames under the writer mutex, with an outcome

The outbound trace was taken before the writer mutex, which broke it in the
two ways the switch exists to prevent.

Order. Heartbeats, agent replies and inbox pushes all reach wsSender.write
concurrently on one connection. A goroutine could emit its dir=out line, be
descheduled before taking the mutex, and let another trace-and-write ahead of
it — so the log named the wrong frame as first. The mutex is where those
senders become ordered, so a record taken inside it matches the wire by
construction; one taken outside is only correlated with it. The extraction
stays outside: redacting bearer tokens with a regexp and cutting the body to
120 runes is the expensive half and needs no such guarantee. What runs under
the mutex is a nil check when tracing is off, and two log lines when it is on,
against a socket write already in the same section.

Outcome. A frame rejected by SetWriteDeadline or WriteMessage left a dir=out
line identical to a delivered frame's, so "did this actually reach the wire?"
— the question an operator turns the switch on to ask — had no answer in the
log. Each outbound frame is now recorded twice: dir=out when it is about to be
written, dir=out.done with ok=true/false and, on failure, the stage that
failed and the socket's error. One line would not do either job: written
before the write it cannot report the outcome, and written after it, a write
that hangs or a process killed mid-write leaves nothing at all.

The pair shares a seq rather than a req_id. A pong echoes the server's req_id,
which may be empty or repeated, so req_id is not a key; seq is assigned under
the writer mutex, never goes on the wire, and doubles as the frame's position
in the write order, so a reader can recover the order from the field and can
spot an attempt with no outcome.

Tests: two writers where the first is held inside its own trace emission until
the second has run a whole write() — deterministic, because with the record
under the mutex the second blocks and with the record outside it does not; the
same property under 16 concurrent writers, which also pins seq and the
attempt/outcome pairing; and a socket stub failing WriteMessage and
SetWriteDeadline separately.

Docs: the operational half of the switch — restart to change it, who can read
the logs it writes, and whose retention policy governs them — in
SELF_HOSTING_ADVANCED.md.

Co-authored-by: multica-agent <github@multica.ai>

* docs(wecom): a written frame is not an accepted frame

The trace section said an out/out.done pair 'tells you what the server
saw'. It does not: ok=true means WriteMessage returned, which is the
local socket accepting the bytes. WeCom's verdict arrives later, on the
dir=in line carrying the same req_id, and a frame can be written
successfully and rejected there. An operator reading ok=true as
'delivered' would stop exactly one step short of the errcode that
explains the failure they are chasing.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: Bohan-J <bohan@devv.ai>
2026-08-09 00:05:40 +08:00

174 lines
8.5 KiB
YAML
Raw Permalink Blame History

# Self-hosting Docker Compose — starts PostgreSQL, backend, and frontend.
#
# Services bind to 127.0.0.1 only. For cross-machine or public access, front
# them with a reverse proxy (Caddy / nginx / Cloudflare Tunnel) that terminates
# TLS and forwards to 127.0.0.1:8080 (backend) and 127.0.0.1:3000 (frontend).
# Do NOT change these bindings to 0.0.0.0 — Docker bypasses host firewalls
# (UFW/iptables) by default, so the raw ports would be exposed to the internet
# with the default JWT_SECRET and Postgres credentials. See:
# apps/docs/content/docs/self-host-quickstart.mdx
#
# Usage:
# cp .env.example .env
# # Edit .env — change JWT_SECRET at minimum
# docker compose -f docker-compose.selfhost.yml up -d
#
# Frontend: http://localhost:${FRONTEND_PORT:-3000}
# Backend: http://localhost:${BACKEND_PORT:-${API_PORT:-${SERVER_PORT:-${PORT:-8080}}}}
#
# The published values above are HOST ports; the containers always listen on
# 8080 / 3000 internally, so changing them never needs a rebuild. PORT is the
# variable to edit; BACKEND_PORT, API_PORT and SERVER_PORT are optional aliases
# that override it in that order. Keep this alias order identical to Makefile
# and scripts/local-env.sh. The web dev fallback intentionally omits PORT
# because Next uses that variable for its own frontend listener.
#
# Note that *which source* wins differs per entry point — Compose lets the
# calling environment outrank this file, while make lets the included env file
# outrank the environment. Nothing re-derives the published port from the inputs
# any more: `make selfhost` (via scripts/selfhost-wait.sh) and both installers
# read it back with `docker compose port`, so the health check and the printed
# URL always match what was actually published.
name: multica
services:
postgres:
image: pgvector/pgvector:pg17
environment:
POSTGRES_DB: ${POSTGRES_DB:-multica}
POSTGRES_USER: ${POSTGRES_USER:-multica}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-multica}
volumes:
- pgdata:/var/lib/postgresql/data
restart: unless-stopped
healthcheck:
test:
[
"CMD-SHELL",
"pg_isready -U ${POSTGRES_USER:-multica} -d ${POSTGRES_DB:-multica}",
]
interval: 5s
timeout: 5s
retries: 5
backend:
image: ${MULTICA_BACKEND_IMAGE:-ghcr.io/multica-ai/multica-backend}:${MULTICA_IMAGE_TAG:-latest}
depends_on:
postgres:
condition: service_healthy
ports:
- "127.0.0.1:${BACKEND_PORT:-${API_PORT:-${SERVER_PORT:-${PORT:-8080}}}}:8080"
volumes:
- backend_uploads:/app/data/uploads
environment:
DATABASE_URL: postgres://${POSTGRES_USER:-multica}:${POSTGRES_PASSWORD:-multica}@postgres:5432/${POSTGRES_DB:-multica}?sslmode=disable
PORT: "8080"
METRICS_ADDR: ${METRICS_ADDR:-}
JWT_SECRET: ${JWT_SECRET:-change-me-in-production}
FRONTEND_ORIGIN: ${FRONTEND_ORIGIN:-http://localhost:3000}
CORS_ALLOWED_ORIGINS: ${CORS_ALLOWED_ORIGINS:-}
RESEND_API_KEY: ${RESEND_API_KEY:-}
RESEND_FROM_EMAIL: ${RESEND_FROM_EMAIL:-noreply@multica.ai}
SMTP_HOST: ${SMTP_HOST:-}
SMTP_PORT: ${SMTP_PORT:-25}
SMTP_USERNAME: ${SMTP_USERNAME:-}
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
SMTP_TLS: ${SMTP_TLS:-}
SMTP_TLS_INSECURE: ${SMTP_TLS_INSECURE:-false}
SMTP_EHLO_NAME: ${SMTP_EHLO_NAME:-}
GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID:-}
GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET:-}
GOOGLE_REDIRECT_URI: ${GOOGLE_REDIRECT_URI:-http://localhost:3000/auth/callback}
S3_BUCKET: ${S3_BUCKET:-}
S3_REGION: ${S3_REGION:-us-west-2}
AWS_ENDPOINT_URL: ${AWS_ENDPOINT_URL:-}
S3_USE_PATH_STYLE: ${S3_USE_PATH_STYLE:-}
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID:-}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY:-}
ATTACHMENT_DOWNLOAD_MODE: ${ATTACHMENT_DOWNLOAD_MODE:-auto}
ATTACHMENT_DOWNLOAD_URL_TTL: ${ATTACHMENT_DOWNLOAD_URL_TTL:-30m}
CLOUDFRONT_DOMAIN: ${CLOUDFRONT_DOMAIN:-}
CLOUDFRONT_KEY_PAIR_ID: ${CLOUDFRONT_KEY_PAIR_ID:-}
CLOUDFRONT_PRIVATE_KEY: ${CLOUDFRONT_PRIVATE_KEY:-}
COOKIE_DOMAIN: ${COOKIE_DOMAIN:-}
APP_ENV: ${APP_ENV:-production}
MULTICA_DEV_VERIFICATION_CODE: ${MULTICA_DEV_VERIFICATION_CODE:-}
MULTICA_APP_URL: ${MULTICA_APP_URL:-http://localhost:3000}
MULTICA_SHUTDOWN_HOLD_DURATION: ${MULTICA_SHUTDOWN_HOLD_DURATION:-}
ALLOW_SIGNUP: ${ALLOW_SIGNUP:-true}
ALLOWED_EMAILS: ${ALLOWED_EMAILS:-}
ALLOWED_EMAIL_DOMAINS: ${ALLOWED_EMAIL_DOMAINS:-}
DISABLE_WORKSPACE_CREATION: ${DISABLE_WORKSPACE_CREATION:-}
GITHUB_APP_SLUG: ${GITHUB_APP_SLUG:-}
GITHUB_WEBHOOK_SECRET: ${GITHUB_WEBHOOK_SECRET:-}
# Public URL the API is reachable at from the open internet, no
# trailing slash. Used to mint absolute webhook URLs for autopilot
# webhook triggers. Leave unset behind a same-origin reverse proxy
# (e.g. plain localhost dev); the frontend will compose the URL
# from window.origin + webhook_path in that case. Headers are
# intentionally NOT used to derive this value, to avoid Host /
# X-Forwarded-Host spoofing on misconfigured proxies.
MULTICA_PUBLIC_URL: ${MULTICA_PUBLIC_URL:-}
# Comma-separated CIDRs whose source IP is allowed to set
# X-Forwarded-For / X-Real-IP for the webhook per-IP rate limiter.
# Empty default = headers ignored, RemoteAddr used. Set e.g.
# "127.0.0.1/32" when running behind a same-host reverse proxy.
MULTICA_TRUSTED_PROXIES: ${MULTICA_TRUSTED_PROXIES:-}
# Lark / Feishu bot integration. MULTICA_LARK_SECRET_KEY is the
# opt-in: unset = integration disabled. Mainland 飞书 and international
# Lark are auto-detected per installation and served side by side, so
# the two base-URL knobs should normally stay EMPTY. They are optional
# deployment-wide overrides that force every installation onto one host
# (proxy / mock / single-cloud staging). Upgrading from a setup that
# used https://open.larksuite.com here? The server relabels existing
# installs to region=lark on first boot, then you can clear them.
# See docs/lark-bot-integration.
MULTICA_LARK_SECRET_KEY: ${MULTICA_LARK_SECRET_KEY:-}
MULTICA_LARK_HTTP_BASE_URL: ${MULTICA_LARK_HTTP_BASE_URL:-}
MULTICA_LARK_CALLBACK_BASE_URL: ${MULTICA_LARK_CALLBACK_BASE_URL:-}
# Slack bot integration. MULTICA_SLACK_SECRET_KEY is the opt-in: unset =
# integration disabled. It decrypts the per-installation bot/app tokens,
# which are brought by each workspace via OAuth/BYO and stored encrypted
# in the database, so this single deployment-wide key is all the operator
# needs to set here.
MULTICA_SLACK_SECRET_KEY: ${MULTICA_SLACK_SECRET_KEY:-}
# Self-hosted Git provider integration (Forgejo / Gitea / GitLab). This
# is a self-host-only feature, so the compose file turns it on by default;
# the managed cloud leaves it unset (off). It still needs a valid
# MULTICA_VCS_SECRET_KEY below to actually work.
MULTICA_VCS_INTEGRATION_ENABLED: ${MULTICA_VCS_INTEGRATION_ENABLED:-true}
# VCS integration at-rest encryption key for token-based providers
MULTICA_VCS_SECRET_KEY: ${MULTICA_VCS_SECRET_KEY:-}
# WeCom smart-bot integration. MULTICA_WECOM_SECRET_KEY is the
# opt-in: unset = integration disabled. It encrypts each installation's
# smart-bot secret at rest. The server needs it to unseal a bot's
# subscribe credentials at connection time, and to seal the secret when
# an installation is created from the WeCom settings tab.
MULTICA_WECOM_SECRET_KEY: ${MULTICA_WECOM_SECRET_KEY:-}
# 1 = log every inbound and outbound WeCom frame, including the first
# 120 runes of each message body, so a real-device session can be
# checked against the server afterwards. Off unless set; turn it on only
# for a debugging session and unset it when that session ends, because
# what it records is user message content.
MULTICA_WECOM_TRACE: ${MULTICA_WECOM_TRACE:-}
restart: unless-stopped
frontend:
image: ${MULTICA_WEB_IMAGE:-ghcr.io/multica-ai/multica-web}:${MULTICA_IMAGE_TAG:-latest}
depends_on:
- backend
ports:
- "127.0.0.1:${FRONTEND_PORT:-3000}:3000"
environment:
HOSTNAME: "0.0.0.0"
REMOTE_API_URL: ${REMOTE_API_URL:-http://backend:8080}
DOCS_URL: ${DOCS_URL:-}
NEXT_PUBLIC_API_URL: ${NEXT_PUBLIC_API_URL:-}
NEXT_PUBLIC_WS_URL: ${NEXT_PUBLIC_WS_URL:-}
restart: unless-stopped
volumes:
pgdata:
backend_uploads:
Reference in New Issue View Git Blame Copy Permalink