mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-25 20:15:37 +02:00
* feat(integrations): add platform-agnostic channel foundation Introduce server/internal/integrations/channel — the contract every inbound IM integration implements, so the core never learns a platform's event JSON. Four pieces: - Channel interface (Type/Connect/Disconnect/Send/Capabilities) + Factory + Config (channel_type + opaque JSON blob, maps to channel_installation). - Normalized InboundMessage/OutboundMessage envelopes + Source/MediaRef/ ReplyCtx/MsgType/ChatType. Envelope holds only cross-platform-true fields; platform specifics live in Raw, read only by the adapter. - Capability bitmask: declaration only, no degrade logic in core. - Registry: Type->Factory map, last-writer-wins, concurrency-safe. Pure package (no DB/network/platform deps). Foundation for MUL-3515; the lark cutover + lark_*->channel_* generalization land in follow-up PRs. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * feat(channel): generalize lark_* tables into channel_* (DB layer) Migration 123 creates channel_installation / channel_user_binding / channel_chat_session_binding / channel_inbound_message_dedup / channel_inbound_audit / channel_outbound_card_message / channel_binding_token. Each carries a channel_type discriminator and a JSONB config for platform-specific identifiers/credentials; cross-platform columns stay flat. Existing Feishu rows are backfilled (channel_type= 'feishu', app_secret_encrypted via base64). NO foreign keys / cascades (MUL-3515 §4) — integrity moves to the app layer in the cutover. queries/channel.sql ports the lark query surface to channel_*, JSONB-aware, plus DeleteChannelUserBindingsByWorkspaceMember / DeleteChannelChatSessionBindingBySession for the app-layer cleanup that replaces the removed cascades. lark_* tables/queries are left in place here and removed once the Go cutover lands, so this commit ships green on its own. Verified: sqlc generate, go build ./..., full migrate chain (1..123) on Postgres 17, and a real-data backfill spot-check (base64 round-trip, NULL-strip, functional unique index on (channel_type, app_id)). MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * fix(channel): name app_id query param + multi-IM install key + null-safe binding merge Addresses review on MUL-3515 (PR #4412): - GetChannelInstallationByAppID: explicitly name params and cast app_id to ::text so sqlc emits AppID string. A bare $2 next to `config ->> 'app_id'` was mis-attributed to the JSONB config column, generating Config []byte. - channel_installation uniqueness -> (workspace_id, agent_id, channel_type), with the UpsertChannelInstallation conflict key matched. Lets one agent hold one installation per IM (feishu + slack + ...) instead of a later install clobbering an earlier one. Behaviorally identical in the current feishu-only world; "one agent, at most one IM overall" stays an app-layer rule per MUL-3515 §4, not a DB constraint. - CreateChannelUserBinding merges jsonb_strip_nulls(EXCLUDED.config) so a re-bind carrying {"union_id": null} no longer erases an already-captured union_id, restoring the old COALESCE(EXCLUDED.union_id, ...) semantics. Regenerated with sqlc v1.31.1. Verified on PG17: re-install replaces in place, feishu+slack coexist, null re-bind keeps union_id, real union_id wins. Co-authored-by: multica-agent <github@multica.ai> * feat(lark): channel-backed Feishu store + fix base64 backfill wrapping Cutover step 1 of switching the lark Go code from lark_* onto the channel_* tables (MUL-3515). Introduces the JSONB config boundary the rest of the cutover sits on, and fixes a latent backfill bug surfaced while building it. - migration 123: strip newlines from the app_secret_encrypted base64 backfill. PostgreSQL encode(...,'base64') MIME-wraps at 76 chars, and a secretbox- sealed ~72-byte secret exceeds that. Go's encoding/json decodes a JSON string into []byte with base64.StdEncoding, which rejects embedded newlines, so without the strip every migrated installation would fail to decrypt its app secret once reads move to channel_installation.config. - store.go: flat domain types (Installation / UserBinding / ChatSessionBinding) with field parity to the retired db.Lark* rows, plus the feishu config codec. Row->domain mappers decode the JSONB config; the secret decoder is whitespace-tolerant so legacy MIME-wrapped data still round-trips, while the encoder emits unwrapped base64. Binding config encodes an absent union_id as "{}" so the upsert's jsonb_strip_nulls merge never clobbers a stored union_id. - store_test.go: 72-byte secret round-trip, MIME-wrapped tolerance, optional null-strip, and flat-column preservation. Verified on PG17. Field parity keeps the upcoming ~190 db.LarkInstallation call sites a mechanical rename. No call sites switched yet; behavior unchanged. Co-authored-by: multica-agent <github@multica.ai> * feat(lark): route inbound integration onto channel_* + explicit membership checks Cutover step 2 (MUL-3515): switch the Feishu Go code from the lark_* queries to channel_* via a ChannelStore adapter, and replace the removed member foreign key with explicit application-layer membership checks. No user-visible behavior change. - channel_store.go: ChannelStore embeds *db.Queries and SHADOWS the ~24 lark query methods with channel_*-backed equivalents, keeping the db.Lark* signatures so the dispatcher/hub/services and their ~20k lines of tests stay untouched; the feishu JSONB config is (de)coded by store.go. Adds IsWorkspaceMember and a tx-aware WithTx. Only production wiring swaps *db.Queries for *ChannelStore. - Membership re-check (§4 removed the lark_user_binding -> member FK, so a binding row no longer proves current membership): * the dispatcher inbound identity step verifies membership after the binding lookup; a former member's stale binding is dropped as non_workspace_member + audited and never reaches chat_session (§4.3 safety property). * RedeemAndBind and BindInstallerTx replace the now-dead FK (23503) branch with an explicit IsWorkspaceMember gate, preserving the existing ErrBindingNotWorkspaceMember outcome without burning the token. - router wires the ChannelStore into the patcher, typing indicator, dispatcher, hub, and the union_id/region backfills; constructor-based services wrap *db.Queries internally so their signatures and nil-check tests are unchanged. Verified: go build ./... ; go vet ; gofmt ; go test -race ./internal/integrations/... (full lark suite green unchanged + new membership drop/error tests). Adapter field mappings (secret base64, union_id RMW, chat-id/open-id remaps, dedup, token, card) checked end-to-end against a PG17 channel_* schema. lark_* tables and queries remain (unused at runtime) until the S3 cleanup-hooks and S4 drop-tables/rename commits. Co-authored-by: multica-agent <github@multica.ai> * fix(channel): renumber generalization migration 123 -> 124 main merged 123_issue_stage after this branch forked, so the branch's 123_channel_generalization now collides on the migration number. The runner keys schema_migrations by full version string and would still apply both, but a duplicate number is a merge hazard and convention violation, so move the channel migration to the next free slot (124). issue_stage (ALTER issue ADD COLUMN stage) and the channel generalization touch disjoint tables; verified on PG17 that 123_issue_stage applies cleanly on a DB already carrying 124_channel_generalization, so the two are order-independent. sqlc regenerated (v1.31.1): only the migration-number comment changed. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * feat(channel): prune channel bindings on member removal + chat session delete MUL-3515 §4 dropped every channel_* foreign key, so the old ON DELETE CASCADE that cleared a user's channel_user_binding when they left a workspace, and a chat's channel_chat_session_binding when its chat_session was deleted, no longer fires. Re-establish that integrity in the application layer, inside the existing transactions: revokeAndRemoveMember -> DeleteChannelUserBindingsByWorkspaceMember, DeleteChatSession -> DeleteChannelChatSessionBindingBySession. Adds real-DB tests for both paths, including a scoping check that a remaining member's binding survives the prune. Verified on PG17: both new tests plus the existing revocation tests and the full handler package pass. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * fix(channel): scope Lark/Feishu store reads to channel_type='feishu' The S2 cutover routed the Feishu integration onto channel_*, but the Lark-facing ChannelStore wrappers read installation / chat-session-binding / outbound-card rows across ALL channel_type values. Once a second IM exists, that would let the Lark hub supervise a non-Feishu installation, the Lark install list show it, /lark/installations/{id} revoke another channel's row, and the outbound patcher / typing indicator act on a non-Feishu chat binding or card. Add a channel_type predicate to the six read/list channel queries and pass channelTypeFeishu from every wrapper: GetChannelInstallation, GetChannelInstallationInWorkspace, ListChannelInstallationsByWorkspace, ListActiveChannelInstallations, GetChannelChatSessionBindingBySession, GetChannelOutboundCardByTask. The S3 cleanup deletes (DeleteChannelUserBindingsByWorkspaceMember / DeleteChannelChatSessionBindingBySession) stay all-channel on purpose: a member leaving or a chat_session being deleted should clear every IM's binding. Adds a real-DB test that seeds a Slack installation/binding/card next to the Feishu ones and asserts the Lark wrappers never return them. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * refactor(channel): replace db.Lark* translation layer with lark domain types S2 introduced ChannelStore as a translation layer that read/wrote channel_* but kept the retired db.Lark* struct/param shapes so the dispatcher/hub/services and their ~20k lines of tests did not have to change. This collapses that layer: the store now takes and returns the package's flat domain types (Installation, UserBinding, ChatSessionBinding, InboundMessageDedup, BindingTokenRow, OutboundCardMessage) and the *Params types in params.go, with channel-neutral field names (ChannelUserID / ChannelChatID / ...). All call sites, fakes, and tests move to the domain types. No behavior change: only channel_* is read/written (as before); db.Lark* is now unused, and the lark_* tables + queries/lark.sql are removed in the next commit. Verified on PG17: go build / vet / gofmt clean, go test -race ./internal/integrations/... green (the ~20k-line fake suite), and the lark + handler suites pass. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * refactor(channel): drop lark_* tables and queries (remove old path) The Go cutover (previous commit) moved the lark package entirely onto channel_* and the domain types, leaving the lark_* tables, queries/lark.sql, and the generated db.Lark* models unused. Remove them per the design (§5: replace, do not keep both): migration 125 drops the seven lark_* tables (data already lives in channel_* since migration 124), and queries/lark.sql is deleted + sqlc regenerated, removing the db.Lark* models and lark query methods. The 125 down recreates the authoritative pre-drop schema (bot_union_id, region, per-installation dedup PK, thread-reply columns). Verified on PG17: fresh migrate up ends with lark_* gone + channel_* present; isolated 125 down/up round-trips correctly; go build / vet / gofmt clean; go test -race ./internal/integrations/... and the handler suite pass. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * fix(migrations): remove trailing blank line at EOF of 125 down migration git diff --check flagged a blank line at EOF of 125_drop_lark_tables.down.sql (a pg_dump-generation artifact). Whitespace only; the recreate SQL is unchanged. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * refactor(channel): defer lark_* table drop to a follow-up migration Preflight deploy review: dropping lark_* in the same release that cuts over (old migration 125) is not rollback/rolling-safe — the v0.3.27 release still reads lark_*, so a rolling deploy or a post-deploy code rollback would hit "relation does not exist". Remove the drop and keep the old tables for one release (standard expand/contract): migration 124 already backfilled lark_* -> channel_*, the new code reads/writes only channel_*, and the physical drop moves to a separate cleanup migration once this ships and is observed. The lark_* tables remain in the schema, so sqlc regenerates the (now unused) db.Lark* models; queries/lark.sql stays deleted (the new code uses channel_*). No code path reads lark_* — only the destructive drop is deferred, keeping the design's no-compat-layer / no-dual-write rule while being deploy-safe. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * fix(channel): skip orphaned installations in hub-boot active scan Preflight deploy review: channel_installation dropped the workspace/agent FK (MUL-3515 §4), so unlike lark_installation it does not cascade away when its workspace is deleted or its agent is hard-deleted (e.g. runtime teardown). The hub-boot query then keeps opening a WebSocket for a bot whose owner is gone. JOIN ListActiveChannelInstallations to live workspace + agent so an orphaned installation is never connected, uniformly for every deletion path. The JOIN matches the old ON DELETE CASCADE semantics (row existence, not agent archival), so an archived-but-present agent's installation is still listed; the orphaned row's encrypted secret is thereby never decrypted/used. Tests: a real-DB handler test asserts a deleted-workspace/agent installation and a non-Feishu one are both excluded; the lark scope test's active-list assertion moved there since the JOIN now needs real workspace/agent fixtures. (Physically deleting dormant orphaned channel rows on workspace/agent deletion is a separate app-layer-cleanup follow-up.) MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * docs(channel): document non-rolling cutover constraint for the lark->channel migration Elon deploy review: keeping the lark_* tables (deferred drop) stops old v0.3.27 code from crashing, but is not full expand/contract. Migration 124 is a one-time backfill; afterwards new code runs on channel_* (lease + dedup on channel_*) while pre-cutover code runs on lark_* (lease + dedup on lark_*). If both run concurrently during a rolling deploy, each side claims the same Feishu bot's WS lease on its own table and double-processes inbound events. This release therefore requires a NON-ROLLING cutover (stop the old hub before applying migration 124 + starting new code; rollback is not lossless once new code writes channel_*). Documented where deployers/reviewers see it: migration 124 header gains a ROLLOUT note; the channel_store.go header is corrected (lark_* tables are retained one release for rollback safety, not "gone"; the store still never touches them). Comment-only — no schema/codegen/behavior change. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * feat(lark): add MULTICA_LARK_HUB_DISABLED switch for the channel cutover The lark_*->channel_* cutover needs a way to make the Feishu bot briefly unavailable WITHOUT taking down the whole multica-api process — the Lark hub is a goroutine inside it, not a separate Deployment. MULTICA_LARK_HUB_DISABLED=true parks the hub at startup: the API serves HTTP normally but never claims a WS lease or opens a Feishu connection. Rollout (see migration 124 ROLLOUT note): ship the new release with the flag SET so new pods run API-only while old pods (hub on lark_*) drain during the rolling deploy — the two hubs never overlap. After the old pods are gone and migration 124 has run, flip the flag off; the new hub comes up on channel_*. The old backend does NOT need this switch — its hub stops when k8s terminates the old pods, not via a flag. Nil-ing LarkHub reuses the existing not-configured path so both the startup start and the shutdown join skip it. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * docs(channel): point migration 124 ROLLOUT note at the hub-disable switch Refine the rollout note to use MULTICA_LARK_HUB_DISABLED for a bot-only cutover (new pods serve API with the hub parked while old pods drain; flip the switch off after the migration), instead of the earlier whole-API recreate. Comment-only. MUL-3515 Co-authored-by: multica-agent <github@multica.ai> * docs(channel): fix migration 124 rollout order and document self-host cutover The previous ROLLOUT note shipped the new (channel_*) build before running migration 124, so the channel_*-backed HTTP paths (installation list/install/revoke, chat-session delete, member revoke) would 500 in the window between new-pod boot and the deferred migration. Restate the runbook around two explicit invariants — channel_* must exist before the new build serves those paths, and the old/new hubs must never overlap — and order the steps so channel_* is created first (park old hub -> snapshot -> deploy parked new build -> unpark). Document that default self-host (entrypoint migrate + single-replica Recreate) satisfies both invariants automatically and needs no manual steps; only prd / multi-replica rolling self-host needs the switch procedure. Clarify in main.go that the hub-park switch is generation-agnostic (parks whichever hub the build carries), which is what enables the preparatory release. Refs MUL-3515 Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai>
176 lines
6.9 KiB
Go
176 lines
6.9 KiB
Go
package channel
|
|
|
|
import "encoding/json"
|
|
|
|
// ChatType discriminates a 1:1 direct conversation with the bot from a
|
|
// multi-party group chat. Product behavior differs: direct chats ingest
|
|
// every message; group chats only ingest messages explicitly addressed
|
|
// to the bot (@-mention or reply to a bot message). The wire values match
|
|
// the existing lark_chat_session_binding.lark_chat_type constraint so the
|
|
// generalized channel_* table backfills 1:1.
|
|
type ChatType string
|
|
|
|
const (
|
|
// ChatTypeP2P is a direct (peer-to-peer) conversation with the bot.
|
|
ChatTypeP2P ChatType = "p2p"
|
|
// ChatTypeGroup is a multi-party group conversation.
|
|
ChatTypeGroup ChatType = "group"
|
|
)
|
|
|
|
// MsgType is the normalized, cross-platform message kind. Adapters map
|
|
// their platform's native type onto this small closed set; the platform's
|
|
// raw type string (Lark "post" / "merge_forward" / "interactive", …) is
|
|
// NOT represented here — it stays in InboundMessage.Raw and is read only
|
|
// by the adapter. The core only ever needs to know "text vs media, and
|
|
// which media".
|
|
type MsgType string
|
|
|
|
const (
|
|
// MsgTypeText is a plain or rich text message. The human-readable
|
|
// content is flattened into InboundMessage.Text by the adapter.
|
|
MsgTypeText MsgType = "text"
|
|
// MsgTypeImage is an image attachment.
|
|
MsgTypeImage MsgType = "image"
|
|
// MsgTypeFile is a generic file attachment.
|
|
MsgTypeFile MsgType = "file"
|
|
// MsgTypeAudio is a voice / audio attachment.
|
|
MsgTypeAudio MsgType = "audio"
|
|
// MsgTypeVideo is a video attachment.
|
|
MsgTypeVideo MsgType = "video"
|
|
// MsgTypeUnknown is the fallback for a platform type the adapter does
|
|
// not map. The core treats it as a non-text, non-actionable message.
|
|
MsgTypeUnknown MsgType = "unknown"
|
|
)
|
|
|
|
// Source carries the cross-platform routing identity of an inbound
|
|
// message — every field here is true on every platform. Platform-specific
|
|
// routing keys (a Lark app_id, a Slack team id) are resolved to an
|
|
// installation by the adapter and do NOT appear on Source.
|
|
type Source struct {
|
|
// ChannelType is the platform the message arrived on; it equals the
|
|
// owning Channel's Type.
|
|
ChannelType Type
|
|
|
|
// ChatID is the platform conversation identifier. One ChatID maps to
|
|
// one Multica chat_session via the channel_chat_session_binding.
|
|
ChatID string
|
|
|
|
// ChatType discriminates direct from group conversations.
|
|
ChatType ChatType
|
|
|
|
// SenderID is the platform-native, per-installation user identifier
|
|
// (Lark open_id, Slack user id, …). It is stable WITHIN one
|
|
// installation and is the key the identity binding is stored under.
|
|
// It is NOT comparable across installations.
|
|
SenderID string
|
|
|
|
// SenderStableID is the platform's cross-installation stable identity
|
|
// for the sender when one exists (Lark union_id, …), otherwise empty.
|
|
// Captured opportunistically for future cross-installation identity
|
|
// merging; the core treats an empty value as "not available".
|
|
SenderStableID string
|
|
|
|
// ThreadID is the platform thread / topic the message belongs to,
|
|
// when threading applies and the message is inside a thread. Empty
|
|
// means a top-level conversation message. The core persists it so a
|
|
// decoupled outbound reply can be threaded back into the same topic.
|
|
ThreadID string
|
|
}
|
|
|
|
// MediaRef references a media attachment that the adapter has ALREADY
|
|
// persisted to object storage before the message reaches the core. The
|
|
// core never holds raw bytes — only this reference — so the envelope
|
|
// stays small and platform-neutral.
|
|
type MediaRef struct {
|
|
// Type is the normalized media kind (image / file / audio / video).
|
|
Type MsgType
|
|
// StorageKey locates the persisted object in Multica object storage.
|
|
StorageKey string
|
|
// Filename is the original display name, when the platform supplies
|
|
// one.
|
|
Filename string
|
|
// MimeType is the content type, when known.
|
|
MimeType string
|
|
// SizeBytes is the object size in bytes, or 0 when unknown.
|
|
SizeBytes int64
|
|
}
|
|
|
|
// ReplyCtx describes the message an inbound message quotes / replies to.
|
|
// It is nil when the inbound message is not a reply.
|
|
type ReplyCtx struct {
|
|
// MessageID is the immediate parent message's platform id (the
|
|
// message being quoted).
|
|
MessageID string
|
|
// RootID is the thread/root anchor the platform reports, when any.
|
|
RootID string
|
|
}
|
|
|
|
// InboundMessage is the single normalized shape the core consumes. Every
|
|
// adapter translates its platform's raw payload into this struct; the
|
|
// core's router, dedup, identity check, and persistence read ONLY these
|
|
// fields. Per the boundary rule (MUL-3515 §2) the struct holds only
|
|
// cross-platform-true fields; everything platform-specific lives in Raw.
|
|
type InboundMessage struct {
|
|
// EventID is the platform's delivery/event identifier and MessageID
|
|
// is the platform's message identifier. Together they back the
|
|
// idempotency layer: a platform may redeliver the same event on
|
|
// reconnect, and dedup keys on (installation, MessageID).
|
|
EventID string
|
|
MessageID string
|
|
|
|
// Source is the routing identity (chat, sender, thread).
|
|
Source Source
|
|
|
|
// Type is the normalized message kind.
|
|
Type MsgType
|
|
|
|
// Text is the human-readable content, flattened by the adapter. For
|
|
// non-text messages it may be empty or a short placeholder; the media
|
|
// itself is in MediaRefs.
|
|
Text string
|
|
|
|
// MediaRefs are the attachments, already persisted to object storage.
|
|
MediaRefs []MediaRef
|
|
|
|
// ReplyTo is the quoted/replied-to context, or nil.
|
|
ReplyTo *ReplyCtx
|
|
|
|
// AddressedToBot is the adapter's normalized verdict on whether a
|
|
// GROUP message is an interaction with the bot (@-mention or reply to
|
|
// a bot message). It is meaningless for direct (p2p) chats and the
|
|
// core ignores it there. It is a normalized boolean, not platform
|
|
// data — the platform-specific signals it was derived from (mention
|
|
// arrays, parent ids) stay in Raw.
|
|
AddressedToBot bool
|
|
|
|
// Raw is the untouched platform payload. Adapters stash platform-
|
|
// specific fields here (Lark raw msg_type / parent_id / root_id /
|
|
// mention arrays, …) and read them back only inside the adapter. The
|
|
// core never reads Raw — that is the whole point of the boundary.
|
|
Raw json.RawMessage
|
|
}
|
|
|
|
// OutboundMessage is the minimal outbound reply the core can ask any
|
|
// Channel to deliver: a text body into a chat, optionally threaded or
|
|
// quoting a specific message. Rich cards, media uploads, and outbound
|
|
// webhooks are deliberately NOT modeled here (MUL-3515 decision §6) — an
|
|
// adapter that supports richer output exposes it on its own type, not on
|
|
// this cross-platform envelope.
|
|
type OutboundMessage struct {
|
|
// ChatID is the destination conversation (the platform chat id).
|
|
ChatID string
|
|
// Text is the message body.
|
|
Text string
|
|
// ThreadID, when set, threads the reply into the given platform
|
|
// thread / topic. Empty sends at the chat level.
|
|
ThreadID string
|
|
// ReplyTo, when set, quote-replies to the given platform message id.
|
|
ReplyTo string
|
|
}
|
|
|
|
// SendResult is the outcome of Channel.Send.
|
|
type SendResult struct {
|
|
// MessageID is the platform's identifier for the delivered message.
|
|
MessageID string
|
|
}
|