Synchronize both language guides, API references and examples with the implemented authorization, storage and delivery behavior. Document migration requirements and accepted operational limits.
24 KiB
Storage Layout
KV Storage Layout
KV keeps only cache data and short-lived/ephemeral state. High-frequency writes (webhook dedup, delivery state, message tracking) live in D1 and only fall back to KV when D1 is unavailable or not yet migrated (see Storage decisions).
| Key Pattern | Value | TTL |
|---|---|---|
config:routes / config:groups |
Legacy (pre-D1) configuration, read once to seed empty D1 tables | Permanent |
config:d1-seeded |
Marks that D1 was seeded (an emptied D1 is never re-seeded) | Permanent |
session:{id} |
Admin session { userId, login, loginTrusted } |
1 day |
token:{userId} |
{ userId, accessToken, expiresAt, live? } (a token is used for 10 hours) |
9 hours |
token-reverse:{sha256} |
User id for reverse lookup by token | 9 hours |
state:{hex} |
{ redirectTo, expiresAt, link? } (link: the chat account a bot /gh login binds) |
600 seconds |
link:{nonce} |
A bot account link waiting for the GitHub user's confirmation | 600 seconds |
invite:{token} |
{ groupId, role, expiresAt, createdBy, note?, groupInstance, issuer } |
7 days |
invite:group:{id} |
Token index per group (keeps invite listing consistent) | 7 days |
invite:by-group:{id}:{token} / invite:by-instance:{instance}:{token} |
Per-invite listing markers (by group id and by the issuing group's instance), so concurrent creates or a half-finished rename never hide an invite | Invite lifetime |
login-owner:{login} |
GitHub user id of the first account seen with this login; another account using the login later is matched by user id only | Permanent |
delivery:{provider}:{scope}:{id} |
Webhook delivery dedup fallback (D1 dedup_keys is primary); scope is @global for the global endpoint or @i:{instance} (the receiving group's immutable instance) for a group endpoint |
2–10 minutes |
delivery-state:{provider}:{scope}:{id} |
Queue delivery state fallback (D1 delivery_state is primary) |
7 days |
queue:payload:{provider}:{scope}:{id}:{uuid} |
Oversized webhook payload parked for the queue consumer when there is no R2 binding (one key per enqueue) | 4 days |
tenant:{groupId} / tenant:i:{instance} |
Legacy per-group webhook secrets, read only for groups whose secret was never moved into their D1 row; removed on rotate/delete | Permanent |
group:renamed:{oldId} |
KV-only deployments: where a renamed group went (with D1 this alias is a dedup_keys row written in the rename transaction) |
4 days |
msg:{chain}:{platform}:{target} |
Message tracking fallback (D1 message_tracking is primary) |
30 days |
msg:lock:… |
Lock serializing updates of one message chain when D1 is unavailable (with D1 it is a dedup_keys row) |
240 seconds |
storage:kv-fallback |
Marks that state was written to KV while a D1 table was missing; while it lives, D1 misses are also looked up in KV | 35 days |
cmd:guilds:{appId} |
Per Discord application: each guild's registered command hash and the sync cursor | Permanent |
cmd:registered:global:{appId} |
Hash of the commands last registered globally (re-registered when it changes, and daily) | 1 day |
config:discord-app-id:{tokenHash} |
Discord application id resolved for a bot token | Permanent |
discord-modal:{id} |
Comment-modal context, only when it does not fit in the 100-character custom_id |
1 day |
feishu:token:{appId} |
Feishu tenant_access_token per app (also held in memory) |
Token expiry − 6 minutes |
telegram:webhook:{botId} |
Hash of the webhook registration last set (setWebhook is skipped while it matches) |
1 day |
i18n:{lang} |
Translation overrides merged on top of English | Permanent |
config:identity-migrated |
KV-only deployments: group identities are managed by this server (the one-time identity migration ran) | Permanent |
D1 Storage Layout
The D1 database (DB binding, database webhooker) holds the source of truth for configuration, delivery logs and high-frequency ephemeral state:
| Table | Purpose |
|---|---|
d1_groups |
Groups (authoritative config, seeded from legacy KV config:groups). Each row also holds server-only fields that are never sent to the console: the immutable instance id, legacyId, and the group's webhook secret |
d1_routes |
Routes per group (authoritative config, seeded from legacy KV config:routes) |
send_logs |
One row per dispatch attempt (route id, event, target, ok/error, duration, error code, detail) |
audit_logs |
One row per admin operation (login/logout, group/route/member/invite changes) |
dedup_keys |
Atomic one-time claims and records (INSERT ... ON CONFLICT UPSERT, key + expiry): webhook delivery dedup and its confirmations (accepted:*), custom-webhook nonces (nonce:{secret-scope}:{nonce}), single-use and revoked invites (invite-used:*, invite-revoked:*), bot command/event dedup (bot:{platform}:…), revoked sessions and tokens and each user's live token (session-revoked:*, token-revoked:*, token-live:*), used account-link confirmations (link-used:*), retired group ids (group-retired:*) and rename aliases (group-alias:*), installation provisioning records (installation-provisioned:*, installation-bound:*), route × targets already finished or failed for good by a queued delivery (delivery-target:*, delivery-failed:*), group log summaries already sent, message-chain locks (msg:lock:*) and one-time setup markers |
delivery_state |
Queue delivery state (pending/processing/delivered/retrying/failed/dead); taking a delivery is a conditional UPSERT (a lease), so two consumers never dispatch it at once |
message_tracking |
Message tracking for in-place updates (event_id + target_id → message id and the version it shows); chains are keyed by the group's instance, so a group created later under the same id never edits its messages |
d1_fragments |
Named filter fragments per group |
feishu_links |
Maps feishu_user_id → github_user_id for /gh Feishu commands |
discord_links |
Maps discord_user_id → github_user_id for /gh Discord commands |
telegram_links |
Maps telegram_user_id → github_user_id for /gh Telegram commands |
audit_logs is pruned by the scheduled trigger after AUDIT_RETENTION_DAYS (default 90). The storage-prune task removes expired dedup_keys, delivery_state rows older than 7 days, message_tracking rows not updated for 30 days (every edit refreshes a row) and orphaned R2 payloads. send_logs is not pruned automatically. See Logs for the log row fields.
Migration 0013_prune_indexes.sql adds the indexes these reads rely on: delivery_state(updated_at) for the prune, two partial indexes on failed send_logs rows for the metrics page, and audit_logs(group_id, ts) in place of the single-column group index. Every index also costs a D1 row write when its columns are written, so only indexes whose reads would otherwise cost far more are added.
R2 Storage Layout
R2 (PAYLOAD binding, bucket webhooker-payloads) stores oversized webhook payloads that are too large for a queue message or KV:
| Object Pattern | Purpose | Retention |
|---|---|---|
webhooks/YYYY/MM/DD/<uuid>.json |
Oversized payload parked for the queue consumer | deleted after dispatch; orphans pruned after 5 days |
A payload is parked when the queue message would exceed 120,000 bytes (UTF-8). When the PAYLOAD binding is absent, oversized payloads fall back to the KV key queue:payload:{provider}:{scope}:{id}:{uuid} (4 day TTL, matching the queue's message retention). If sending the queue message fails, the parked copy is kept (the queue may have accepted the message even though the call failed); unused copies expire or are pruned. A parked payload that is gone when the consumer needs it is never dispatched as an empty event: the delivery stays retrying and a PAYLOAD_MISSING send-log entry asks for the webhook to be redelivered from its source.
Storage Decisions
- D1 is authoritative for config and delivery metadata; KV holds only caches and short-lived state.
- The
canUseD1probe (server/lib/storage/d1.ts, checks forprepare+batch) gates every D1 store. The high-frequency stores (dedup and other one-time claims, delivery state, message tracking) also fall back to KV while their D1 table does not exist yet (an unmigrated database). Any other D1 failure propagates — the request answers with an error and the sender or queue retries — because switching stores per call would split a claim between D1 and KV. - When D1 configuration reads fail, the request answers with an error and the sender or queue retries; there is no KV copy of the configuration, and dispatch never treats unreadable configuration as empty. A stored route or group row that cannot be read only makes its own group's deliveries fail visibly (and retry) — see Routes → Unreadable routes.
- Configuration writes never fall back: with a D1 binding, a failed write is reported as an error instead of being parked in KV. Writes are per group (one row, or one group's routes/fragments in one transaction) with compare-and-swap on the group row, so concurrent edits of different groups never overwrite each other. Reads for access checks, read-modify-write paths, webhook ingress and the queue consumer bypass every cache (a revoked route or target stops at once); other reads are cached in memory for at most 60 seconds per isolate.
- Legacy KV configuration is copied into D1 once, only while both D1 tables are empty, and never over existing rows; a marker prevents an emptied configuration from being resurrected.
- This keeps per-event KV writes near zero on the Workers Free plan (1,000 writes/day): dedup, delivery state and message tracking now write D1 rows instead (D1 Free allows 100,000 rows written/day).
- R2's free tier (10 GB-month storage, 1M Class A ops/month) comfortably absorbs payload parking without touching the KV write quota.