mirror of
https://github.com/Elysola/WebHooker.git
synced 2026-11-20 17:13:49 +00:00
fix(security): harden webhook delivery pipeline and group storage (#11)
* fix(log): preserve structured error details Error objects lose their useful details when serialized as plain JSON. Keep the message, stack and cause in structured log output. * fix(runtime): bound shared in-flight waits A canceled request can leave its shared promise pending forever. Bound joins and coordinate a single replacement so later requests make progress without multiplying upstream work. * fix(bot): reject oversized callback bodies Reject declared oversized bot callbacks before reading their bodies. This keeps unauthenticated requests from wasting worker resources. * test(d1): exercise real sqlite transactions Exercise real foreign keys, transactions and conditional statements instead of mocks that cannot reproduce lost updates or cascades. * fix(storage): preserve claims across kv fallback Only missing tables justify KV fallback; other failures must surface. Remember fallback writes so migration to D1 cannot forget claims and replay completed work. Share marker checks without extra writes. * fix(filter): use isolated linear-time patterns Backtracking patterns can exhaust worker CPU on untrusted payloads. Use linear-time matching and separate bounded cache namespaces so glob and regex entries cannot corrupt each other. * fix(filter): read each event branch correctly Deployment and check branches live in event-specific nested fields. Read those fields and remove only the actual leading ref prefix. * fix(filter): read only own payload properties Inherited object properties are not webhook payload data. Read own properties only so JSONPath cannot traverse prototypes. * fix(filter): validate executable condition trees Malformed trees and unsupported patterns must not silently turn an exclude rule into a pass. Validate the shape and only the patterns the evaluator actually executes. * fix(i18n): bound and cache translation loads Repeated misses and simultaneous override loads spend KV reads for the same language. Cache confirmed misses and share bounded loads without caching read failures as missing translations. * i18n(bot): explain command and account failures Explain account linking, command failures and skipped delivery outcomes consistently in both supported languages. * fix(github): bound and reuse app token fetches Repeated token exchanges waste requests and JWT work for every event. Reuse valid installation tokens, share refreshes and back off briefly after failures while keeping every shared wait bounded. * fix(github): restrict enrichment request urls Webhook-provided URLs must never receive an installation token on another host. Restrict enrichment to GitHub repository API paths, reject redirects and bound optional requests. * fix(formatter): preserve invalid code points Out-of-range numeric entities can throw while formatting a delivery. Keep invalid code points verbatim instead of aborting the message. * fix(formatter): preserve bounded markdown text Code spans, fences, indentation and autolinks are Markdown content, not HTML tags to strip. Preserve them while bounding cleanup input and reusing results across routes for the same event. * fix(formatter): strip only leading ref prefixes A short branch or tag may itself contain refs/heads or refs/tags. Strip one leading full-ref prefix at the caller and keep short names unchanged when generating labels and links. * fix(gitea): link push authors to their forge A Gitea author is not the same-named GitHub account. Build author profile links from the originating forge. * fix(formatter): render deployment state values The deployment status object stringifies as [object Object]. Render its actual state in the status field. * fix(formatter): read security severity levels The scanner severity is not the security severity of an alert. Prefer the security level so critical and high alerts get the right label and color. * fix(formatter): hide actions on other forges Action buttons call the GitHub API as the linked user. Hide them for other forges to avoid operating on a same-named GitHub repository. * fix(check): isolate updates by check suite Same-named checks from different suites must not edit one message. Include the suite or app identity and retain the former key for controlled adoption of existing tracked messages. * fix(logs): scope queries before limiting results Filter authorized groups in SQL before LIMIT and aggregate only visible rows. Preserve current group attribution for late writes, surface read failures and combine related totals in one scan. * perf(d1): index pruning and failed send queries Pruning and failure views should not scan all historical rows. Index the high-value access paths without adding a write-heavy index for infrequent delivery-detail lookups. * fix(discord): bound individual api requests An unbounded API request can outlive an update-chain lock. Bound each request so another holder cannot race a late edit. * fix(discord): defer long rate-limit waits Long rate-limit sleeps block delivery progress inside a worker. Return a retryable 429 for the queue when the wait exceeds the bound. * fix(discord): encode channel path segments Stored channel and message ids are path segments, not URL paths. Encode them so aliases cannot address another Discord resource. * fix(discord): reuse nonces after lost responses A successful create may lose its response. Reuse a stable nonce with enforce_nonce so retries recover that message instead of posting duplicates or accepting an unreadable success. * fix(discord): constrain embeds and role mentions Respect combined embed and component limits as well as field caps. Restrict allowed mentions to validated route roles so payload text cannot notify everyone. * fix(discord): resume bounded command syncs Application changes, command rollbacks and large guild lists can leave commands stale indefinitely. Track the current hash per app and resume bounded paginated work without needless KV writes. * fix(github): verify installation management An installation id is user-controlled, and repository access is not installation management. Verify the owning account or an active organization admin using the user token and read:org scope. * fix(github): resolve immutable member ids A login can be renamed and reclaimed before its first sign-in. Resolve the current immutable user id when membership is granted so a future holder of the login cannot inherit access. * fix(auth): revoke stale sessions and tokens Revoked or rotated credentials must stop working despite stale KV replicas. Track the live token and revocations in D1 and preserve immutable identity bindings rather than trusting a recycled login. * fix(auth): bind sessions and tokens to identity Automatic retries can repeat non-idempotent user actions. Disable those retries and expose a write hook so bot handlers can record completion immediately before the GitHub write. * fix(discord): defer and deduplicate interactions Acknowledge slow actions before Discord times out and deduplicate interaction writes. Preserve comment text and reject edits whose source changed after the modal opened. * fix(telegram): always verify webhook secrets Webhook verification must not silently disable itself when the optional secret is absent. Derive a stable secret from the bot token and reject callbacks when verification is unavailable. * fix(telegram): bound individual api requests Bound individual platform calls so they cannot outlive the lock protecting a message update or stall a whole delivery invocation. * fix(telegram): honor nested rate-limit delays Telegram places retry_after inside parameters, not at the top level. Read that value and defer long waits to the delivery queue. * fix(telegram): report unreadable send results An unreadable successful response may already have sent the message. Report an explicit uncertain result instead of retrying a create without an idempotency key or pretending it succeeded. * fix(telegram): keep editable messages as text Guessing photo versus text from message content makes later edits use the wrong API and a smaller caption budget. Always send text and use the avatar as a link preview. * fix(telegram): render hard breaks as newlines Telegram HTML does not support br tags. Render Markdown hard breaks as plain newlines so otherwise valid messages are accepted. * fix(telegram): clamp visible html characters Telegram counts visible characters, not HTML source bytes. Truncate without splitting tags, entities or surrogate pairs. * fix(telegram): honor the author emoji toggle The author marker must obey the group emoji preference just like formatter-added decoration. * fix(telegram): keep completed commands claimed Redelivery after a completed write must never post another comment. Keep completion claims despite reply failures, scope updates per bot and only issue account-login links in private chats. * fix(feishu): bound individual api requests Bound individual platform calls so they cannot outlive the lock protecting a message update or stall a whole delivery invocation. * fix(feishu): classify retryable api failures Business error codes can be transient even in HTTP 200 responses. Classify token and rate-limit errors for retry and defer long waits. * fix(feishu): retry sends with stable uuids An unreadable response does not prove a send failed. Reuse the same uuid for retries so recovering the result cannot create a second Feishu message. * fix(feishu): refresh and evict tokens safely Separate app token caches, stop reusing refused tokens and share refreshes. Cache failures briefly and keep valid tokens in memory when a best-effort KV cache write fails. * fix(feishu): use the card update endpoint Delayed card updates use the card update endpoint; the config endpoint cannot update an interaction response. * fix(feishu): neutralize untrusted mentions Forge text is untrusted and can contain at-all tags. Neutralize mentions in card bodies and footers before rendering. * fix(feishu): authenticate and dedupe callbacks Authenticate signed or token-verified callbacks before dispatching commands. Decode encrypted envelopes and handle card actions using the correct fields without claiming irrelevant chat events. * fix(target): normalize actual destinations Different target spellings can name the same actual destination. Normalize channel, thread and topic identities for ownership checks and per-route duplicate suppression. * fix(config): transact per-group configuration Whole-table writes and cached snapshots can overwrite other groups or resurrect removed configuration. Use per-group guarded writes, one-time identity migration and explicit unreadable-row failures. * fix(group): preserve identity through lifecycle Group names are mutable and reusable, so they cannot own durable state. Preserve immutable identities across lifecycle operations and commit invite use with membership under transaction-time guards. * fix(oauth): confirm account links and installs A login link or installation id is not proof of account ownership. Verify installation management, bind browser state and require an explicit account-link confirmation before granting chat access. * fix(queue): fence leases and record target outcomes Retries must remember completed targets and distinguish active lease holders. Fence state updates, preserve permanent outcomes and reject missing parked payloads instead of dispatching empty data. * fix(dispatch): isolate and resume tenant sends Check tenant sources before filters and never dispatch orphan routes. Record each target outcome, isolate update chains by forge and group identity, and keep stale events from overwriting newer messages. * fix(queue): bound batches and resume deliveries A queue invocation has shared query and subrequest limits. Use a small batch and shared send budget, renew fenced leases and resume only unfinished work without exhausting unstarted messages. * fix(admin): guard revisions and tenant targets Read-time authorization alone cannot protect concurrent edits. Guard revisions, current roles and destination ownership in the write transaction, and use per-group endpoints for isolated changes. * fix(webhook): confirm only accepted deliveries Acknowledging before queue acceptance permanently loses deliveries. Confirm only accepted work, release failed claims, scope replay identities and strip tenant-supplied installation authority. * fix(storage): prune abandoned parked payloads Oversized payloads can remain after interrupted enqueue attempts. Prune objects beyond their retry lifetime without deleting payloads that a current delivery may still need. * fix(console): expose api conflict statuses Callers need the HTTP status to distinguish stale edits from other failures and reload current data after a conflict. * refactor(console): model group revisions Carry server revisions explicitly so console writes can reject changes based on an outdated group snapshot. * fix(console): send loaded route revisions Send the revision actually loaded with the route list and adopt the server response after a save. Reload on conflict instead of silently overwriting another editor. * fix(console): guard fragment saves by revision Fragments share a group version with other configuration writes. Submit the loaded revision and reload on conflict to preserve concurrent edits. * fix(console): save groups with guarded writes Whole-list saves can overwrite unrelated groups. Use per-group revision-aware writes, retain rename results and confirm deletion on the exact group, routes and version shown to the user. * fix(console): reject edits of stale routes A conflict reload can leave an older route editor open. Compare its original value before saving so it cannot overwrite the newly loaded route. * fix(console): show overview loading failures Failed overview requests must not look like an empty dashboard or a successful refresh. Show the reason while retaining prior data. * fix(console): explain rejected install bindings Explain why an installation binding was refused instead of silently returning to the console with no visible outcome. * fix(console): allow selecting a group named all The all sentinel collides with a valid group id. Use a value that cannot be a group id so that group stays selectable. * fix(console): discard stale log detail responses An older detail request can finish after a newer selection or close. Discard it so the dialog never shows the previously clicked record. * fix(console): retain legacy group owners Groups predating member roles store owners in adminIds. Load those as owners so adding a member never drops all old owners. * fix(console): lock member edits during saves Edits made during a save are overwritten by its result. Disable member controls and guard handlers until the write settles. * fix(console): show invite loading failures Failed invite loads must not look like no pending invites. Show the error and discard responses from a previously shown group. * fix(console): retain legacy discord log targets A stored log target without platform means Discord. Keep that meaning in the editor instead of clearing the target. * fix(console): retain undisplayed forge sources An editor that only displays some providers must preserve the others. Keep stored custom sources when saving unrelated group settings. * fix(console): lock the group editor while saving Prevent edits while a group save is in flight and make super-only installation settings readonly instead of silently ignoring edits. * fix(console): lock route edits and show failures Lock the route editor while saving and show fragment load failures with a retry action rather than presenting a false empty list. * fix(config): expose the legal contact override Declare the public runtime key so the documented environment override reaches legal pages in deployed Nuxt builds. * build(deploy): migrate d1 before deployment Apply pending D1 migrations before deploying code that requires them. Stop deployment when migration fails instead of running against an incomplete schema. * docs(worker): align guides with runtime behavior Synchronize both language guides, API references and examples with the implemented authorization, storage and delivery behavior. Document migration requirements and accepted operational limits. * fix(storage): repair orphaned KV routes Repair orphaned KV routes before serving KV-only configuration. * fix(security): fully escape forge HTML --------- Co-authored-by: RhenCloud <i@rhen.cloud>
This commit is contained in:
1 parent
dbf162bf74
commit
978c53eeb3
163 files changed
+18459
-3607
No files matched your search
+17
-9
@@ -1,23 +1,27 @@
|
||||
# Bot Commands
|
||||
|
||||
After [linking your GitHub account](#linking-your-account), you can act on GitHub **as yourself** from both Discord and Telegram — comments are posted with your own OAuth token and GitHub enforces permission. If GitHub rejects an action (e.g. editing someone else's comment) the bot tells you so.
|
||||
After [linking your GitHub account](#linking-your-account), you can act on GitHub **as yourself** from Discord, Telegram and Feishu — comments are posted with your own OAuth token and GitHub enforces permission. If GitHub rejects an action (e.g. editing someone else's comment) the bot tells you so.
|
||||
|
||||
## Linking Your Account
|
||||
|
||||
You must link your GitHub account once before using any command:
|
||||
You must link your GitHub account before using any command. The authorization lasts 10 hours; after that, run `/gh login` again:
|
||||
|
||||
| Platform | Command | Effect |
|
||||
| -------- | --------------------------------- | ---------------------------------------------------------------- |
|
||||
| Discord | `/gh login` | Returns an ephemeral OAuth link to authorize your GitHub account |
|
||||
| Discord | `/gh logout` | Unlinks your GitHub account |
|
||||
| Telegram | `/gh login` (reply to a message) | Returns an OAuth link |
|
||||
| Telegram | `/gh login` (private chat only) | Returns an OAuth link |
|
||||
| Telegram | `/gh logout` (reply to a message) | Unlinks your GitHub account |
|
||||
|
||||
The link is stored server-side (KV) and mapped to your Discord/Telegram user id in D1.
|
||||
The link is stored server-side (KV) and mapped to your Discord/Telegram/Feishu user id in D1.
|
||||
|
||||
After authorizing on GitHub you see a confirmation page naming **both** accounts (your GitHub login and the chat account that requested the link); nothing is linked until you confirm. If you did not start the login yourself — e.g. someone forwarded you their link — just close the page. On Telegram and Feishu, `/gh login` is only answered in the private chat with the bot, so authorization links are never posted to a group.
|
||||
|
||||
Commands run once per platform event: a redelivered Telegram update, a retried Feishu event or a replayed Discord interaction does not repeat a comment, merge or close. A command that failed before reaching GitHub (for example a storage error) is given back to the platform and runs again on its redelivery; once a command may have reached GitHub it is never repeated — if the bot's reply then fails, check GitHub before sending the command again.
|
||||
|
||||
## Discord
|
||||
|
||||
Discord commands are **slash commands** and **message context-menu commands**, synced by the scheduled trigger (every 5 minutes): per-guild for instant availability, and globally (24h dedup, ~1h propagation). All replies are ephemeral (only you see them).
|
||||
Discord commands are **slash commands** and **message context-menu commands**, synced by the scheduled trigger (every 5 minutes): per-guild for instant availability (at most 30 Discord API calls per run, so a bot in many guilds catches up over several runs), and globally (~1h propagation). Both are registered again when the command definitions change, and the global set at least daily. All replies are ephemeral (only you see them).
|
||||
|
||||
### Comment on an issue / PR
|
||||
|
||||
@@ -32,11 +36,11 @@ Two equivalent ways:
|
||||
/gh comment del link:<comment url> url must contain #issuecomment-<id>
|
||||
```
|
||||
|
||||
For `edit` / `del`, copy the specific comment link on GitHub (comment ⋯ menu → **Copy link**). `add` / `edit` open a modal to enter/adjust the comment body (prefilled for edit).
|
||||
For `edit` / `del`, copy the specific comment link on GitHub (comment ⋯ menu → **Copy link**). `add` / `edit` open a modal to enter/adjust the comment body (prefilled for edit, up to Discord's 4000-character limit; longer comments must be edited on GitHub so nothing is cut off). Saving an edit is refused when the comment changed on GitHub after the modal opened — reopen it to start from the current text. The body is posted as typed (indentation and line breaks kept). Only `https://github.com/...` links are accepted.
|
||||
|
||||
### Merge / close a PR
|
||||
|
||||
Notifications for open PRs include **Merge / Close** buttons (labels follow the group's message language):
|
||||
Notifications for open PRs on github.com include **Merge / Close** buttons (labels follow the group's message language; Gitea and other forges get no buttons, since the actions call the GitHub API):
|
||||
|
||||
- Clicking a button merges (squash) or closes the PR as your linked GitHub account; GitHub enforces permission.
|
||||
- On success the buttons are removed from the notification and the result is shown in an ephemeral reply.
|
||||
@@ -54,9 +58,13 @@ Notifications for open PRs include **Merge / Close** buttons (labels follow the
|
||||
|
||||
Telegram `/gh` commands work by **replying to a notification message**:
|
||||
|
||||
- `/gh login` — link your GitHub account (returns an OAuth link)
|
||||
- `/gh login` — link your GitHub account (private chat with the bot only; returns an OAuth link)
|
||||
- `/gh logout` — unlink
|
||||
- `/gh comment <text>` — reply to an issue/PR notification to comment as yourself
|
||||
- `/gh merge` / `/gh close` — reply to a PR notification to merge/close it
|
||||
|
||||
The target issue/PR is parsed from the message you reply to (the notification embed links). Commands arrive via the Telegram webhook (`POST /telegram/webhook`, optionally verified with `TELEGRAM_WEBHOOK_SECRET`).
|
||||
The target issue/PR is parsed from the message you reply to (the notification embed links). In groups, the `/gh@YourBot …` form is accepted too (commands addressed to another bot are ignored). Comment text keeps its line breaks and indentation. Commands arrive via the Telegram webhook (`POST /telegram/webhook`), which always verifies the secret token — `TELEGRAM_WEBHOOK_SECRET`, or one derived from `TELEGRAM_TOKEN` when unset.
|
||||
|
||||
## Feishu
|
||||
|
||||
Feishu `/gh` commands are messages that start with `/gh` (after any @mention of the bot): `/gh login` (private chat only), `/gh logout`, `/gh comment <PR/Issue link> <text>`, `/gh merge <PR link>`, `/gh close <PR link>`; cards also carry **Merge** / **Close** buttons. Messages that merely mention `/gh` elsewhere are not commands. See [Feishu Bot Setup](./deployment#feishu-bot-setup) for the event subscription.
|
||||
+34
-23
@@ -27,29 +27,40 @@ WebHooker requires several secrets to function. For local development, store the
|
||||
| `FEISHU_APP_ID` | Feishu app ID (from the app Credentials page) — required for Feishu routes |
|
||||
| `FEISHU_APP_SECRET` | Feishu app secret — required for Feishu routes |
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Inbound Feishu commands and card buttons (`POST /feishu/webhook`) require
|
||||
> `FEISHU_ENCRYPT_KEY` and/or `FEISHU_VERIFICATION_TOKEN` (the app's **Events &
|
||||
> Callbacks → Encryption Strategy** values). Without either, the endpoint refuses
|
||||
> every callback, since an unauthenticated request could act as any linked user.
|
||||
|
||||
<!-- separate the two alerts (MD028) -->
|
||||
|
||||
> [!NOTE]
|
||||
> `GITHUB_APP_ID` and `GITHUB_PRIVATE_KEY` (PKCS#8 PEM) are used by the GitHub App
|
||||
> **install flow** (`/auth/github/install`) to resolve the installing account's login
|
||||
> via an App JWT. They are optional — when unset, the install page still works but
|
||||
> shows an anonymous `inst-{installationId}` group without the account name. The
|
||||
> OAuth flow itself only needs `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`.
|
||||
> **install flow** (`/auth/github/install`) to resolve the installation's account via
|
||||
> an App JWT — which is also how the flow verifies that the signed-in user may manage
|
||||
> the installation. Without them only super admins can bind installations. The OAuth
|
||||
> flow itself only needs `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`; it requests the
|
||||
> `repo` and `read:org` scopes (the latter to check organization admin rights).
|
||||
|
||||
### Optional Secrets
|
||||
|
||||
| Variable | Description | Default |
|
||||
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
|
||||
| `DISCORD_PUBLIC_KEY` | Discord application public key (Developer Portal) — required for interactions | Unset → interactions return `401` |
|
||||
| `DISCORD_APPLICATION_ID` | Discord application id; auto-resolved when omitted | Auto-resolved |
|
||||
| `TELEGRAM_WEBHOOK_SECRET` | Secret token for `POST /telegram/webhook` verification (X-Telegram-Bot-Api-Secret-Token) | Disabled (no verification) |
|
||||
| `TELEGRAM_RICH_HEADER_HOST` | Base URL of an external rich-header service; when unset, the built-in `GET /api/richheader` serves the Telegram avatar card | Built-in `/api/richheader` |
|
||||
| `BASE_URL` | Public URL for OAuth callbacks | `http://localhost:8787` |
|
||||
| `ADMIN_USER_IDS` | Comma-separated GitHub user IDs (or logins) allowed to access the Web UI | Disabled |
|
||||
| `DEBUG_MODE` | Local development only: exact `true` bypasses all `/admin` authentication and grants full access | Disabled |
|
||||
| `ALLOW_SELF_SIGNUP` | When enabled (`1`/`true`), GitHub users without any group access get a personal group on first login instead of `403` | Disabled |
|
||||
| `AUDIT_RETENTION_DAYS` | Audit-log retention in days for the scheduled cleanup | `90` |
|
||||
| `NUXT_PUBLIC_DOCS_URL` | Docs site URL used by the landing page (client-side runtime config) | Landing page defaults |
|
||||
| `NUXT_PUBLIC_REPO_URL` | GitHub repo URL used by the landing page | Landing page defaults |
|
||||
| `NUXT_PUBLIC_LEGAL_CONTACT` | Contact shown on `/terms` and `/privacy` | Unset → placeholder text |
|
||||
| Variable | Description | Default |
|
||||
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
|
||||
| `DISCORD_PUBLIC_KEY` | Discord application public key (Developer Portal) — required for interactions | Unset → interactions return `401` |
|
||||
| `DISCORD_APPLICATION_ID` | Discord application id; auto-resolved when omitted | Auto-resolved |
|
||||
| `TELEGRAM_WEBHOOK_SECRET` | Secret token for `POST /telegram/webhook` verification (X-Telegram-Bot-Api-Secret-Token) | Derived from `TELEGRAM_TOKEN` |
|
||||
| `FEISHU_ENCRYPT_KEY` | Feishu event Encrypt Key: verifies `X-Lark-Signature` and decrypts encrypted callbacks | Unset |
|
||||
| `FEISHU_VERIFICATION_TOKEN` | Feishu event Verification Token, compared with the token in every callback | Unset |
|
||||
| `TELEGRAM_RICH_HEADER_HOST` | Base URL of an external rich-header service; when unset, the built-in `GET /api/richheader` serves the Telegram avatar card | Built-in `/api/richheader` |
|
||||
| `BASE_URL` | Public URL of the worker, for OAuth callbacks and the Telegram webhook sync | Unset — required in production |
|
||||
| `ADMIN_USER_IDS` | Comma-separated GitHub user IDs (or logins) allowed to access the Web UI; prefer numeric ids — a login is bound to the first account that signs in with it | Disabled |
|
||||
| `DEBUG_MODE` | Local development only: exact `true` bypasses all `/admin` authentication and grants full access | Disabled |
|
||||
| `ALLOW_SELF_SIGNUP` | When enabled (`1`/`true`), GitHub users without any group access get an isolated personal group on first login instead of `403` | Disabled |
|
||||
| `AUDIT_RETENTION_DAYS` | Audit-log retention in days for the scheduled cleanup | `90` |
|
||||
| `NUXT_PUBLIC_DOCS_URL` | Docs site URL used by the landing page (client-side runtime config) | Landing page defaults |
|
||||
| `NUXT_PUBLIC_REPO_URL` | GitHub repo URL used by the landing page | Landing page defaults |
|
||||
| `NUXT_PUBLIC_LEGAL_CONTACT` | Contact shown on `/terms` and `/privacy` | Unset → placeholder text |
|
||||
|
||||
## Web UI
|
||||
|
||||
@@ -57,7 +68,7 @@ WebHooker ships with a built-in config console at `/admin` for managing routes,
|
||||
|
||||
### Setup
|
||||
|
||||
1. Configure `ADMIN_USER_IDS` with the GitHub user IDs allowed to manage everything. Logins are also accepted, e.g. `ADMIN_USER_IDS=12345,RhenCloud`. If unset, the console is disabled (unless `ALLOW_SELF_SIGNUP` is enabled).
|
||||
1. Configure `ADMIN_USER_IDS` with the GitHub user IDs allowed to manage everything. Logins are also accepted, e.g. `ADMIN_USER_IDS=12345,RhenCloud`, but numeric ids are safer: GitHub logins can be renamed and re-registered, so a login is bound to the first account that signs in with it. If unset, there are no super admins: members of a group still reach the console for their groups, and `ALLOW_SELF_SIGNUP` gives other users a personal group.
|
||||
2. Open `/admin` and sign in with GitHub.
|
||||
3. Users without any access get `403`, except when `ALLOW_SELF_SIGNUP=1` (they receive a personal group) or when they follow a group [invite link](./groups#invites).
|
||||
|
||||
@@ -67,7 +78,7 @@ The console is served as an SPA at `/admin`; its tabs are deep-linkable via the
|
||||
|
||||
Destructive actions — deleting routes, groups, filter conditions, fragments, targets or forge hosts, removing members, revoking invites, disabling a group's webhook — ask for confirmation in an in-app dialog (empty, never-filled rows are removed directly). The route and group editors ignore clicks on the backdrop so unsaved input is never lost; close them with Cancel, ✕ or <kbd>Esc</kbd>.
|
||||
|
||||
All management endpoints (`/admin/api/*`) are documented in the [Admin API](../api/admin). Saved routes and groups are persisted to D1 (`d1_routes` / `d1_groups`) immediately, the KV cache is invalidated and the config cache is refreshed so the webhook pipeline picks them up on the next run.
|
||||
All management endpoints (`/admin/api/*`) are documented in the [Admin API](../api/admin). Saved routes and groups are persisted to D1 (`d1_routes` / `d1_groups`) immediately; webhook ingress and the queue consumer read the configuration fresh, so the next event already uses them.
|
||||
|
||||
## Filter Types
|
||||
|
||||
@@ -89,11 +100,11 @@ See the [Filter Tutorial](./filters) for a hands-on guide with worked examples.
|
||||
- Set `"exclude": true` on any filter to invert it (NOT logic)
|
||||
- Every filter type supports the same pattern forms: plain text, `*`/`?` **globs** (`*` = any run, `?` = one character), and `/regular expression/` — all case-insensitive
|
||||
- Field filters (`event`/`repo`/`actor`/`action`/`branch`/`field`) glob-match the whole value; `keyword` globs and regexes search anywhere in the payload; plain `keyword` text is a substring search
|
||||
- Every filter except `keyword` accepts an `op` operator — `eq` (default, classic behaviour), `ne`, `contains`, `startsWith`, `endsWith`, `regex`, `gt`, `gte`, `lt`, `lte`, `in`, `exists` — see the [Filter Tutorial](./filters#operators)
|
||||
- Every filter except `keyword` (which only supports the default `eq`) accepts an `op` operator — `eq` (default, classic behaviour), `ne`, `contains`, `startsWith`, `endsWith`, `regex`, `gt`, `gte`, `lt`, `lte`, `in`, `exists` — see the [Filter Tutorial](./filters#operators)
|
||||
- `field` filters use a dot-separated JSONPath `path` into the payload; arrays are expanded so any matching element satisfies the filter
|
||||
- Routes may nest filters in an `ast` node (`all` / `any` / `not`) instead of a flat `filters` list; `ast` takes precedence when present
|
||||
- Patterns longer than 200 characters are not compiled as glob/regex; an invalid `//`-wrapped regex matches nothing
|
||||
- `branch` filter works for push, pull_request, pull_request_review, pull_request_review_comment, create/delete, workflow_run, workflow_job, check_suite, deployment, and code_scanning_alert events
|
||||
- Globs and regexes run on the RE2 engine (linear time). A pattern the matcher cannot compile — longer than 200 characters, or using syntax RE2 lacks such as backreferences or lookaround — is rejected when saved; one stored before makes its route [unreadable](./routes#unreadable-routes) instead of silently never matching
|
||||
- `branch` filter works for push, pull_request, pull_request_review, pull_request_review_comment, create/delete, workflow_run, workflow_job, check_run, check_suite, deployment, deployment_status, and code_scanning_alert events
|
||||
|
||||
### Match Values
|
||||
|
||||
|
||||
+30
-18
@@ -2,6 +2,8 @@
|
||||
|
||||
## Cloudflare Setup
|
||||
|
||||
`wrangler.jsonc` ships with the maintainers' `account_id` and resource ids. Replace `account_id` with your own Cloudflare account id (`bunx wrangler whoami`), or remove the field to deploy to the account you are logged in with, and replace the ids in the steps below.
|
||||
|
||||
### 1. Create KV Namespace
|
||||
|
||||
```bash
|
||||
@@ -32,7 +34,8 @@ bunx wrangler secret put DISCORD_PUBLIC_KEY # Discord app public key (Develope
|
||||
bunx wrangler secret put TELEGRAM_TOKEN # Telegram bot token (BotFather) — required for Telegram routes
|
||||
bunx wrangler secret put FEISHU_APP_ID # Feishu app ID — required for Feishu routes
|
||||
bunx wrangler secret put FEISHU_APP_SECRET # Feishu app secret — required for Feishu routes
|
||||
bunx wrangler secret put ADMIN_USER_IDS # comma-separated GitHub IDs/logins allowed into the Web UI
|
||||
bunx wrangler secret put ADMIN_USER_IDS # comma-separated GitHub user ids allowed into the Web UI (numeric ids recommended)
|
||||
bunx wrangler secret put BASE_URL # public URL of the worker — OAuth callbacks and the Telegram webhook sync need it
|
||||
```
|
||||
|
||||
::: tip Target channels are configured per route
|
||||
@@ -40,7 +43,7 @@ There is no global channel secret. Each route in the [Web UI](/guide/configurati
|
||||
:::
|
||||
|
||||
::: tip GitHub App ID / private key are optional
|
||||
`GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` (PKCS#8 PEM) are used by the [App install flow](#github-app-setup) to resolve the installing account's login on the post-install choice page, and to mint installation tokens for API enrichment — fetching `workflow_run` job lists and check-suite build logs. You can skip them — the install page then shows an anonymous `inst-{installationId}` group, workflow job lists still load for public repositories (unauthenticated), and private-repo build logs are omitted. The OAuth flow only needs `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`.
|
||||
`GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` (PKCS#8 PEM) are used by the [App install flow](#github-app-setup) to resolve the installing account's login on the post-install choice page, and to mint installation tokens for API enrichment — fetching `workflow_run` job lists and check-suite build logs. Without them only super admins can bind installations — the flow cannot resolve the installation's account, so it cannot verify that a signed-in user manages it — workflow job lists still load for public repositories (unauthenticated), and private-repo build logs are omitted. The OAuth flow only needs `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`.
|
||||
:::
|
||||
|
||||
Discord interactions arrive via the HTTPS Interactions Endpoint, so set `DISCORD_PUBLIC_KEY` and point the **Interactions Endpoint URL** at `https://your-domain/discord/interactions`. See [Interactions Endpoint](#interactions-endpoint) below.
|
||||
@@ -72,10 +75,10 @@ bun run db:migrate:prod # apply migrations to the remote D1 database
|
||||
bun run db:migrate # apply migrations to the local (Miniflare) database
|
||||
```
|
||||
|
||||
The `db:migrate` scripts run `wrangler d1 migrations apply webhooker` (see `package.json`), which applies each SQL file in `migrations/` and records applied versions in the `d1_migrations` table.
|
||||
The `db:migrate` scripts run `wrangler d1 migrations apply webhooker` (see `package.json`), which applies each SQL file in `migrations/` and records applied versions in the `d1_migrations` table. `bun run deploy` runs `db:migrate:prod` before deploying, so migrations added later are applied on the next deploy; already-applied ones are skipped, and a failed migration stops the deploy.
|
||||
|
||||
::: tip Databases previously migrated with `d1 execute`
|
||||
If the database already has these tables/columns (e.g. previously migrated with `wrangler d1 execute --file`), the `d1_migrations` tracking table may be missing and `db:migrate:prod` will try to re-run every migration. The `ALTER TABLE ... ADD COLUMN` statements in `0002_log_detail.sql` then fail because the columns already exist. In that case run the files directly instead:
|
||||
If the database already has these tables/columns (e.g. previously migrated with `wrangler d1 execute --file`), the `d1_migrations` tracking table may be missing and `db:migrate:prod` will try to re-run every migration. The `ALTER TABLE ... ADD COLUMN` statements in `0002_log_detail.sql` then fail because the columns already exist. In that case apply the files your database does not have yet directly (the `ALTER TABLE ... ADD COLUMN` files fail when run twice, so skip those already applied):
|
||||
|
||||
```bash
|
||||
bunx wrangler d1 execute webhooker --remote --file ./migrations/0001_init.sql
|
||||
@@ -89,8 +92,11 @@ bunx wrangler d1 execute webhooker --remote --file ./migrations/0008_storage_d1.
|
||||
bunx wrangler d1 execute webhooker --remote --file ./migrations/0010_fragments.sql
|
||||
bunx wrangler d1 execute webhooker --remote --file ./migrations/0011_feishu_links.sql
|
||||
bunx wrangler d1 execute webhooker --remote --file ./migrations/0012_route_position.sql
|
||||
bunx wrangler d1 execute webhooker --remote --file ./migrations/0013_prune_indexes.sql
|
||||
```
|
||||
|
||||
Such a database has no migration history, so `bun run deploy` (which runs `db:migrate:prod` first) would try every migration again and stop. Deploy it with `bunx wrangler deploy`, and apply later migration files the same way.
|
||||
|
||||
:::
|
||||
|
||||
### 4. Create R2 Bucket (Optional)
|
||||
@@ -101,25 +107,30 @@ The `PAYLOAD` binding parks oversized webhook payloads in R2 (`webhooker-payload
|
||||
bunx wrangler r2 bucket create webhooker-payloads
|
||||
```
|
||||
|
||||
The bucket is already declared in `wrangler.jsonc` (`r2_buckets`), so no binding change is needed.
|
||||
Then uncomment the `r2_buckets` block in `wrangler.jsonc` (it ships commented out); without the binding, oversized payloads stay in KV.
|
||||
|
||||
### 5. Create Queues (Optional)
|
||||
|
||||
The `QUEUE` binding routes webhook delivery through Cloudflare Queues (async dispatch with retry backoff and a dead-letter queue). Skip this step to keep dispatch inline (synchronous).
|
||||
The `QUEUE` binding routes webhook delivery through Cloudflare Queues (async dispatch with retry backoff and a dead-letter queue). To keep dispatch inline (synchronous) instead, remove or comment out the `queues` block in `wrangler.jsonc`; while it is there, both queues must exist.
|
||||
|
||||
```bash
|
||||
bunx wrangler queues create webhooker-delivery
|
||||
bunx wrangler queues create webhooker-delivery-dlq
|
||||
```
|
||||
|
||||
The queues are already declared in `wrangler.jsonc` (`queues.producers` / `queues.consumers`), so no binding change is needed. The `webhooker-delivery` consumer retries retryable failures with exponential backoff (5s/30s/2m/10m) up to `max_retries`, after which the message is moved to `webhooker-delivery-dlq` and marked dead.
|
||||
The queues are already declared in `wrangler.jsonc` (`queues.producers` / `queues.consumers`), so no binding change is needed. The `webhooker-delivery` consumer retries retryable failures (5xx, 429, network, rate-limit exhaustion, Feishu token errors) with exponential backoff (5s/30s/2m/10m) up to `max_retries`, after which the message is moved to `webhooker-delivery-dlq` and marked dead. Retries are per target: targets that already succeeded — or failed permanently — are recorded and not attempted again, so one bad target never cancels the others' retries and a retry does not resend a recorded target (a send whose record could not be written may be repeated). Each consumer invocation takes at most 3 messages and shares a budget of 12 minutes and 20 target sends across them, because the Workers Free limits (50 D1 queries, 50 subrequests) apply per invocation: a message the budget did not reach is retried, and a delivery with more targets than the budget continues in a new queue message. A delivery whose every target was handled still ends as `failed` if any target failed permanently along the way. A delivery is taken with a 5-minute lease, so two consumers never dispatch the same delivery at once.
|
||||
|
||||
> [!NOTE]
|
||||
> R2 has no per-object TTL. Parked payloads are deleted after dispatch, and the scheduled `storage-prune` task removes orphans older than five days; you may also add a bucket [lifecycle rule](https://developers.cloudflare.com/r2/buckets/object-lifecycles/) for the `webhooks/` prefix.
|
||||
|
||||
### 6. Deploy
|
||||
|
||||
```bash
|
||||
bunx wrangler deploy
|
||||
bun run deploy # applies pending D1 migrations, then runs wrangler deploy
|
||||
```
|
||||
|
||||
Run from a terminal, wrangler asks for confirmation before applying pending migrations; in CI it proceeds. If you deploy through Cloudflare's Git integration (Workers Builds), set its deploy command to `bun run deploy` so migrations are applied there too.
|
||||
|
||||
Your worker is now live at `https://webhooker.<your-subdomain>.workers.dev`.
|
||||
|
||||
### 7. Configure GitHub Webhook
|
||||
@@ -151,7 +162,7 @@ Your worker is now live at `https://webhooker.<your-subdomain>.workers.dev`.
|
||||
- **Organization permissions**: Members (read) — if needed
|
||||
4. Subscribe to events (all 28 supported):
|
||||
- Push, Pull request, Issues, Issue comment, Workflow run, Workflow job, Status, Deployment, Deployment status, Ping, Release, Create, Delete, Star, Fork, Check run, Check suite, Pull request review, Pull request review comment, Commit comment, Member, Label, Milestone, Discussion, Discussion comment, Repository, Code scanning alert, Dependabot alert
|
||||
5. Generate private key — optional; set `GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY` to show the installing account's login on the post-install page and to authenticate API enrichment (workflow job lists, build logs) for private repositories (see the tip above).
|
||||
5. Generate a private key and set `GITHUB_APP_ID` + `GITHUB_PRIVATE_KEY`: the install flow needs them to verify that the installing user manages the installation (without them only super admins can bind installations), and API enrichment (workflow job lists, build logs) uses them for private repositories (see the tip above).
|
||||
|
||||
### 2. Install App
|
||||
|
||||
@@ -186,20 +197,20 @@ Messages are sent via the Discord **REST API**, so pushing works with just `DISC
|
||||
2. Set the **Interactions Endpoint URL** to `https://your-domain/discord/interactions`.
|
||||
3. Every interaction request is verified with Ed25519 signatures (`X-Signature-Ed25519` over `X-Signature-Timestamp + body`).
|
||||
|
||||
The `/gh` slash command and the `GitHub: 添加/编辑/删除评论` message commands are synced by the scheduled trigger (every 5 minutes): per-guild for instant availability, plus a global registration (24h dedup, ~1h propagation). The bot never connects to the Discord Gateway, so it shows as **offline** — messaging is unaffected (always REST).
|
||||
The `/gh` slash command and the `GitHub: 添加/编辑/删除评论` message commands are synced by the scheduled trigger (every 5 minutes): per-guild for instant availability (at most 30 API calls per run), plus a global registration (~1h propagation); both are registered again when the command definitions change. The bot never connects to the Discord Gateway, so it shows as **offline** — messaging is unaffected (always REST).
|
||||
|
||||
Users run `/gh login` to link their GitHub account and can then comment on issues/PRs as themselves. See the [Bot Commands](/guide/commands) page for the full command reference.
|
||||
|
||||
## Telegram Bot Setup
|
||||
|
||||
1. Create a bot with [@BotFather](https://t.me/BotFather) and copy its token to `TELEGRAM_TOKEN`.
|
||||
2. (Optional) Set `TELEGRAM_WEBHOOK_SECRET`; the webhook registration passes it to Telegram as `secret_token`, and `POST /telegram/webhook` verifies it with a timing-safe compare.
|
||||
3. The worker syncs the webhook from the scheduled trigger (`setWebhook` to `{BASE_URL}/telegram/webhook`), so no manual `setWebhook` call is needed — just make sure `BASE_URL` is set.
|
||||
4. Add the bot to a group (or enable topics) and route events to `chatId` / `topicId` in the route config.
|
||||
2. (Optional) Set `TELEGRAM_WEBHOOK_SECRET`; the webhook registration passes it to Telegram as `secret_token`, and `POST /telegram/webhook` verifies it with a timing-safe compare. When unset, a secret derived from `TELEGRAM_TOKEN` is used instead, so the endpoint is never left unauthenticated (the next scheduled sync re-registers the webhook with it).
|
||||
3. The worker syncs the webhook from the scheduled trigger (`setWebhook` to `{BASE_URL}/telegram/webhook`, only when the registration changed and once a day), so no manual `setWebhook` call is needed — just make sure `BASE_URL` is set.
|
||||
4. Add the bot to a group (or enable topics) and route events to `chatId` / `topicId` in the route config. Use the numeric chat id (e.g. `-1001234567890`); `@username` targets are not accepted.
|
||||
|
||||
In Telegram, `/gh` commands (`/gh login`, `/gh logout`, `/gh comment <text>`, `/gh merge`, `/gh close`) work by replying to a notification message — see the [Bot Commands](/guide/commands) page.
|
||||
In Telegram, `/gh` commands (`/gh login`, `/gh logout`, `/gh comment <text>`, `/gh merge`, `/gh close`) work by replying to a notification message — see the [Bot Commands](/guide/commands) page. `/gh login` only works in the private chat with the bot, and the GitHub user confirms the binding on a page that names both accounts.
|
||||
|
||||
Avatars are rendered as a link-preview card using the built-in `GET /api/richheader` (overridable with `TELEGRAM_RICH_HEADER_HOST`).
|
||||
Avatars are rendered as a link-preview card using the built-in `GET /api/richheader` (overridable with `TELEGRAM_RICH_HEADER_HOST`; without `BASE_URL` the avatar image itself is the preview). Messages are always text messages, so in-place updates keep the full 4096-character budget.
|
||||
|
||||
## Feishu Bot Setup
|
||||
|
||||
@@ -210,9 +221,9 @@ Avatars are rendered as a link-preview card using the built-in `GET /api/richhea
|
||||
- `im:message:send_as_bot` (send messages as an app)
|
||||
- `im:message:send` (historical version)
|
||||
4. In **Bot** tab, turn on the bot capability. Add the bot to the target group chat (or create a new group) and copy the **Chat ID** from the group settings.
|
||||
5. In WebHooker `/admin`, create or edit a route and add a target with `platform: "feishu"`, `chatId` set to the Feishu **Chat ID**, and optional `topicId` for a topic inside the chat.
|
||||
5. In WebHooker `/admin`, create or edit a route and add a target with `platform: "feishu"` and `chatId` set to the Feishu **Chat ID** (`oc_…`); Feishu targets have no topic.
|
||||
|
||||
WebHooker uses the app-level credentials (`FEISHU_APP_ID` / `FEISHU_APP_SECRET`) to request a `tenant_access_token`, caches it until expiry, and sends messages as an interactive card (`interactive` message type). The same token is used to edit messages in place for `workflow_run` / `check_run` progress updates.
|
||||
WebHooker uses the app-level credentials (`FEISHU_APP_ID` / `FEISHU_APP_SECRET`) to request a `tenant_access_token`, caches it per app (in memory and KV) until shortly before it expires, drops it as soon as Feishu refuses it, and sends messages as an interactive card (`interactive` message type). The same token is used to edit messages in place for `workflow_run` / `check_run` progress updates.
|
||||
|
||||
### Inbound: commands & buttons
|
||||
|
||||
@@ -230,7 +241,8 @@ To enable inbound:
|
||||
2. Subscribe to the events:
|
||||
- `im.message.receive_v1` — receive `/gh` commands (requires the `im:message` scope).
|
||||
- `card.action.trigger` — receive button clicks on cards.
|
||||
3. In **Credentials & Basic Info**, set the **App Secret** (already used for `FEISHU_APP_SECRET`) — it also signs the inbound callback via the `X-Lark-Signature` header, and WebHooker verifies it.
|
||||
3. In **Events & Callbacks → Encryption Strategy**, copy the **Encrypt Key** to `FEISHU_ENCRYPT_KEY` (WebHooker then verifies `X-Lark-Signature` — SHA-256 of timestamp + nonce + key + body — and decrypts encrypted callbacks) and/or the **Verification Token** to `FEISHU_VERIFICATION_TOKEN`. At least one is required; without them every callback is refused. Each event id runs once, even when Feishu retries it.
|
||||
4. `/gh login` only works in the private (p2p) chat with the bot; the GitHub user confirms the binding on a page that names both accounts.
|
||||
|
||||
### Required permissions
|
||||
|
||||
|
||||
+2
-2
@@ -30,7 +30,7 @@ Branch deletions via `git push --delete` arrive as push events with `deleted: tr
|
||||
|
||||
## How do I point a group at its own webhook endpoint?
|
||||
|
||||
See [Per-group endpoint](./ingress#per-group-endpoint) — generate a secret from the group's **Webhook endpoint** panel (owner role), then use `POST /webhook/{groupId}` with the group secret.
|
||||
See [Per-group endpoint](./ingress#per-group-endpoint-post-webhook-groupid) — generate a secret from the group's **Webhook endpoint** panel (owner role), then use `POST /webhook/{groupId}` with the group secret.
|
||||
|
||||
## Can I run this outside Cloudflare Workers?
|
||||
|
||||
@@ -38,4 +38,4 @@ No — the worker requires the KV and D1 bindings declared in `wrangler.jsonc` a
|
||||
|
||||
## Where is data stored?
|
||||
|
||||
Configuration lives in D1 (`d1_routes`/`d1_groups`, with KV as a cache); webhook dedup, delivery state and message-update tracking also live in D1 with KV fallback; send/audit logs and platform↔GitHub links live in D1. R2 optionally parks oversized payloads. See [Storage Layout](./storage#storage-decisions).
|
||||
Configuration lives in D1 (`d1_routes`/`d1_groups`, cached in memory for at most 60 seconds); webhook dedup, delivery state and message-update tracking also live in D1 with KV fallback; send/audit logs and platform↔GitHub links live in D1. R2 optionally parks oversized payloads. See [Storage Layout](./storage#storage-decisions).
|
||||
+26
-21
@@ -124,16 +124,17 @@ This fires when a pull request is opened or reopened (and not on merge/close/edi
|
||||
|
||||
Matches the branch involved in the event. What counts as "the branch" depends on the event type:
|
||||
|
||||
| Event | Branch extracted |
|
||||
| --------------------------- | ------------------------------------------- |
|
||||
| `push` | The branch that was pushed to |
|
||||
| `pull_request` (and review) | The pull request's **head** (source) branch |
|
||||
| `create` / `delete` | The created/deleted branch or tag |
|
||||
| `workflow_run` | The `head_branch` the workflow ran on |
|
||||
| `workflow_job` | The `head_branch` the job ran on |
|
||||
| `check_suite` | The `head_branch` of the check suite |
|
||||
| `deployment` | The deployment ref (strips `refs/heads/`) |
|
||||
| `code_scanning_alert` | The branch the alert belongs to |
|
||||
| Event | Branch extracted |
|
||||
| ---------------------------------- | ------------------------------------------------------------------------------------ |
|
||||
| `push` | The branch that was pushed to |
|
||||
| `pull_request` (and review) | The pull request's **head** (source) branch |
|
||||
| `create` / `delete` | The created/deleted branch or tag |
|
||||
| `workflow_run` | The `head_branch` the workflow ran on |
|
||||
| `workflow_job` | The `head_branch` the job ran on |
|
||||
| `check_run` | The `head_branch` of the run's check suite |
|
||||
| `check_suite` | The `head_branch` of the check suite |
|
||||
| `deployment` / `deployment_status` | The deployment ref (`deployment.ref`, strips a leading `refs/heads/`) |
|
||||
| `code_scanning_alert` | The alert's ref, without a leading `refs/heads/` (use `main`, not `refs/heads/main`) |
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -179,8 +180,9 @@ Fires when the payload contains `deploy` anywhere. Because the payload is lowerc
|
||||
|
||||
Behavior details:
|
||||
|
||||
- Patterns longer than 200 characters are **not** compiled as glob/regex and fall back to plain matching.
|
||||
- A `//`-wrapped pattern that is not a valid regex matches **nothing** (the filter stays false) rather than erroring.
|
||||
- Patterns longer than 200 characters cannot be compiled as glob/regex; such patterns are rejected when saved.
|
||||
- Globs and regular expressions run on the RE2 engine, whose matching time is linear in the input — no pattern can stall the worker. RE2 syntax has no backreferences (`\1`) or lookaround (`(?=…)`, `(?<!…)`); the console and API reject such patterns.
|
||||
- A `//`-wrapped pattern that is not a valid regex is rejected when saved.
|
||||
- To search for text that is a glob or regex special character (e.g. `v1.2.3`), rely on the plain-text form — a pattern without `*`, `?`, or `//` wrapping matches literally.
|
||||
- The search covers the **entire** payload: commit messages, PR titles and bodies, labels, refs, even repository and sender names.
|
||||
|
||||
@@ -221,7 +223,7 @@ Field filters (and every filter type except `keyword`) accept an `op` to change
|
||||
| `gt` / `gte` | Numeric greater-than / greater-or-equal |
|
||||
| `lt` / `lte` | Numeric less-than / less-or-equal |
|
||||
| `in` | Value equals any of the listed patterns |
|
||||
| `exists` | The field is present (non-null); `match` is ignored |
|
||||
| `exists` | The field is present (non-null); `match` may be omitted |
|
||||
|
||||
```json
|
||||
{ "type": "field", "path": "pull_request.commits", "op": "gt", "match": "1" }
|
||||
@@ -239,6 +241,9 @@ A route can use a nested `ast` to combine filters with explicit grouping instead
|
||||
{
|
||||
"id": "grouped",
|
||||
"name": "Grouped",
|
||||
"enabled": true,
|
||||
"groupId": "eng",
|
||||
"filters": [],
|
||||
"ast": {
|
||||
"all": [
|
||||
{ "type": "event", "match": "pull_request" },
|
||||
@@ -250,11 +255,11 @@ A route can use a nested `ast` to combine filters with explicit grouping instead
|
||||
}
|
||||
]
|
||||
},
|
||||
"targets": [{ "channelId": "..." }]
|
||||
"targets": [{ "platform": "discord", "channelId": "..." }]
|
||||
}
|
||||
```
|
||||
|
||||
When `ast` is present it takes precedence over `filters`. The admin console's route editor builds `ast` visually (all/any/not groups), shows a live explanation of the tree, and can test it against a pasted JSON payload via the **Test match** panel.
|
||||
When `ast` is present it takes precedence over `filters`; the route still carries a `filters` array (it may be empty). The admin console's route editor builds `ast` visually (all/any/not groups), shows a live explanation of the tree, and can test it against a pasted JSON payload via the **Test match** panel.
|
||||
|
||||
Values, field paths and the test event use searchable dropdowns. Suggestions come from GitHub's and Gitea's webhook definitions — events, the actions of the events the route filters on, common payload paths and their known values — each shown as its English key with a description in the console language; search matches the key first, then the English or Chinese description. Anything outside the list can still be typed and is validated before it is added; only values that can never match are rejected (an event name with spaces, a branch with characters git forbids, invalid regex syntax, a non-number for a comparison operator, …). Repository and actor values stay free-form because custom webhooks may carry any string. Text that is typed but not added with <kbd>Enter</kbd> blocks saving, fragment saving and test runs instead of being added or dropped silently. Several values can be typed or pasted at once, separated by commas or new lines; a `/regex/` keeps its own commas, and with the **matches regex** (`regex`) operator only new lines separate values (a raw pattern such as `^v{1,3}` may contain commas). Suggestions follow the operator: the substring operators compare text literally, so a glob suggestion becomes the text it implies (`release/*` → `release/` for **starts with**, `*[bot]` → `[bot]` for **ends with**), **matches regex** offers each suggestion as the escaped, anchored pattern it means (`^dependabot\[bot\]$`, `^release/`), and comparison operators get no suggestions. Switching between **equals** / **not equals** / **in** and **matches regex** rewrites `/x/` ↔ `x` in the values already added — both forms compile to the same regular expression, so the condition keeps matching exactly the same. Other values stay as typed (literal, glob and substring comparisons differ from regular expressions on line breaks and Unicode case folding), and values that can never match under the new operator are flagged. Arrays are expanded along a `field` path, but a path that **ends** on an array (e.g. `commits.modified`) is compared as a whole — match one of its elements with `contains`; the editor switches to `contains` when such a suggested path is picked.
|
||||
|
||||
@@ -275,13 +280,13 @@ Forward pull request activity, but ignore bot authors and draft PRs, to a `#prs`
|
||||
"filters": [
|
||||
{ "type": "event", "match": "pull_request" },
|
||||
{ "type": "actor", "match": "dependabot[bot]", "exclude": true },
|
||||
{ "type": "keyword", "match": "\"draft\": true", "exclude": true }
|
||||
{ "type": "field", "path": "pull_request.draft", "match": "true", "exclude": true }
|
||||
],
|
||||
"target": { "channelId": "111111111111111111" }
|
||||
"targets": [{ "platform": "discord", "channelId": "111111111111111111" }]
|
||||
}
|
||||
```
|
||||
|
||||
The `"draft": true` pattern matches the `draft` field that GitHub includes in pull request payloads; combined with `exclude: true` it filters out draft PRs.
|
||||
The `field` filter reads `pull_request.draft` from the payload; combined with `exclude: true` it filters out draft PRs.
|
||||
|
||||
## Worked Example 2: Release-only channel
|
||||
|
||||
@@ -298,7 +303,7 @@ Forward only published releases from a specific repo:
|
||||
{ "type": "action", "match": "published" },
|
||||
{ "type": "repo", "match": "myorg/backend" }
|
||||
],
|
||||
"target": { "channelId": "222222222222222222" }
|
||||
"targets": [{ "platform": "discord", "channelId": "222222222222222222" }]
|
||||
}
|
||||
```
|
||||
|
||||
@@ -317,14 +322,14 @@ Forward workflow runs that ended in failure on any branch, to a `#ci` channel:
|
||||
{ "type": "action", "match": "completed" },
|
||||
{ "type": "keyword", "match": "\"conclusion\":\"failure\"" }
|
||||
],
|
||||
"target": { "channelId": "333333333333333333" }
|
||||
"targets": [{ "platform": "discord", "channelId": "333333333333333333" }]
|
||||
}
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Wildcards are globs, not regex.** `repo: "myorg/*"` matches any repo under `myorg` (and `myorg/sub/backend`), but `repo: "myorg/.*"` matches literally. Use `//` wrapping for regex: `"/myorg\/.*/"`.
|
||||
- **A `//`-wrapped invalid regex never matches.** Unlike plain text, an unwrapped invalid pattern is matched literally — wrap patterns only when they are real regular expressions.
|
||||
- **A `//`-wrapped invalid regex is rejected.** The console and API refuse it when saved; a route stored with one before is reported as [unreadable](./routes#unreadable-routes) and its group's deliveries retry until it is fixed. Unlike `//`-wrapped text, an unwrapped pattern is matched literally — wrap patterns only when they are real regular expressions.
|
||||
- **An `action` filter on an action-less event never matches.** Check the event has an `action` field first (see [Filter Compatibility](../events/supported#filter-compatibility)).
|
||||
- **`branch` on an event without a branch never matches.** A `branch` filter on an `issues` event will always be false. Use `keyword` if you need branch-like matching there.
|
||||
- **`keyword` searches everything.** Because it scans the whole payload, a pattern like `"fix"` can match commit messages, issue titles, _and_ repository names. Be as specific as possible.
|
||||
|
||||
@@ -33,12 +33,12 @@ GITHUB_CLIENT_ID=your-client-id
|
||||
GITHUB_CLIENT_SECRET=your-client-secret
|
||||
DISCORD_TOKEN=your-bot-token
|
||||
DISCORD_PUBLIC_KEY=your-public-key
|
||||
ADMIN_USER_IDS=your-github-id,your-github-login
|
||||
ADMIN_USER_IDS=your-github-id
|
||||
BASE_URL=http://localhost:8787
|
||||
```
|
||||
|
||||
::: tip
|
||||
`GITHUB_APP_ID` / `GITHUB_PRIVATE_KEY` are optional and only used by the App install flow to resolve the installing account's login (the OAuth flow only needs the client ID/secret). Target channels are set per route in the Web UI, so no `DISCORD_CHANNEL_ID` is needed. To enable `/gh` commands locally, copy the **Public Key** from the Developer Portal into `DISCORD_PUBLIC_KEY` and set the Interactions Endpoint URL to `http://localhost:8787/discord/interactions`.
|
||||
`GITHUB_APP_ID` / `GITHUB_PRIVATE_KEY` are only needed for the GitHub App install flow and API enrichment: without them only super admins can bind installations, because the flow cannot verify that a signed-in user manages the installation (the OAuth flow itself only needs the client ID/secret). Target channels are set per route in the Web UI, so no `DISCORD_CHANNEL_ID` is needed. To enable `/gh` commands locally, copy the **Public Key** from the Developer Portal into `DISCORD_PUBLIC_KEY` and set the Interactions Endpoint URL to `http://localhost:8787/discord/interactions`.
|
||||
:::
|
||||
|
||||
::: warning
|
||||
@@ -48,10 +48,11 @@ BASE_URL=http://localhost:8787
|
||||
### 2. Start Dev Server
|
||||
|
||||
```bash
|
||||
bun run dev
|
||||
bun run db:migrate # first run: apply the D1 migrations to the local database
|
||||
bun run cf:dev # build, then serve the worker with wrangler dev (Miniflare)
|
||||
```
|
||||
|
||||
This starts a local Miniflare environment at `http://localhost:8787`.
|
||||
This serves the built worker at `http://localhost:8787`. Without the migrations the worker refuses to serve (`D1 migrations are incomplete`). For UI work, `bun run dev` starts the Nuxt dev server with HMR instead (`http://localhost:3000`; queue messages have no consumer there).
|
||||
|
||||
### 3. Verify
|
||||
|
||||
@@ -66,7 +67,7 @@ curl http://localhost:8787/health
|
||||
| ---------------------- | ------------------------------------------- |
|
||||
| `bun run dev` | Start Nuxt dev server (HMR) |
|
||||
| `bun run build` | Production build (cloudflare_module preset) |
|
||||
| `bun run deploy` | Deploy to Cloudflare |
|
||||
| `bun run deploy` | Apply pending D1 migrations and deploy |
|
||||
| `bun run typecheck` | TypeScript type checking |
|
||||
| `bun run lint` | ESLint |
|
||||
| `bun run lint:md` | Markdownlint |
|
||||
|
||||
+33
-18
@@ -1,6 +1,6 @@
|
||||
# Groups & Access Control
|
||||
|
||||
Routes belong to groups. Groups scope admin access and can restrict which events flow into them. They are stored in D1 (`d1_groups`, seeded from the legacy KV `config:groups` key on first load), managed via the [Web UI](./configuration#web-ui) or the [Admin API](../api/admin). At most **100 groups** can be saved per instance.
|
||||
Routes belong to groups. Groups scope admin access and can restrict which events flow into them. They are stored in D1 (`d1_groups`, seeded from the legacy KV `config:groups` key on first load), managed via the [Web UI](./configuration#web-ui) or the [Admin API](../api/admin). The bulk groups write accepts at most **100 groups** per save.
|
||||
|
||||
## Group Schema
|
||||
|
||||
@@ -20,19 +20,22 @@ Routes belong to groups. Groups scope admin access and can restrict which events
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
| ---------------- | -------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | string | Yes | Lowercase id (`a-z0-9`, `-`); referenced by each route's `groupId`. Editable: renaming a group re-points its routes, per-group webhook secret and pending invites |
|
||||
| `name` | string | Yes | Human-readable group name |
|
||||
| `members` | object[] | No | `{ login, role }` entries; role is `owner`, `admin`, or `viewer` |
|
||||
| `adminIds` | string[] | No | Deprecated legacy field; treated as `members` with role `owner` when present |
|
||||
| `owners` | string[] | No | Org/user logins whose events are accepted into this group; empty = all |
|
||||
| `providers` | string[] | No | Source platforms allowed into this group (`github`, `gitea`); empty = all |
|
||||
| `installationId` | number | No | GitHub App installation id bound to this group; only that installation's events are accepted (empty = all) |
|
||||
| `emoji` | boolean | No | Whether to include emoji in this group's messages (default `true`) |
|
||||
| `forgeSources` | object[] | No | Forge hosts: `{ host, type, name? }` entries (`type` is `github` or `gitea`, `name` is an optional display label) that label this group's message footers; empty = no label |
|
||||
| `lang` | string | No | Message language for every route in this group (e.g. `en`, `zh`; custom via KV `i18n:<lang>`) — see [Message Language](./i18n) — defaults to `en` |
|
||||
| `logTarget` | object | No | Webhook log channel: a Discord `{ platform, channelId, threadId? }` or Telegram `{ platform, chatId, topicId? }` target that receives a summary of every webhook the group's routes dispatch |
|
||||
| Field | Type | Required | Description |
|
||||
| ---------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `id` | string | Yes | Lowercase id (`a-z0-9`, `-`); referenced by each route's `groupId`. Editable: renaming re-points its routes, fragments, history, webhook secret and pending invites; a deleted or renamed id stays retired for a few days |
|
||||
| `name` | string | Yes | Human-readable group name |
|
||||
| `members` | object[] | Yes¹ | `{ login, role }` entries; role is `owner`, `admin`, or `viewer`. Each entry is bound to a GitHub user id by the server (resolved when the member is added, or at their first sign-in); a bound entry matches only that account, even after the login changes hands |
|
||||
| `adminIds` | string[] | No | Deprecated legacy field; treated as `members` with role `owner` when present |
|
||||
| `owners` | string[] | No | Org/user logins whose events are accepted into this group; empty = all |
|
||||
| `providers` | string[] | No | Source platforms allowed into this group (`github`, `gitea`, `custom`); empty = all |
|
||||
| `installationId` | number | No | GitHub App installation id bound to this group; only that installation's events are accepted (empty = all). Super-admin only (set by the verified install flow) |
|
||||
| `isolated` | boolean | No | Isolated groups (self sign-up personal groups) only receive events from their own `/webhook/{groupId}` ingress until a super admin binds a source (`installationId` or `owners`). Super-admin only |
|
||||
| `emoji` | boolean | No | Whether to include emoji in this group's messages (default `true`) |
|
||||
| `forgeSources` | object[] | No | Forge hosts: `{ host, type, name? }` entries (`type` is `github` or `gitea`, `name` is an optional display label) that label this group's message footers; empty = no label |
|
||||
| `lang` | string | No | Message language for every route in this group (e.g. `en`, `zh`; custom via KV `i18n:<lang>`) — see [Message Language](./i18n) — defaults to `en` |
|
||||
| `logTarget` | object | No | Webhook log channel: a Discord `{ platform, channelId, threadId? }`, Telegram `{ platform, chatId, topicId? }` or Feishu `{ platform, chatId }` target that receives a summary of every webhook the group's routes dispatch |
|
||||
|
||||
¹ `members` or the legacy `adminIds` must be present. A non-empty `members` list needs at least one `owner`; an empty list is allowed (only super admins manage such a group). `forgeSources` takes at most 20 entries, each `name` at most 50 characters, and no two entries with the same type and host.
|
||||
|
||||
## Roles
|
||||
|
||||
@@ -46,16 +49,21 @@ Every group member has one of three roles. Super admins (`ADMIN_USER_IDS`) alway
|
||||
|
||||
## Access Model
|
||||
|
||||
- **Super admins** (`ADMIN_USER_IDS`) see and edit every group and all routes; only they can edit a group's `owners` list.
|
||||
- **Super admins** (`ADMIN_USER_IDS`) see and edit every group and all routes; only they can edit a group's source-isolation fields (`owners`, `installationId`, `isolated`) — an owner's save keeps the stored values.
|
||||
- **Owners** manage their group's routes, members, invites, name, id, `emoji`, and `providers`. They cannot remove the last owner or demote themselves when no other owner remains.
|
||||
- **Admins** edit routes inside their groups and view logs; **viewers** get a read-only console.
|
||||
- Group admin endpoints operate on a single group at a time via `/admin/api/groups/:id/routes`; `groupId` is forced from the path parameter.
|
||||
- The `owners` list restricts which event actors (sender logins) the group's routes will dispatch at all.
|
||||
- The `owners` list restricts the group's routes to events of repositories owned by these users or organizations (`repository.owner.login` / `organization.login`); to filter by who triggered an event, use an `actor` filter.
|
||||
- A route whose group no longer exists never fires.
|
||||
- Every group write is a single-row, compare-and-swap update: two people editing different groups never overwrite each other, and an edit made from a stale copy of the same group is refused (`409`, the console reloads) instead of reverting the newer change. Route and fragment lists work the same way per group.
|
||||
- Roles are checked again inside the write itself: a request authorized before the writer lost their role (or was removed) is refused when it commits.
|
||||
- Deleting a group requires the confirmation snapshot the console shows (`revision`, `routesRevision` and the group `version`): anything added or changed after the confirmation — a route, a fragment, a setting — makes the delete fail with `409` instead of being deleted unseen. A group with routes that cannot be read is not deleted.
|
||||
- Delivery targets are owned: a non-super admin cannot point routes or the log channel at a channel/chat another group already delivers to, unless they may edit that group's routes too (`403`). A channel that two groups already shared before keeps working only where it is already used.
|
||||
- The `providers` list restricts which forge's events (`github`, `gitea`) the group's routes will dispatch. This lets you keep GitHub and Gitea groups separate even when org/user names collide.
|
||||
|
||||
## Webhook Log Channel
|
||||
|
||||
A group may set `logTarget` to a Discord channel/thread or Telegram chat/topic. Whenever the group's routes dispatch a webhook, a single summary message is sent there: the event type/action, the repo, the delivery id, and one line per route×target with an ✅/❌ outcome (including the error for failed sends; at most the first 10 lines are listed, the rest is summarized as `+N`). The message is green when every dispatch succeeded and red when any failed. The summary uses the group's message language. Log messages are sent best-effort and are not themselves recorded in the D1 send log.
|
||||
A group may set `logTarget` to a Discord channel/thread, Telegram chat/topic or Feishu chat. Whenever the group's routes dispatch a webhook, a single summary message is sent there: the event type/action, the repo, the delivery id, and one line per route×target with an ✅/❌ outcome (including the error for failed sends; targets finished by an earlier attempt are listed as such; at most the first 10 lines are listed, the rest is summarized as `+N`). The message is green when every dispatch succeeded, red when any failed, and gray when earlier attempts' outcomes are not known. A retried delivery with the same outcome does not send the summary again. The summary uses the group's message language. Log messages are sent best-effort and are not themselves recorded in the D1 send log.
|
||||
|
||||
## Forge Source Label
|
||||
|
||||
@@ -79,8 +87,15 @@ The label is independent of `Group.emoji` and follows every message that group d
|
||||
|
||||
## Invites
|
||||
|
||||
Owners (and super admins) can create single-use invite links valid for 7 days from the group's _Members_ panel. Accepting an invite adds the user with the invited role (`admin` or `viewer` — never `owner`); an existing `viewer` is upgraded to `admin`. Invites are stored in KV as `invite:{token}`.
|
||||
Owners (and super admins) can create single-use invite links valid for 7 days from the group's _Members_ panel. Accepting an invite adds the user with the invited role (`admin` or `viewer` — never `owner`); an existing `viewer` is upgraded to `admin`. Invites are stored in KV as `invite:{token}` for their whole lifetime; the "used" record commits in the same D1 transaction as the membership change, so a link can be used only once even when two people open it at the same time.
|
||||
|
||||
- An invite belongs to the group instance that issued it: a group later created under the same id never accepts it.
|
||||
- Revoking an invite is recorded in D1 and takes effect at once, even where KV still has the invite.
|
||||
- An invite stops working when its issuer is no longer an owner of the group.
|
||||
- If the login is still on a member entry bound to another GitHub account (the former holder of that login), accepting fails with a conflict and the invite stays usable; an owner first removes the stale entry.
|
||||
|
||||
## Self Sign-up
|
||||
|
||||
With `ALLOW_SELF_SIGNUP=1`, a GitHub user who has no group access gets a personal group (`u-{userId}`, owned by them) on first login instead of a `403`. This is the entry point for a fully self-service SaaS install; disable it to keep the console invite-only.
|
||||
|
||||
Personal groups are **isolated** (groups named `u-<digits>` created before the flag existed count as isolated too, unless a super admin sets `isolated: false`): the operator's global `POST /webhook` endpoint never dispatches to them, so a self-service user cannot receive other tenants' events by adding a catch-all route. They receive events through their own [per-group webhook](./ingress) — or from a GitHub App installation they prove they administer (the install flow checks it with the user's own GitHub token).
|
||||
+7
-5
@@ -8,17 +8,19 @@ Set `Group.lang` (e.g. `"zh"`) to choose the message language for every route in
|
||||
|
||||
## Custom Overrides
|
||||
|
||||
Translation overrides are merged from KV under the key `i18n:<lang>` as a flat JSON object of key → text. Any key from the built-in dictionaries (en/zh) can be overridden; unknown keys fall back to English.
|
||||
Additional message languages are read from KV under the key `i18n:<lang>`: a JSON object with the same nested structure as the built-in dictionaries (`server/lib/lib/locales/en.ts`). Every key you do not provide falls back to English. The built-in `en` and `zh` are never read from KV, so they cannot be overridden this way; use another language code (e.g. `zh-custom`) and set it as the group's `lang`.
|
||||
|
||||
```jsonc
|
||||
// KV key: i18n:zh
|
||||
// KV key: i18n:zh-custom
|
||||
{
|
||||
"events.push.title": "{repo}: 推送了 {count} 个提交到 {ref}",
|
||||
"events": {
|
||||
"push": { "title": "{repo}: 推送了 {count} 个提交到 {ref}" },
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Overrides apply to group messages (and console UI where applicable). To add a brand-new language, store a full dictionary at `i18n:<lang>` — every key you do not provide falls back to English.
|
||||
A loaded dictionary stays cached in each worker instance until it restarts, so a changed override may take a while to apply; a language with no override is looked up again after 5 minutes. Overrides affect group messages only — the console has its own built-in translations.
|
||||
|
||||
## Emoji Toggle
|
||||
|
||||
`Group.emoji` (default `true`) controls whether event emoji appear in that group's messages. When disabled, all emoji are stripped from titles, descriptions, fields, and links. Milestone progress bars (🟢🟡🟠⬜) are data visualization and stay regardless of the toggle.
|
||||
`Group.emoji` (default `true`) controls whether event emoji appear in that group's messages. When disabled, the emoji the formatters add are left out of titles, descriptions, fields, and links; text taken from the payload (commit messages, issue titles, custom webhook fields) keeps any emoji it contains. Milestone progress bars (🟢🟡🟠⬜) are data visualization and stay regardless of the toggle.
|
||||
+27
-18
@@ -11,24 +11,29 @@ WebHooker ingests webhooks from multiple forges through the same `POST /webhook`
|
||||
|
||||
Delivery-id dedup uses `X-GitHub-Delivery` (GitHub) or `X-Gitea-Delivery` (Gitea) when present.
|
||||
|
||||
With the delivery queue enabled, a webhook is acknowledged (`200`) only after it was durably enqueued. If the queue rejects it, the endpoint answers `503` and releases the dedup claim, so the sender's redelivery is accepted instead of being dropped as a duplicate. A redelivery of a webhook that was already accepted answers `200 { "duplicate": true }`; one that arrives while the first copy is still being processed answers `503`, so the sender tries again later.
|
||||
|
||||
Without the queue binding, the webhook is dispatched inline and answered only once dispatch finished: `200` when every target was handled, `503` when a target failed transiently (the sender's redelivery then skips the targets that already succeeded). Inline dispatch stops sending after 8 minutes and answers `503` for the rest, so one request never outlives its dedup claim.
|
||||
|
||||
Gitea payloads are normalized to the same internal shape as GitHub events, so routes, filters, and the 28 formatters work unchanged. Unknown or unmapped Gitea events fall back to the generic formatter. Repository/commit/user links are derived from the payload's `repository.html_url`, so they point at your Gitea instance.
|
||||
|
||||
## Global Endpoint (`POST /webhook`)
|
||||
|
||||
The global endpoint verifies payloads against the operator's global secrets (`GITHUB_WEBHOOK_SECRET`, `GITEA_WEBHOOK_SECRET`) and dispatches into **all** routes. GitHub App installations deliver here; use `installationId` on groups to keep tenants isolated.
|
||||
The global endpoint verifies payloads against the operator's global secrets (`GITHUB_WEBHOOK_SECRET`, `GITEA_WEBHOOK_SECRET`) and dispatches into **all** routes — except those of [isolated groups](./groups#self-sign-up) without a bound source. GitHub App installations deliver here; use `installationId` on groups to keep tenants isolated.
|
||||
|
||||
## Per-Group Endpoint (`POST /webhook/{groupId}`)
|
||||
|
||||
Every group can opt into its own webhook ingress with an independent secret (generated from the group page — Webhook endpoint panel, owner role). Payloads are verified against the **group's** secret instead of the global ones, and only that group's routes are eligible. This is how SaaS users configure Gitea, classic GitHub, or custom webhooks without sharing (or knowing) the operator's secrets.
|
||||
|
||||
- Supported for any provider: GitHub (`X-Hub-Signature-256`), Gitea (`X-Gitea-Signature`), custom (`X-WebHooker-Signature`)
|
||||
- The secret is a 64-char hex string; regenerate from the console invalidates the old one immediately
|
||||
- Delivery-id dedup keys are provider- and tenant-scoped (`delivery:{provider}:{groupId}:{id}`)
|
||||
- When the group has no secret (or no longer exists) the endpoint returns `404`
|
||||
- The secret is a 64-char hex string stored with the group in D1 (it is never sent to the console after generation); regenerating invalidates the old one immediately, and only a current owner can rotate or remove it
|
||||
- Delivery-id dedup keys are provider- and tenant-scoped (`delivery:{provider}:@i:{instance}:{id}` — the group's immutable instance, so a rename keeps the scope and a group re-created under the same id never shares it; the global endpoint uses the scope `@global`)
|
||||
- When the group has no secret (or no longer exists) the endpoint returns `404`; when the group's stored configuration cannot be read it returns `503`
|
||||
- A tenant-signed payload is only as trustworthy as the tenant: its `installation.id` is ignored, so it never uses the operator's GitHub App credentials (no job/build-log enrichment) and never provisions or rebinds installation groups; the installation restriction does not apply to the group's own ingress
|
||||
|
||||
## Custom Webhooks
|
||||
|
||||
Post arbitrary JSON to `POST /webhook/{groupId}` (or the global endpoint) with the body signed as `X-WebHooker-Signature: sha256=<hmac-sha256 hex of the raw body>` using the group's secret. The payload becomes a `custom` event that flows through the normal route pipeline — create a route with `event: custom` (there is a console template) and it dispatches to that route's targets, records `send_logs`, and appears in the group's webhook log channel.
|
||||
Post a JSON object to `POST /webhook/{groupId}` (or the global endpoint) with the body signed as `X-WebHooker-Signature: sha256=<hmac-sha256 hex of the raw body>` using the group's secret (a top-level array or scalar is rejected with `400`). The payload becomes a `custom` event that flows through the normal route pipeline — create a route with `event: custom` (there is a console template) and it dispatches to that route's targets, records `send_logs`, and appears in the group's webhook log channel.
|
||||
|
||||
### Replay Protection
|
||||
|
||||
@@ -40,7 +45,7 @@ Custom webhooks support optional replay protection via two extra headers alongsi
|
||||
When **both** headers are present, the signature is computed over `{timestamp}.{nonce}.{rawBody}` instead of the raw body, and the request is accepted only if:
|
||||
|
||||
1. The timestamp is within ±5 minutes of the server clock (rejects replays and clock-drift abusers)
|
||||
2. The nonce has never been seen before (stored in KV for 10 minutes; a replayed nonce is rejected)
|
||||
2. The nonce has never been seen before for this secret (claimed atomically and kept for 10 minutes; a replayed nonce is rejected, while another tenant may use the same nonce). A nonce whose first request is still being processed, or a failure of the nonce store, answers `503` instead of `401`, so the sender retries
|
||||
|
||||
```bash
|
||||
input="${timestamp}.${nonce}.${body}"
|
||||
@@ -69,22 +74,26 @@ Payload schema:
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `title` | string | Message title (falls back to "Custom message") |
|
||||
| `description` | string | Optional message body |
|
||||
| `color` | string | Optional embed color: a word (`red`, `green`, `yellow`, `blue`, `purple`, `orange`, `cyan`, `gray`) or `#rrggbb` |
|
||||
| `url` | string | Optional link for the title |
|
||||
| `repo` | string | Optional `owner/repo`; prefixes the title and is used as the footer |
|
||||
| `author` | object | Optional `{ name, iconUrl, url }` |
|
||||
| `fields` | object[] | Optional embed fields `{ name, value, inline }` |
|
||||
| `footer` | string | Optional footer override |
|
||||
| `deliveryId` | string | Optional id for sender-side dedup (retries) |
|
||||
| Field | Type | Description |
|
||||
| ------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `title` | string | Message title (falls back to "Custom message") |
|
||||
| `description` | string | Optional message body |
|
||||
| `color` | string | Optional embed color: a word (`red`, `green`, `yellow`, `blue`, `purple`, `orange`, `cyan`, `gray`) or `#rrggbb` |
|
||||
| `url` | string | Optional link for the title |
|
||||
| `repo` | string | Optional `owner/repo`; prefixes the title and is used as the footer |
|
||||
| `author` | object | Optional `{ name, iconUrl, url }` |
|
||||
| `fields` | object[] | Optional embed fields `{ name, value, inline }` |
|
||||
| `footer` | string | Optional footer override |
|
||||
| `deliveryId` | string | Optional id for sender-side dedup (retries). Without a `deliveryId` the nonce identifies the request; with neither, a retried request is delivered again |
|
||||
|
||||
## GitHub App Tenant Isolation
|
||||
|
||||
When the GitHub App is installed, its events arrive at the global endpoint for **every** installation. To keep tenants apart, bind each group to the installation id that should feed it: `"installationId": 12345678`. The id is visible in the App's installation webhook payload (`installation.id`) or on the GitHub App installation page URL. Events from any other installation are rejected for that group even if its `owners` list is empty. Groups without `installationId` keep the legacy behavior (`owners` filtering).
|
||||
|
||||
Binding is **auto-configured** — the GitHub App's _Setup URL_ should point to `{BASE_URL}/auth/github/install`. Right after a user installs the App, the browser lands there (the page requires a signed-in admin session — not signed in users are redirected through the OAuth flow first) and they choose where the installation binds: a **new group** (`inst-{installationId}`, default) or any **existing group they own** (owner role checked again on submit; `POST /auth/github/install/bind` performs the provisioning). No manual id entry is needed. As a fallback (e.g. when the Setup URL is not configured), the `installation.created` webhook event creates/binds the group automatically — existing groups whose `owners` match the installing account are bound, otherwise a dedicated `inst-{installationId}` group is created. Then just add routes/members in the console.
|
||||
Binding is **auto-configured** — the GitHub App's _Setup URL_ should point to `{BASE_URL}/auth/github/install`. Right after a user installs the App, the browser lands there (the page requires a signed-in admin session — not signed in users are redirected through the OAuth flow first) and they choose where the installation binds: a **new group** (`inst-{installationId}`, default) or any **existing group they own** (owner role checked again on submit; `POST /auth/github/install/bind` performs the provisioning). No manual id entry is needed.
|
||||
|
||||
The `installation_id` in the Setup URL can be set to anything, so both the page and the bind step first verify — with the signed-in user's own GitHub token — that the user may manage that installation: they are the installation's own account, or an active **admin** of its organization (merely having access to it as a collaborator is not enough). Otherwise they are sent back with `?error=install-forbidden`. Super admins may bind any installation. With `ALLOW_SELF_SIGNUP`, the installer becomes owner of the dedicated `inst-{installationId}` group only while it is bound to that installation and has no members yet — no other group is ever handed to a new owner this way. Ids starting with `inst-` or `u-` are reserved for these system groups: only super admins may create or rename groups to them, and an existing unbound `inst-*` group is never bound automatically.
|
||||
|
||||
As a fallback (e.g. when the Setup URL is not configured), the `installation.created` webhook event — on the global endpoint only — creates/binds the group automatically: existing groups whose `owners` match the installing account are bound, otherwise a dedicated `inst-{installationId}` group is created. Then just add routes/members in the console. The provisioning runs after the delivery's dedup check and is recorded per installation and per bound group for 7 days, so a redelivered event never re-binds a group a super admin has unbound since; if provisioning fails, the endpoint answers `503` so the event is redelivered.
|
||||
|
||||
To show the installing account's login on the choice page, set `GITHUB_APP_ID` and `GITHUB_PRIVATE_KEY` — see [Secrets](./configuration#secrets).
|
||||
@@ -22,7 +22,7 @@ GitHub / Gitea Webhook → Cloudflare Worker (Nuxt 4 / Nitro)
|
||||
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Cloudflare Worker** | HTTP ingress, signature verification, delivery dedup, event parsing, route matching, platform dispatch |
|
||||
| **Interactions Endpoint** | Verifies Ed25519 signatures and handles `/gh` interactions (slash commands, context-menu commands, buttons, modals) |
|
||||
| **KV** | Token storage (`token:{userId}`), OAuth state (`state:{hex}`), admin sessions, per-group secrets, config cache, delivery dedup/state/message-tracking fallback when D1 is unavailable (`delivery:*`, `delivery-state:*`, `msg:*`), message-update locks (`msg:lock:*`) |
|
||||
| **KV** | Token storage (`token:{userId}`), OAuth state (`state:{hex}`), admin sessions, invites, delivery dedup/state/message-tracking fallback when D1 is unavailable (`delivery:*`, `delivery-state:*`, `msg:*`), message-update locks (`msg:lock:*`) |
|
||||
| **D1** | Routes/groups (`d1_routes`/`d1_groups`), send logs (`send_logs`), audit logs (`audit_logs`), dedup (`dedup_keys`), delivery state (`delivery_state`), message tracking (`message_tracking`), Discord↔GitHub links (`discord_links`), Telegram↔GitHub links (`telegram_links`), with R2 optionally parking oversized payloads |
|
||||
|
||||
### Data Flow
|
||||
@@ -40,7 +40,7 @@ GitHub / Gitea Webhook → Cloudflare Worker (Nuxt 4 / Nitro)
|
||||
- **Runtime**: Cloudflare Workers
|
||||
- **HTTP Framework**: Nuxt 4 / Nitro (H3)
|
||||
- **Discord delivery**: Discord REST API (interactions via an Ed25519-verified HTTPS Interactions Endpoint)
|
||||
- **Telegram delivery**: Telegram Bot API (webhook with optional secret-token verification)
|
||||
- **Telegram delivery**: Telegram Bot API (webhook calls always verified with a secret token — explicit, or derived from the bot token)
|
||||
- **Web UI**: Nuxt 4 (Vue 3 + Tailwind CSS v4 + `@elysola/ui`) — server-rendered home/legal pages, client-side `/admin` console
|
||||
- **Storage**: Cloudflare D1 (authoritative for config & logs) + KV (cache/transient state) + optional R2 (oversized payloads)
|
||||
- **Auth**: Web Crypto API (HMAC-SHA256, Ed25519), octokit (GitHub API)
|
||||
|
||||
+26
-20
@@ -4,27 +4,33 @@
|
||||
|
||||
Every dispatch attempt is recorded in the D1 `send_logs` table and browsable in the console (**Logs** tab). Fields:
|
||||
|
||||
| Field | Meaning |
|
||||
| ------------ | ---------------------------------------------------------------- |
|
||||
| `routeId` | Route that matched |
|
||||
| `groupId` | Route's group |
|
||||
| `event` | Event type (e.g. `push`, `pull_request`, `custom`) |
|
||||
| `repo` | Repository full name (when present) |
|
||||
| `target` | Target id the message was sent to |
|
||||
| `platform` | `discord`, `telegram` or `feishu` |
|
||||
| `ok` | Whether the send succeeded |
|
||||
| `status` | HTTP status from the platform API (when applicable) |
|
||||
| `error` | Error message (when failed) |
|
||||
| `errorCode` | Stable error code (e.g. `NO_TARGET`, `NO_TOKEN`, `RATE_LIMITED`) |
|
||||
| `attempts` | Send attempts including retries |
|
||||
| `durationMs` | Time spent sending |
|
||||
| `deliveryId` | Webhook delivery id (when provided) |
|
||||
| `messageId` | Platform message id (used for in-place edits) |
|
||||
| `actor` | Sender login |
|
||||
| `action` | Event action (when present) |
|
||||
| `detail` | Extra JSON details (when present) |
|
||||
| Field | Meaning |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| `routeId` | Route that matched |
|
||||
| `groupId` | Route's group |
|
||||
| `event` | Event type (e.g. `push`, `pull_request`, `custom`) |
|
||||
| `repo` | Repository full name (when present) |
|
||||
| `target` | Target id the message was sent to |
|
||||
| `platform` | `discord`, `telegram` or `feishu` |
|
||||
| `ok` | Whether the send succeeded |
|
||||
| `status` | HTTP status from the platform API (when applicable) |
|
||||
| `error` | Error message (when failed) |
|
||||
| `errorCode` | Stable error code (e.g. `NO_TARGET`, `NO_TOKEN`, `RATE_LIMITED`, `FORMAT`, `PAYLOAD_MISSING`, `TELEGRAM_UNCERTAIN`) |
|
||||
| `attempts` | Send attempts including retries |
|
||||
| `durationMs` | Time spent sending |
|
||||
| `deliveryId` | Webhook delivery id (when provided) |
|
||||
| `messageId` | Platform message id (used for in-place edits) |
|
||||
| `actor` | Sender login |
|
||||
| `action` | Event action (when present) |
|
||||
| `detail` | Extra JSON details (when present) |
|
||||
|
||||
The console's **Logs** tab lists recent entries (filterable by group) and shows full details for a single entry. Entries are written best-effort — a failed insert never breaks dispatch.
|
||||
The console's **Logs** tab lists recent entries (filterable by group) and shows full details for a single entry. Entries are written best-effort — a failed insert is retried once and never breaks dispatch. `send_logs` is not pruned automatically.
|
||||
|
||||
Notable error codes:
|
||||
|
||||
- `PAYLOAD_MISSING` — the parked payload of a queued delivery is gone; the delivery stays retrying and the entry asks for the webhook to be redelivered from its source.
|
||||
- `TELEGRAM_UNCERTAIN` — Telegram answered `200` but the response could not be read; the message may or may not have been sent, so it is not retried (Telegram offers no idempotency key).
|
||||
- `FORMAT` — the event could not be formatted; nothing was sent for that route.
|
||||
|
||||
## Audit Log (`audit_logs`)
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ The repo comes from `payload.repository.full_name` (falling back to a generic "r
|
||||
|
||||
Only the repo head is hyperlinked — never the whole title:
|
||||
|
||||
- **Discord** (embed titles cannot contain partial links): the title is the repo head `{repo}{#number}`, linked to the repository; the `: {subject}` text is rendered as the first line of the description, unlinked.
|
||||
- **Discord** (embed titles cannot contain partial links): the title is the repo head `{repo}{#number}`, linked to the event's object (the issue, PR, comment or run page); the `: {subject}` text is rendered as the first line of the description, unlinked.
|
||||
- **Telegram** (HTML supports inline links): the one-line title keeps the subject, with only the repo head wrapped in a link.
|
||||
|
||||
Messages whose title has no colon separator (a `:` followed by a space) keep the legacy whole-title link behavior.
|
||||
@@ -37,7 +37,7 @@ With `Group.forgeSources` (a list of `{ host, type, name? }` entries) the messag
|
||||
|
||||
## In-Place Updates
|
||||
|
||||
`workflow_run` and `check_run` messages are sent once and edited in place as the run progresses (queued → running → success/failure) — no duplicate messages. Tracking uses KV `msg:*` with a stable `updateKey` per run: `workflow_run` keys on the workflow id, while `check_run` keys on the check name + commit SHA (falling back to the run id) so deployments that emit a fresh run id per phase — e.g. Cloudflare Pages — still edit the same message.
|
||||
Messages about an object that changes over time are sent once and edited in place as it changes — `workflow_run` and `check_run` / `check_suite` as the run progresses (queued → running → success/failure), deployments and their statuses, issue comments, pull request reviews and review comments as they are edited or deleted — instead of posting a new message. Each edit re-renders the whole message from the newest event. Tracking uses D1 `message_tracking` (KV `msg:*` as fallback) with a stable `updateKey` per object: `workflow_run` keys on the run id, `check_suite` on the suite id, while `check_run` keys on the check suite (or app) + check name + commit SHA (falling back to the run id) so deployments that emit a fresh run id per phase — e.g. Cloudflare Pages — still edit the same message. The chain is also keyed by the forge (provider + repository host) and the target platform, so two Gitea instances with the same `owner/repo` never edit each other's messages. An event older than the state a message already shows (delivered late) does not overwrite it; a failed edit is reported as a failure rather than posting a second message — a new message is sent only when the old one no longer exists.
|
||||
|
||||
## Workflow Jobs
|
||||
|
||||
|
||||
+16
-6
@@ -1,8 +1,8 @@
|
||||
# Routes & Targets
|
||||
|
||||
Routes define which events get forwarded to which channel (Discord or Telegram). They are stored in D1 (`d1_routes`, seeded from the legacy KV `config:routes` key on first load), managed via the [Web UI](./configuration#web-ui), the [Admin API](../api/admin), or `config.example.yaml`.
|
||||
Routes define which events get forwarded to which channel (Discord or Telegram). They are stored in D1 (`d1_routes`, seeded from the legacy KV `config:routes` key on first load), managed via the [Web UI](./configuration#web-ui) or the [Admin API](../api/admin). `config.example.yaml` shows the schema as YAML; the worker does not load it.
|
||||
|
||||
There are **no default routes** — each route must define its own target. If no routes are configured, no events are forwarded. At most **200 routes** can be saved per instance.
|
||||
There are **no default routes** — each route must define its own target. If no routes are configured, no events are forwarded. One save accepts at most **200 routes**.
|
||||
|
||||
## Route Schema
|
||||
|
||||
@@ -29,14 +29,20 @@ There are **no default routes** — each route must define its own target. If no
|
||||
}
|
||||
```
|
||||
|
||||
Each entry of `targets` is a push destination, so one route can forward to several channels at once (e.g. a Discord channel **and** a Telegram group). `target.platform` selects the platform: `discord` (default), `telegram` or `feishu`. For **Discord**, `target.channelId` is required (a thread in `target.threadId` is optional). For **Telegram** and **Feishu**, `target.chatId` (the group/supergroup chat id, e.g. `-1001234567890`) is required and `target.topicId` (the `message_thread_id` of a topic, equivalent of a Discord thread) is optional. There is no fallback to a default channel.
|
||||
Each entry of `targets` is a push destination, so one route can forward to several channels at once (e.g. a Discord channel **and** a Telegram group). `target.platform` selects the platform: `discord` (default), `telegram` or `feishu`. For **Discord**, `target.channelId` is required (a thread in `target.threadId` is optional). For **Telegram**, `target.chatId` (the group/supergroup chat id, e.g. `-1001234567890`) is required and `target.topicId` (the `message_thread_id` of a topic, equivalent of a Discord thread) is optional. For **Feishu**, `target.chatId` (the group's chat id, `oc_…`) is required; Feishu targets have no topic. There is no fallback to a default channel.
|
||||
|
||||
Target ids are validated so that every destination has exactly one spelling (otherwise the cross-group ownership check could be bypassed):
|
||||
|
||||
- **Discord** — `channelId` / `threadId` use only `A–Z a–z 0–9 _ -` (up to 64 characters); numeric ids have no leading zero.
|
||||
- **Telegram** — `chatId` is the numeric chat id in canonical form (no leading zeros, at most 15 digits; `@username` is not accepted); `topicId` is a positive integer up to 2147483647.
|
||||
- A route cannot list the same destination twice: `{ "channelId": T }` and `{ "channelId": P, "threadId": T }` both send to thread `T` and count as one destination, as do Telegram topic ids written differently.
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
| ---------------- | -------- | -------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| `groupId` | string | Yes | Id of the [group](./groups) this route belongs to |
|
||||
| `fallback` | boolean | No | When `true`, fires only if no non-fallback route of the same group matched the event; its filters still apply |
|
||||
| `stop` | boolean | No | When `true` and this route matches, no further routes of the same group are evaluated for this event |
|
||||
| `discordRoleIds` | string[] | No | Discord role ids to ping when this route fires; applied to Discord targets only |
|
||||
| `discordRoleIds` | string[] | No | Discord role ids to ping when this route fires (at most 50); applied to Discord targets only |
|
||||
| `ast` | object | No | Boolean filter tree (`{all:[...]}` / `{any:[...]}` / `{not:{...}}`); takes precedence over `filters` when present |
|
||||
|
||||
## Route Order
|
||||
@@ -45,6 +51,10 @@ Routes are evaluated in the order the console shows them, group by group. `stop`
|
||||
|
||||
**Upgrade note:** If your configuration relied on `stop` in one group preventing routes in another group from running, or on a regular route match in one group suppressing fallback sends in another, review your existing routes when upgrading. These effects now apply only within the same group, so routes in other groups may start sending. Move routes that need to be mutually exclusive into the same group, then check their order and `stop` / `fallback` settings.
|
||||
|
||||
## Unreadable Routes
|
||||
|
||||
A stored route that cannot be read — corrupt JSON, a field of the wrong shape, an invalid filter, or a pattern the matcher cannot compile (for example a lookahead regex written for another engine) — is never treated as "no match". The group's deliveries fail visibly and are retried (every other group is unaffected), and `GET /admin/api/groups/:id/routes` reports `unreadable: true`. Saving the group's routes is refused while such a row exists, so it is never deleted unseen; to replace it on purpose, send the save with `discardUnreadable: true`.
|
||||
|
||||
## Discord Role Mentions
|
||||
|
||||
Set `discordRoleIds` on a route to ping one or more Discord roles (身份组) whenever that route fires. The mention (`<@&roleId>`) is prepended to the message content of every **Discord** target of the route; Telegram targets ignore this field. Mentions only trigger notifications when the bot has the `Mention Everyone` permission (or the role is marked mentionable), and the bot must be able to see the role.
|
||||
@@ -79,13 +89,13 @@ Every route carries a `filters` array (all must match — AND logic). When the r
|
||||
"filters": [
|
||||
{ "type": "repo", "match": "myorg/backend" },
|
||||
{ "type": "event", "match": "pull_request" },
|
||||
{ "type": "actor", "match": "[bot]", "exclude": true }
|
||||
{ "type": "actor", "match": "*[bot]", "exclude": true }
|
||||
],
|
||||
"targets": [
|
||||
{
|
||||
"platform": "telegram",
|
||||
"chatId": "-1001234567890",
|
||||
"topicId": "9876543210"
|
||||
"topicId": "42"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
+53
-37
@@ -4,58 +4,74 @@
|
||||
|
||||
KV keeps only cache data and short-lived/ephemeral state. High-frequency writes (webhook dedup, delivery state, message tracking) live in D1 and only fall back to KV when D1 is unavailable or not yet migrated (see [Storage decisions](./storage#storage-decisions)).
|
||||
|
||||
| Key Pattern | Value | TTL |
|
||||
| ------------------------------------------ | ----------------------------------------------------------------------------- | ------------------ |
|
||||
| `config:routes` | Route config cache (D1 `d1_routes` is authoritative) | 1 hour |
|
||||
| `config:groups` | Group config cache (D1 `d1_groups` is authoritative) | 1 hour |
|
||||
| `session:{id}` | Admin session `{ userId, login }` | 7 days |
|
||||
| `token:{userId}` | `{ userId, accessToken, expiresAt, refreshToken? }` | 0.9 × token expiry |
|
||||
| `token-reverse:{sha256}` | User id for reverse lookup by token | 0.9 × token expiry |
|
||||
| `state:{hex}` | `{ redirectTo, expiresAt, discordUserId?, telegramUserId?, telegramChatId? }` | 600 seconds |
|
||||
| `invite:{token}` | `{ groupId, role, expiresAt, createdBy, note? }` | 7 days |
|
||||
| `invite:group:{id}` | Token index per group (keeps invite listing consistent) | 7 days |
|
||||
| `delivery:{provider}:{groupId}:{id}` | Webhook delivery dedup fallback (D1 `dedup_keys` is primary) | 7 days |
|
||||
| `delivery-state:{provider}:{groupId}:{id}` | Queue delivery state fallback (D1 `delivery_state` is primary) | 1 hour |
|
||||
| `queue:payload:{provider}:{groupId}:{id}` | Oversized webhook payload parked for the queue consumer (R2 is primary) | 1 hour |
|
||||
| `nonce:{nonce}` | Custom-webhook replay protection nonce (single use) | 600 seconds |
|
||||
| `tenant:{groupId}` | Per-group webhook secret (64-char hex, generated from the console) | Permanent |
|
||||
| `msg:{routeId}:{key}:{target}` | Message id tracking fallback (D1 `message_tracking` is primary) | 1 day |
|
||||
| `cmd:guild:{id}` | Guild id whose commands were registered (dedup) | Permanent |
|
||||
| `cmd:registered:global` | Global command registration marker (dedup) | 1 day |
|
||||
| `config:discord-app-id` | Cached Discord application id | Permanent |
|
||||
| `i18n:{lang}` | Translation overrides merged on top of English | Permanent |
|
||||
| Key Pattern | Value | TTL |
|
||||
| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ |
|
||||
| `config:routes` / `config:groups` | Legacy (pre-D1) configuration, read once to seed empty D1 tables | Permanent |
|
||||
| `config:d1-seeded` | Marks that D1 was seeded (an emptied D1 is never re-seeded) | Permanent |
|
||||
| `session:{id}` | Admin session `{ userId, login, loginTrusted }` | 1 day |
|
||||
| `token:{userId}` | `{ userId, accessToken, expiresAt, live? }` (a token is used for 10 hours) | 9 hours |
|
||||
| `token-reverse:{sha256}` | User id for reverse lookup by token | 9 hours |
|
||||
| `state:{hex}` | `{ redirectTo, expiresAt, link? }` (`link`: the chat account a bot `/gh login` binds) | 600 seconds |
|
||||
| `link:{nonce}` | A bot account link waiting for the GitHub user's confirmation | 600 seconds |
|
||||
| `invite:{token}` | `{ groupId, role, expiresAt, createdBy, note?, groupInstance, issuer }` | 7 days |
|
||||
| `invite:group:{id}` | Token index per group (keeps invite listing consistent) | 7 days |
|
||||
| `invite:by-group:{id}:{token}` / `invite:by-instance:{instance}:{token}` | Per-invite listing markers (by group id and by the issuing group's instance), so concurrent creates or a half-finished rename never hide an invite | Invite lifetime |
|
||||
| `login-owner:{login}` | GitHub user id of the first account seen with this login; another account using the login later is matched by user id only | Permanent |
|
||||
| `delivery:{provider}:{scope}:{id}` | Webhook delivery dedup fallback (D1 `dedup_keys` is primary); `scope` is `@global` for the global endpoint or `@i:{instance}` (the receiving group's immutable instance) for a group endpoint | 2–10 minutes |
|
||||
| `delivery-state:{provider}:{scope}:{id}` | Queue delivery state fallback (D1 `delivery_state` is primary) | 7 days |
|
||||
| `queue:payload:{provider}:{scope}:{id}:{uuid}` | Oversized webhook payload parked for the queue consumer when there is no R2 binding (one key per enqueue) | 4 days |
|
||||
| `tenant:{groupId}` / `tenant:i:{instance}` | Legacy per-group webhook secrets, read only for groups whose secret was never moved into their D1 row; removed on rotate/delete | Permanent |
|
||||
| `group:renamed:{oldId}` | KV-only deployments: where a renamed group went (with D1 this alias is a `dedup_keys` row written in the rename transaction) | 4 days |
|
||||
| `msg:{chain}:{platform}:{target}` | Message tracking fallback (D1 `message_tracking` is primary) | 30 days |
|
||||
| `msg:lock:…` | Lock serializing updates of one message chain when D1 is unavailable (with D1 it is a `dedup_keys` row) | 240 seconds |
|
||||
| `storage:kv-fallback` | Marks that state was written to KV while a D1 table was missing; while it lives, D1 misses are also looked up in KV | 35 days |
|
||||
| `cmd:guilds:{appId}` | Per Discord application: each guild's registered command hash and the sync cursor | Permanent |
|
||||
| `cmd:registered:global:{appId}` | Hash of the commands last registered globally (re-registered when it changes, and daily) | 1 day |
|
||||
| `config:discord-app-id:{tokenHash}` | Discord application id resolved for a bot token | Permanent |
|
||||
| `discord-modal:{id}` | Comment-modal context, only when it does not fit in the 100-character `custom_id` | 1 day |
|
||||
| `feishu:token:{appId}` | Feishu `tenant_access_token` per app (also held in memory) | Token expiry − 6 minutes |
|
||||
| `telegram:webhook:{botId}` | Hash of the webhook registration last set (`setWebhook` is skipped while it matches) | 1 day |
|
||||
| `i18n:{lang}` | Translation overrides merged on top of English | Permanent |
|
||||
| `config:identity-migrated` | KV-only deployments: group identities are managed by this server (the one-time identity migration ran) | Permanent |
|
||||
|
||||
## D1 Storage Layout
|
||||
|
||||
The D1 database (`DB` binding, database `webhooker`) holds the source of truth for configuration, delivery logs and high-frequency ephemeral state:
|
||||
|
||||
| Table | Purpose |
|
||||
| ------------------ | ---------------------------------------------------------------------------------------------- |
|
||||
| `d1_groups` | Groups (authoritative config, seeded from legacy KV `config:groups`) |
|
||||
| `d1_routes` | Routes per group (authoritative config, seeded from legacy KV `config:routes`) |
|
||||
| `send_logs` | One row per dispatch attempt (route id, event, target, ok/error, duration, error code, detail) |
|
||||
| `audit_logs` | One row per admin operation (login/logout, group/route/member/invite changes) |
|
||||
| `dedup_keys` | Webhook delivery dedup (atomic `INSERT ... ON CONFLICT` UPSERT, key + expiry) |
|
||||
| `delivery_state` | Queue delivery state (`pending`/`processing`/`delivered`/`retrying`/`failed`/`dead`) |
|
||||
| `message_tracking` | Message id tracking for in-place updates (`event_id` + `target_id` → `message_id`) |
|
||||
| `discord_links` | Maps `discord_user_id` → `github_user_id` for `/gh` Discord commands |
|
||||
| `telegram_links` | Maps `telegram_user_id` → `github_user_id` for `/gh` Telegram commands |
|
||||
| Table | Purpose |
|
||||
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `d1_groups` | Groups (authoritative config, seeded from legacy KV `config:groups`). Each row also holds server-only fields that are never sent to the console: the immutable `instance` id, `legacyId`, and the group's webhook secret |
|
||||
| `d1_routes` | Routes per group (authoritative config, seeded from legacy KV `config:routes`) |
|
||||
| `send_logs` | One row per dispatch attempt (route id, event, target, ok/error, duration, error code, detail) |
|
||||
| `audit_logs` | One row per admin operation (login/logout, group/route/member/invite changes) |
|
||||
| `dedup_keys` | Atomic one-time claims and records (`INSERT ... ON CONFLICT` UPSERT, key + expiry): webhook delivery dedup and its confirmations (`accepted:*`), custom-webhook nonces (`nonce:{secret-scope}:{nonce}`), single-use and revoked invites (`invite-used:*`, `invite-revoked:*`), bot command/event dedup (`bot:{platform}:…`), revoked sessions and tokens and each user's live token (`session-revoked:*`, `token-revoked:*`, `token-live:*`), used account-link confirmations (`link-used:*`), retired group ids (`group-retired:*`) and rename aliases (`group-alias:*`), installation provisioning records (`installation-provisioned:*`, `installation-bound:*`), route × targets already finished or failed for good by a queued delivery (`delivery-target:*`, `delivery-failed:*`), group log summaries already sent, message-chain locks (`msg:lock:*`) and one-time setup markers |
|
||||
| `delivery_state` | Queue delivery state (`pending`/`processing`/`delivered`/`retrying`/`failed`/`dead`); taking a delivery is a conditional UPSERT (a lease), so two consumers never dispatch it at once |
|
||||
| `message_tracking` | Message tracking for in-place updates (`event_id` + `target_id` → message id and the version it shows); chains are keyed by the group's instance, so a group created later under the same id never edits its messages |
|
||||
| `d1_fragments` | Named filter fragments per group |
|
||||
| `feishu_links` | Maps `feishu_user_id` → `github_user_id` for `/gh` Feishu commands |
|
||||
| `discord_links` | Maps `discord_user_id` → `github_user_id` for `/gh` Discord commands |
|
||||
| `telegram_links` | Maps `telegram_user_id` → `github_user_id` for `/gh` Telegram commands |
|
||||
|
||||
`audit_logs` is pruned by the scheduled trigger after `AUDIT_RETENTION_DAYS` (default 90). The `storage-prune` task removes expired `dedup_keys`, `delivery_state` rows older than 7 days and `message_tracking` rows older than 30 days. See [Logs](./logs) for the log row fields.
|
||||
`audit_logs` is pruned by the scheduled trigger after `AUDIT_RETENTION_DAYS` (default 90). The `storage-prune` task removes expired `dedup_keys`, `delivery_state` rows older than 7 days, `message_tracking` rows not updated for 30 days (every edit refreshes a row) and orphaned R2 payloads. `send_logs` is not pruned automatically. See [Logs](./logs) for the log row fields.
|
||||
|
||||
Migration `0013_prune_indexes.sql` adds the indexes these reads rely on: `delivery_state(updated_at)` for the prune, two partial indexes on failed `send_logs` rows for the metrics page, and `audit_logs(group_id, ts)` in place of the single-column group index. Every index also costs a D1 row write when its columns are written, so only indexes whose reads would otherwise cost far more are added.
|
||||
|
||||
## R2 Storage Layout
|
||||
|
||||
R2 (`PAYLOAD` binding, bucket `webhooker-payloads`) stores oversized webhook payloads that are too large for a queue message or KV:
|
||||
|
||||
| Object Pattern | Purpose | Retention |
|
||||
| --------------------------------- | ----------------------------------------------- | ---------------------- |
|
||||
| `webhooks/YYYY/MM/DD/<uuid>.json` | Oversized payload parked for the queue consumer | deleted after dispatch |
|
||||
| Object Pattern | Purpose | Retention |
|
||||
| --------------------------------- | ----------------------------------------------- | --------------------------------------------------- |
|
||||
| `webhooks/YYYY/MM/DD/<uuid>.json` | Oversized payload parked for the queue consumer | deleted after dispatch; orphans pruned after 5 days |
|
||||
|
||||
When the `PAYLOAD` binding is absent, oversized payloads fall back to the KV key `queue:payload:{provider}:{groupId}:{id}` (1 hour TTL).
|
||||
A payload is parked when the queue message would exceed 120,000 bytes (UTF-8). When the `PAYLOAD` binding is absent, oversized payloads fall back to the KV key `queue:payload:{provider}:{scope}:{id}:{uuid}` (4 day TTL, matching the queue's message retention). If sending the queue message fails, the parked copy is kept (the queue may have accepted the message even though the call failed); unused copies expire or are pruned. A parked payload that is gone when the consumer needs it is never dispatched as an empty event: the delivery stays `retrying` and a `PAYLOAD_MISSING` send-log entry asks for the webhook to be redelivered from its source.
|
||||
|
||||
## Storage Decisions
|
||||
|
||||
- **D1 is authoritative** for config and delivery metadata; KV holds only caches and short-lived state.
|
||||
- The `canUseD1` probe (`server/lib/storage/d1.ts`, checks for `prepare` + `batch`) gates every D1 store: when D1 is unavailable or not yet migrated, all three high-frequency stores (dedup, delivery state, message tracking) transparently fall back to KV so behavior is unchanged during migration.
|
||||
- The `canUseD1` probe (`server/lib/storage/d1.ts`, checks for `prepare` + `batch`) gates every D1 store. The high-frequency stores (dedup and other one-time claims, delivery state, message tracking) also fall back to KV while their D1 table does not exist yet (an unmigrated database). Any other D1 failure propagates — the request answers with an error and the sender or queue retries — because switching stores per call would split a claim between D1 and KV.
|
||||
- When D1 configuration reads fail, the request answers with an error and the sender or queue retries; there is no KV copy of the configuration, and dispatch never treats unreadable configuration as empty. A stored route or group row that cannot be read only makes its own group's deliveries fail visibly (and retry) — see [Routes → Unreadable routes](./routes#unreadable-routes).
|
||||
- Configuration writes never fall back: with a D1 binding, a failed write is reported as an error instead of being parked in KV. Writes are per group (one row, or one group's routes/fragments in one transaction) with compare-and-swap on the group row, so concurrent edits of different groups never overwrite each other. Reads for access checks, read-modify-write paths, webhook ingress and the queue consumer bypass every cache (a revoked route or target stops at once); other reads are cached in memory for at most 60 seconds per isolate.
|
||||
- Legacy KV configuration is copied into D1 once, only while both D1 tables are empty, and never over existing rows; a marker prevents an emptied configuration from being resurrected.
|
||||
- This keeps per-event KV writes near zero on the Workers Free plan (1,000 writes/day): dedup, delivery state and message tracking now write D1 rows instead (D1 Free allows 100,000 rows written/day).
|
||||
- R2's free tier (10 GB-month storage, 1M Class A ops/month) comfortably absorbs payload parking without touching the KV write quota.
|
||||
+6
-6
@@ -2,11 +2,11 @@
|
||||
|
||||
WebHooker runs four maintenance tasks on the scheduled trigger (`*/5 * * * *`, every 5 minutes). They only run on the deployed worker (Cloudflare cron); local `wrangler dev` runs them when triggered via `wrangler dev --test-scheduled`.
|
||||
|
||||
| Task | Purpose |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `discord-sync` | Registers the Discord slash/context-menu commands: per-guild (instant) and globally (24h dedup, ~1h propagation) |
|
||||
| `telegram-sync` | Calls `setWebhook` to `{BASE_URL}/telegram/webhook` (with `TELEGRAM_WEBHOOK_SECRET` as `secret_token` when set) |
|
||||
| `audit-prune` | Deletes `audit_logs` entries older than `AUDIT_RETENTION_DAYS` (default 90) |
|
||||
| `storage-prune` | Deletes expired `dedup_keys` rows, `delivery_state` rows older than 7 days and `message_tracking` rows older than 30 days |
|
||||
| Task | Purpose |
|
||||
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `discord-sync` | Registers the Discord slash/context-menu commands per guild (instant; at most 30 API calls per run, resuming where the last run stopped) and globally (~1h propagation); registered again when the command definitions change |
|
||||
| `telegram-sync` | Calls `setWebhook` to `{BASE_URL}/telegram/webhook` with `TELEGRAM_WEBHOOK_SECRET` (or a secret derived from `TELEGRAM_TOKEN`) as `secret_token` — only when the registration changed, and once a day |
|
||||
| `audit-prune` | Deletes `audit_logs` entries older than `AUDIT_RETENTION_DAYS` (default 90) |
|
||||
| `storage-prune` | Deletes expired `dedup_keys` rows, `delivery_state` rows older than 7 days, `message_tracking` rows older than 30 days and orphaned R2 payloads |
|
||||
|
||||
There is nothing to configure beyond the secrets the tasks use (`DISCORD_TOKEN`, `DISCORD_APPLICATION_ID`, `TELEGRAM_TOKEN`, `BASE_URL`, `AUDIT_RETENTION_DAYS`).
|
||||
Reference in new issue
Block a user