mirror of
https://github.com/Elysola/WebHooker.git
synced 2026-11-20 17:13:49 +00:00
fix(security): harden webhook delivery pipeline and group storage (#11)
* fix(log): preserve structured error details Error objects lose their useful details when serialized as plain JSON. Keep the message, stack and cause in structured log output. * fix(runtime): bound shared in-flight waits A canceled request can leave its shared promise pending forever. Bound joins and coordinate a single replacement so later requests make progress without multiplying upstream work. * fix(bot): reject oversized callback bodies Reject declared oversized bot callbacks before reading their bodies. This keeps unauthenticated requests from wasting worker resources. * test(d1): exercise real sqlite transactions Exercise real foreign keys, transactions and conditional statements instead of mocks that cannot reproduce lost updates or cascades. * fix(storage): preserve claims across kv fallback Only missing tables justify KV fallback; other failures must surface. Remember fallback writes so migration to D1 cannot forget claims and replay completed work. Share marker checks without extra writes. * fix(filter): use isolated linear-time patterns Backtracking patterns can exhaust worker CPU on untrusted payloads. Use linear-time matching and separate bounded cache namespaces so glob and regex entries cannot corrupt each other. * fix(filter): read each event branch correctly Deployment and check branches live in event-specific nested fields. Read those fields and remove only the actual leading ref prefix. * fix(filter): read only own payload properties Inherited object properties are not webhook payload data. Read own properties only so JSONPath cannot traverse prototypes. * fix(filter): validate executable condition trees Malformed trees and unsupported patterns must not silently turn an exclude rule into a pass. Validate the shape and only the patterns the evaluator actually executes. * fix(i18n): bound and cache translation loads Repeated misses and simultaneous override loads spend KV reads for the same language. Cache confirmed misses and share bounded loads without caching read failures as missing translations. * i18n(bot): explain command and account failures Explain account linking, command failures and skipped delivery outcomes consistently in both supported languages. * fix(github): bound and reuse app token fetches Repeated token exchanges waste requests and JWT work for every event. Reuse valid installation tokens, share refreshes and back off briefly after failures while keeping every shared wait bounded. * fix(github): restrict enrichment request urls Webhook-provided URLs must never receive an installation token on another host. Restrict enrichment to GitHub repository API paths, reject redirects and bound optional requests. * fix(formatter): preserve invalid code points Out-of-range numeric entities can throw while formatting a delivery. Keep invalid code points verbatim instead of aborting the message. * fix(formatter): preserve bounded markdown text Code spans, fences, indentation and autolinks are Markdown content, not HTML tags to strip. Preserve them while bounding cleanup input and reusing results across routes for the same event. * fix(formatter): strip only leading ref prefixes A short branch or tag may itself contain refs/heads or refs/tags. Strip one leading full-ref prefix at the caller and keep short names unchanged when generating labels and links. * fix(gitea): link push authors to their forge A Gitea author is not the same-named GitHub account. Build author profile links from the originating forge. * fix(formatter): render deployment state values The deployment status object stringifies as [object Object]. Render its actual state in the status field. * fix(formatter): read security severity levels The scanner severity is not the security severity of an alert. Prefer the security level so critical and high alerts get the right label and color. * fix(formatter): hide actions on other forges Action buttons call the GitHub API as the linked user. Hide them for other forges to avoid operating on a same-named GitHub repository. * fix(check): isolate updates by check suite Same-named checks from different suites must not edit one message. Include the suite or app identity and retain the former key for controlled adoption of existing tracked messages. * fix(logs): scope queries before limiting results Filter authorized groups in SQL before LIMIT and aggregate only visible rows. Preserve current group attribution for late writes, surface read failures and combine related totals in one scan. * perf(d1): index pruning and failed send queries Pruning and failure views should not scan all historical rows. Index the high-value access paths without adding a write-heavy index for infrequent delivery-detail lookups. * fix(discord): bound individual api requests An unbounded API request can outlive an update-chain lock. Bound each request so another holder cannot race a late edit. * fix(discord): defer long rate-limit waits Long rate-limit sleeps block delivery progress inside a worker. Return a retryable 429 for the queue when the wait exceeds the bound. * fix(discord): encode channel path segments Stored channel and message ids are path segments, not URL paths. Encode them so aliases cannot address another Discord resource. * fix(discord): reuse nonces after lost responses A successful create may lose its response. Reuse a stable nonce with enforce_nonce so retries recover that message instead of posting duplicates or accepting an unreadable success. * fix(discord): constrain embeds and role mentions Respect combined embed and component limits as well as field caps. Restrict allowed mentions to validated route roles so payload text cannot notify everyone. * fix(discord): resume bounded command syncs Application changes, command rollbacks and large guild lists can leave commands stale indefinitely. Track the current hash per app and resume bounded paginated work without needless KV writes. * fix(github): verify installation management An installation id is user-controlled, and repository access is not installation management. Verify the owning account or an active organization admin using the user token and read:org scope. * fix(github): resolve immutable member ids A login can be renamed and reclaimed before its first sign-in. Resolve the current immutable user id when membership is granted so a future holder of the login cannot inherit access. * fix(auth): revoke stale sessions and tokens Revoked or rotated credentials must stop working despite stale KV replicas. Track the live token and revocations in D1 and preserve immutable identity bindings rather than trusting a recycled login. * fix(auth): bind sessions and tokens to identity Automatic retries can repeat non-idempotent user actions. Disable those retries and expose a write hook so bot handlers can record completion immediately before the GitHub write. * fix(discord): defer and deduplicate interactions Acknowledge slow actions before Discord times out and deduplicate interaction writes. Preserve comment text and reject edits whose source changed after the modal opened. * fix(telegram): always verify webhook secrets Webhook verification must not silently disable itself when the optional secret is absent. Derive a stable secret from the bot token and reject callbacks when verification is unavailable. * fix(telegram): bound individual api requests Bound individual platform calls so they cannot outlive the lock protecting a message update or stall a whole delivery invocation. * fix(telegram): honor nested rate-limit delays Telegram places retry_after inside parameters, not at the top level. Read that value and defer long waits to the delivery queue. * fix(telegram): report unreadable send results An unreadable successful response may already have sent the message. Report an explicit uncertain result instead of retrying a create without an idempotency key or pretending it succeeded. * fix(telegram): keep editable messages as text Guessing photo versus text from message content makes later edits use the wrong API and a smaller caption budget. Always send text and use the avatar as a link preview. * fix(telegram): render hard breaks as newlines Telegram HTML does not support br tags. Render Markdown hard breaks as plain newlines so otherwise valid messages are accepted. * fix(telegram): clamp visible html characters Telegram counts visible characters, not HTML source bytes. Truncate without splitting tags, entities or surrogate pairs. * fix(telegram): honor the author emoji toggle The author marker must obey the group emoji preference just like formatter-added decoration. * fix(telegram): keep completed commands claimed Redelivery after a completed write must never post another comment. Keep completion claims despite reply failures, scope updates per bot and only issue account-login links in private chats. * fix(feishu): bound individual api requests Bound individual platform calls so they cannot outlive the lock protecting a message update or stall a whole delivery invocation. * fix(feishu): classify retryable api failures Business error codes can be transient even in HTTP 200 responses. Classify token and rate-limit errors for retry and defer long waits. * fix(feishu): retry sends with stable uuids An unreadable response does not prove a send failed. Reuse the same uuid for retries so recovering the result cannot create a second Feishu message. * fix(feishu): refresh and evict tokens safely Separate app token caches, stop reusing refused tokens and share refreshes. Cache failures briefly and keep valid tokens in memory when a best-effort KV cache write fails. * fix(feishu): use the card update endpoint Delayed card updates use the card update endpoint; the config endpoint cannot update an interaction response. * fix(feishu): neutralize untrusted mentions Forge text is untrusted and can contain at-all tags. Neutralize mentions in card bodies and footers before rendering. * fix(feishu): authenticate and dedupe callbacks Authenticate signed or token-verified callbacks before dispatching commands. Decode encrypted envelopes and handle card actions using the correct fields without claiming irrelevant chat events. * fix(target): normalize actual destinations Different target spellings can name the same actual destination. Normalize channel, thread and topic identities for ownership checks and per-route duplicate suppression. * fix(config): transact per-group configuration Whole-table writes and cached snapshots can overwrite other groups or resurrect removed configuration. Use per-group guarded writes, one-time identity migration and explicit unreadable-row failures. * fix(group): preserve identity through lifecycle Group names are mutable and reusable, so they cannot own durable state. Preserve immutable identities across lifecycle operations and commit invite use with membership under transaction-time guards. * fix(oauth): confirm account links and installs A login link or installation id is not proof of account ownership. Verify installation management, bind browser state and require an explicit account-link confirmation before granting chat access. * fix(queue): fence leases and record target outcomes Retries must remember completed targets and distinguish active lease holders. Fence state updates, preserve permanent outcomes and reject missing parked payloads instead of dispatching empty data. * fix(dispatch): isolate and resume tenant sends Check tenant sources before filters and never dispatch orphan routes. Record each target outcome, isolate update chains by forge and group identity, and keep stale events from overwriting newer messages. * fix(queue): bound batches and resume deliveries A queue invocation has shared query and subrequest limits. Use a small batch and shared send budget, renew fenced leases and resume only unfinished work without exhausting unstarted messages. * fix(admin): guard revisions and tenant targets Read-time authorization alone cannot protect concurrent edits. Guard revisions, current roles and destination ownership in the write transaction, and use per-group endpoints for isolated changes. * fix(webhook): confirm only accepted deliveries Acknowledging before queue acceptance permanently loses deliveries. Confirm only accepted work, release failed claims, scope replay identities and strip tenant-supplied installation authority. * fix(storage): prune abandoned parked payloads Oversized payloads can remain after interrupted enqueue attempts. Prune objects beyond their retry lifetime without deleting payloads that a current delivery may still need. * fix(console): expose api conflict statuses Callers need the HTTP status to distinguish stale edits from other failures and reload current data after a conflict. * refactor(console): model group revisions Carry server revisions explicitly so console writes can reject changes based on an outdated group snapshot. * fix(console): send loaded route revisions Send the revision actually loaded with the route list and adopt the server response after a save. Reload on conflict instead of silently overwriting another editor. * fix(console): guard fragment saves by revision Fragments share a group version with other configuration writes. Submit the loaded revision and reload on conflict to preserve concurrent edits. * fix(console): save groups with guarded writes Whole-list saves can overwrite unrelated groups. Use per-group revision-aware writes, retain rename results and confirm deletion on the exact group, routes and version shown to the user. * fix(console): reject edits of stale routes A conflict reload can leave an older route editor open. Compare its original value before saving so it cannot overwrite the newly loaded route. * fix(console): show overview loading failures Failed overview requests must not look like an empty dashboard or a successful refresh. Show the reason while retaining prior data. * fix(console): explain rejected install bindings Explain why an installation binding was refused instead of silently returning to the console with no visible outcome. * fix(console): allow selecting a group named all The all sentinel collides with a valid group id. Use a value that cannot be a group id so that group stays selectable. * fix(console): discard stale log detail responses An older detail request can finish after a newer selection or close. Discard it so the dialog never shows the previously clicked record. * fix(console): retain legacy group owners Groups predating member roles store owners in adminIds. Load those as owners so adding a member never drops all old owners. * fix(console): lock member edits during saves Edits made during a save are overwritten by its result. Disable member controls and guard handlers until the write settles. * fix(console): show invite loading failures Failed invite loads must not look like no pending invites. Show the error and discard responses from a previously shown group. * fix(console): retain legacy discord log targets A stored log target without platform means Discord. Keep that meaning in the editor instead of clearing the target. * fix(console): retain undisplayed forge sources An editor that only displays some providers must preserve the others. Keep stored custom sources when saving unrelated group settings. * fix(console): lock the group editor while saving Prevent edits while a group save is in flight and make super-only installation settings readonly instead of silently ignoring edits. * fix(console): lock route edits and show failures Lock the route editor while saving and show fragment load failures with a retry action rather than presenting a false empty list. * fix(config): expose the legal contact override Declare the public runtime key so the documented environment override reaches legal pages in deployed Nuxt builds. * build(deploy): migrate d1 before deployment Apply pending D1 migrations before deploying code that requires them. Stop deployment when migration fails instead of running against an incomplete schema. * docs(worker): align guides with runtime behavior Synchronize both language guides, API references and examples with the implemented authorization, storage and delivery behavior. Document migration requirements and accepted operational limits. * fix(storage): repair orphaned KV routes Repair orphaned KV routes before serving KV-only configuration. * fix(security): fully escape forge HTML --------- Co-authored-by: RhenCloud <i@rhen.cloud>
This commit is contained in:
1 parent
dbf162bf74
commit
978c53eeb3
163 files changed
+18459
-3607
No files matched your search
@@ -1,25 +1,89 @@
|
||||
import { createAppJwt } from "./oauth";
|
||||
import { sharedFlight, type Flight } from "../lib/shared-flight";
|
||||
|
||||
export const GITHUB_API_VERSION = "2022-11-28";
|
||||
/**
|
||||
* Longest an optional enrichment call (installation token, jobs, check runs)
|
||||
* may take, its body included: a hanging one must not hold up the delivery.
|
||||
*/
|
||||
export const ENRICH_TIMEOUT_MS = 10_000;
|
||||
export const GITHUB_USER_AGENT = "WebHooker (https://github.com/Elysola/WebHooker)";
|
||||
|
||||
/**
|
||||
* A payload-supplied API URL (`jobs_url`, `check_runs_url`) parsed only when
|
||||
* it points at the GitHub REST API's repository endpoints. Anything else — a
|
||||
* forged payload aiming a token at its own server — yields null.
|
||||
*/
|
||||
export function githubApiUrl(value: string | undefined): URL | null {
|
||||
if (!value) return null;
|
||||
try {
|
||||
const url = new URL(value);
|
||||
if (url.protocol !== "https:" || url.host !== "api.github.com") return null;
|
||||
if (!url.pathname.startsWith("/repos/") || url.username || url.password) return null;
|
||||
return url;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Mint a short-lived installation access token for a GitHub App installation.
|
||||
* Returns undefined when the App credentials are missing or the exchange
|
||||
* fails. Best-effort: callers must not depend on it.
|
||||
*/
|
||||
/**
|
||||
* Installation tokens this isolate holds (until shortly before they expire),
|
||||
* and exchanges in flight: one per installation, not one per event — each is
|
||||
* a subrequest plus a JWT signature.
|
||||
*/
|
||||
const installationTokens = new Map<string, { token: string; until: number }>();
|
||||
const exchanging = new Map<string, Flight<string | undefined>>();
|
||||
/**
|
||||
* Installations whose exchange just failed, until when: enrichment is
|
||||
* optional, so the events right after go without it instead of each signing
|
||||
* a JWT and asking GitHub again.
|
||||
*/
|
||||
const failedExchanges = new Map<string, number>();
|
||||
const FAILED_EXCHANGE_MS = 60_000;
|
||||
|
||||
export async function getInstallationToken(
|
||||
appId: string | undefined,
|
||||
privateKey: string | undefined,
|
||||
installationId: number | undefined,
|
||||
): Promise<string | undefined> {
|
||||
if (!appId || !privateKey || !installationId) return undefined;
|
||||
const key = `${appId}:${installationId}`;
|
||||
const held = installationTokens.get(key);
|
||||
if (held && Date.now() < held.until) return held.token;
|
||||
if ((failedExchanges.get(key) ?? 0) > Date.now()) return undefined;
|
||||
return sharedFlight(
|
||||
exchanging,
|
||||
key,
|
||||
async () => {
|
||||
const token = await exchangeInstallationToken(appId, privateKey, installationId);
|
||||
if (token) failedExchanges.delete(key);
|
||||
else {
|
||||
if (failedExchanges.size >= 1000) failedExchanges.clear();
|
||||
failedExchanges.set(key, Date.now() + FAILED_EXCHANGE_MS);
|
||||
}
|
||||
return token;
|
||||
},
|
||||
ENRICH_TIMEOUT_MS + 5_000,
|
||||
);
|
||||
}
|
||||
|
||||
async function exchangeInstallationToken(
|
||||
appId: string,
|
||||
privateKey: string,
|
||||
installationId: number,
|
||||
): Promise<string | undefined> {
|
||||
try {
|
||||
const jwt = await createAppJwt(appId, privateKey);
|
||||
const res = await fetch(
|
||||
`https://api.github.com/app/installations/${installationId}/access_tokens`,
|
||||
{
|
||||
method: "POST",
|
||||
signal: AbortSignal.timeout(ENRICH_TIMEOUT_MS),
|
||||
headers: {
|
||||
Authorization: `Bearer ${jwt}`,
|
||||
Accept: "application/vnd.github+json",
|
||||
@@ -29,8 +93,14 @@ export async function getInstallationToken(
|
||||
},
|
||||
);
|
||||
if (!res.ok) return undefined;
|
||||
const { token } = (await res.json()) as { token?: string };
|
||||
return token ?? undefined;
|
||||
const { token, expires_at } = (await res.json()) as { token?: string; expires_at?: string };
|
||||
if (!token) return undefined;
|
||||
const expires = Date.parse(expires_at ?? "");
|
||||
// Tokens last an hour; given back 5 minutes early (a request in flight
|
||||
// never carries one that just expired).
|
||||
const until = (Number.isFinite(expires) ? expires : Date.now() + 3600_000) - 5 * 60_000;
|
||||
if (until > Date.now()) installationTokens.set(`${appId}:${installationId}`, { token, until });
|
||||
return token;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
import { getInstallationToken, GITHUB_API_VERSION, GITHUB_USER_AGENT } from "./app-token";
|
||||
import {
|
||||
getInstallationToken,
|
||||
githubApiUrl,
|
||||
ENRICH_TIMEOUT_MS,
|
||||
GITHUB_API_VERSION,
|
||||
GITHUB_USER_AGENT,
|
||||
} from "./app-token";
|
||||
|
||||
const CLOUDFLARE_SLUG = "cloudflare-workers-and-pages";
|
||||
|
||||
@@ -18,12 +24,16 @@ export async function getCheckSuiteBuildLogUrl(
|
||||
privateKey: string | undefined,
|
||||
installationId: number | undefined,
|
||||
): Promise<string | undefined> {
|
||||
if (!checkRunsUrl || !appId || !privateKey || !installationId) return undefined;
|
||||
const url = githubApiUrl(checkRunsUrl);
|
||||
if (!url || !appId || !privateKey || !installationId) return undefined;
|
||||
try {
|
||||
const token = await getInstallationToken(appId, privateKey, installationId);
|
||||
if (!token) return undefined;
|
||||
|
||||
const runsRes = await fetch(checkRunsUrl, {
|
||||
// No redirects: a token must never follow a response to another host.
|
||||
const runsRes = await fetch(url.toString(), {
|
||||
redirect: "manual",
|
||||
signal: AbortSignal.timeout(ENRICH_TIMEOUT_MS),
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
Accept: "application/vnd.github+json",
|
||||
|
||||
+169
-31
@@ -1,8 +1,9 @@
|
||||
import { Octokit } from "octokit";
|
||||
import { saveToken, getToken } from "./store";
|
||||
import { saveToken, getToken, type TokenSource } from "./store";
|
||||
|
||||
export function getOAuthURL(clientId: string, state: string): string {
|
||||
return `https://github.com/login/oauth/authorize?client_id=${clientId}&scope=repo&state=${state}`;
|
||||
// `read:org` lets the install flow check organization admin rights.
|
||||
return `https://github.com/login/oauth/authorize?client_id=${clientId}&scope=${encodeURIComponent("repo read:org")}&state=${state}`;
|
||||
}
|
||||
|
||||
function b64url(buf: ArrayBuffer | string): string {
|
||||
@@ -40,31 +41,82 @@ export async function createAppJwt(appId: string, privateKey: string): Promise<s
|
||||
return `${data}.${b64url(sig)}`;
|
||||
}
|
||||
|
||||
export interface InstallationAccount {
|
||||
login: string;
|
||||
id: number;
|
||||
/** "User" or "Organization". */
|
||||
type: string;
|
||||
}
|
||||
|
||||
const API_HEADERS = {
|
||||
Accept: "application/vnd.github+json",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
"User-Agent": "WebHooker",
|
||||
};
|
||||
|
||||
/**
|
||||
* Look up the account (org/user login) that owns a GitHub App installation,
|
||||
* using an App JWT. Returns null when the App credentials are missing or the
|
||||
* lookup fails (the caller falls back to an anonymous installation group).
|
||||
* Look up the account (org/user) that owns a GitHub App installation, using
|
||||
* an App JWT. Returns null when the App credentials are missing or the lookup
|
||||
* fails.
|
||||
*/
|
||||
export async function getInstallationDetails(
|
||||
appId: string,
|
||||
privateKey: string,
|
||||
installationId: number,
|
||||
): Promise<InstallationAccount | null> {
|
||||
if (!appId || !privateKey) return null;
|
||||
try {
|
||||
const jwt = await createAppJwt(appId, privateKey);
|
||||
const res = await fetch(`https://api.github.com/app/installations/${installationId}`, {
|
||||
headers: { ...API_HEADERS, Authorization: `Bearer ${jwt}` },
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const data = (await res.json()) as {
|
||||
account?: { login?: string; id?: number; type?: string };
|
||||
};
|
||||
if (!data.account?.login) return null;
|
||||
// An account without an id can still be named, but never matches a user.
|
||||
return { login: data.account.login, id: data.account.id ?? -1, type: data.account.type ?? "" };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** The login of the account owning an installation (null when unknown). */
|
||||
export async function getInstallationAccount(
|
||||
appId: string,
|
||||
privateKey: string,
|
||||
installationId: number,
|
||||
): Promise<string | null> {
|
||||
if (!appId || !privateKey) return null;
|
||||
return (await getInstallationDetails(appId, privateKey, installationId))?.login ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a GitHub user may administer an App installation. The Setup URL's
|
||||
* `installation_id` is attacker-controlled, so binding it to a group requires
|
||||
* proof from the user's own token: the user is the installation's account, or
|
||||
* an active admin of its organization. Merely having access to it (being
|
||||
* listed in `/user/installations` as a collaborator) is not enough.
|
||||
*/
|
||||
export async function userManagesInstallation(
|
||||
token: string,
|
||||
userId: string,
|
||||
_installationId: number,
|
||||
account: InstallationAccount | null,
|
||||
): Promise<boolean> {
|
||||
const headers = { ...API_HEADERS, Authorization: `Bearer ${token}` };
|
||||
try {
|
||||
const jwt = await createAppJwt(appId, privateKey);
|
||||
const res = await fetch(`https://api.github.com/app/installations/${installationId}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${jwt}`,
|
||||
Accept: "application/vnd.github+json",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
},
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const data = (await res.json()) as { account?: { login?: string } };
|
||||
return data.account?.login ?? null;
|
||||
if (!account) return false;
|
||||
if (account.type === "User") return String(account.id) === userId;
|
||||
const res = await fetch(
|
||||
`https://api.github.com/user/memberships/orgs/${encodeURIComponent(account.login)}`,
|
||||
{ headers },
|
||||
);
|
||||
if (!res.ok) return false;
|
||||
const membership = (await res.json()) as { state?: string; role?: string };
|
||||
return membership.state === "active" && membership.role === "admin";
|
||||
} catch {
|
||||
return null;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -74,6 +126,7 @@ export async function handleOAuthCallback(
|
||||
code: string,
|
||||
_state: string,
|
||||
kv: KVNamespace,
|
||||
db?: D1Database,
|
||||
): Promise<{ userId: string; login: string } | null> {
|
||||
const res = await fetch("https://github.com/login/oauth/access_token", {
|
||||
method: "POST",
|
||||
@@ -102,15 +155,18 @@ export async function handleOAuthCallback(
|
||||
const octokit = new Octokit({ auth: data.access_token });
|
||||
const { data: user } = await octokit.rest.users.getAuthenticated();
|
||||
|
||||
await saveToken(kv, user.id.toString(), data.access_token, 3600 * 10);
|
||||
await saveToken(kv, user.id.toString(), data.access_token, 3600 * 10, db);
|
||||
|
||||
return { userId: user.id.toString(), login: user.login };
|
||||
}
|
||||
|
||||
export async function getUserOctokit(userId: string, kv: KVNamespace): Promise<Octokit | null> {
|
||||
export async function getUserOctokit(userId: string, kv: TokenSource): Promise<Octokit | null> {
|
||||
const token = await getToken(kv, userId);
|
||||
if (!token) return null;
|
||||
return new Octokit({ auth: token });
|
||||
// No automatic retries: the user's actions (a comment, a merge) are not
|
||||
// idempotent, and a retry after a lost response would act twice. A failure
|
||||
// is reported to the user instead, who can try again knowingly.
|
||||
return new Octokit({ auth: token, retry: { enabled: false }, throttle: { enabled: false } });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -125,8 +181,84 @@ function mapGitHubError(err: unknown): Error {
|
||||
return err instanceof Error ? err : new Error(String(err));
|
||||
}
|
||||
|
||||
async function requireOctokit(kv: KVNamespace, githubUserId: string): Promise<Octokit> {
|
||||
const octokit = await getUserOctokit(githubUserId, kv);
|
||||
/**
|
||||
* The immutable user id GitHub has for `login` right now, or null when it is
|
||||
* unknown (no such user, no answer). Looked up with the caller's own token.
|
||||
*/
|
||||
export async function resolveGitHubUserId(token: string, login: string): Promise<string | null> {
|
||||
try {
|
||||
const res = await fetch(`https://api.github.com/users/${encodeURIComponent(login)}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
Accept: "application/vnd.github+json",
|
||||
"User-Agent": "WebHooker",
|
||||
},
|
||||
redirect: "error",
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const data = (await res.json()) as { id?: unknown; login?: unknown };
|
||||
const sameLogin =
|
||||
typeof data.login === "string" && data.login.toLowerCase() === login.toLowerCase();
|
||||
return sameLogin && typeof data.id === "number" ? String(data.id) : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A failure before anything was sent to GitHub (the user's token could not be
|
||||
* read). Bot commands give such a command back for the platform to retry,
|
||||
* instead of consuming it with an error reply.
|
||||
*/
|
||||
export class BeforeGitHubError extends Error {}
|
||||
|
||||
/**
|
||||
* Another run finished this command meanwhile (this one outlived its claim):
|
||||
* nothing is written, and that run's "done" stays.
|
||||
*/
|
||||
export class CommandTakenError extends BeforeGitHubError {}
|
||||
|
||||
/**
|
||||
* GitHub refused the request (or none could be made for want of a token):
|
||||
* nothing was done, so running the command again is safe. Anything else (a
|
||||
* network failure, a 5xx) may have acted.
|
||||
*/
|
||||
export function githubRefused(err: unknown): boolean {
|
||||
const message = err instanceof Error ? err.message : "";
|
||||
if (["GITHUB_TOKEN_EXPIRED", "GITHUB_FORBIDDEN", "GITHUB_NOT_FOUND"].includes(message)) {
|
||||
return true;
|
||||
}
|
||||
const status = (err as { status?: unknown })?.status;
|
||||
return typeof status === "number" && status >= 400 && status < 500;
|
||||
}
|
||||
|
||||
/**
|
||||
* `beforeWrite` runs once only the write itself is left (a bot records the
|
||||
* command as done there, so a lost confirmation can never run the write
|
||||
* again); its failure aborts with nothing done.
|
||||
*/
|
||||
export interface WriteHooks {
|
||||
beforeWrite?: () => Promise<void>;
|
||||
}
|
||||
|
||||
async function beforeWrite(hooks: WriteHooks | undefined): Promise<void> {
|
||||
if (!hooks?.beforeWrite) return;
|
||||
try {
|
||||
await hooks.beforeWrite();
|
||||
} catch (err) {
|
||||
if (err instanceof BeforeGitHubError) throw err;
|
||||
throw new BeforeGitHubError(err instanceof Error ? err.message : String(err));
|
||||
}
|
||||
}
|
||||
|
||||
async function requireOctokit(kv: TokenSource, githubUserId: string): Promise<Octokit> {
|
||||
let octokit: Octokit | null;
|
||||
try {
|
||||
octokit = await getUserOctokit(githubUserId, kv);
|
||||
} catch (err) {
|
||||
throw new BeforeGitHubError(err instanceof Error ? err.message : String(err));
|
||||
}
|
||||
if (!octokit) throw new Error("GITHUB_TOKEN_EXPIRED");
|
||||
return octokit;
|
||||
}
|
||||
@@ -136,23 +268,25 @@ async function requireOctokit(kv: KVNamespace, githubUserId: string): Promise<Oc
|
||||
* so the comment shows up under their own identity instead of the bot.
|
||||
*/
|
||||
export async function commentAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
issueNumber: number,
|
||||
body: string,
|
||||
hooks?: WriteHooks,
|
||||
): Promise<{ htmlUrl: string; login: string }> {
|
||||
const octokit = await requireOctokit(kv, githubUserId);
|
||||
await beforeWrite(hooks);
|
||||
try {
|
||||
const { data: me } = await octokit.rest.users.getAuthenticated();
|
||||
// The author comes with the created comment: no separate `/user` request.
|
||||
const res = await octokit.rest.issues.createComment({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: issueNumber,
|
||||
body,
|
||||
});
|
||||
return { htmlUrl: res.data.html_url, login: me.login };
|
||||
return { htmlUrl: res.data.html_url, login: res.data.user?.login ?? "" };
|
||||
} catch (err) {
|
||||
throw mapGitHubError(err);
|
||||
}
|
||||
@@ -160,7 +294,7 @@ export async function commentAsUser(
|
||||
|
||||
/** Fetch a single issue comment's current body (used to prefill the edit modal). */
|
||||
export async function getCommentAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
@@ -177,7 +311,7 @@ export async function getCommentAsUser(
|
||||
|
||||
/** Edit an existing issue/PR comment. GitHub enforces permission (403 if not allowed). */
|
||||
export async function editCommentAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
@@ -200,7 +334,7 @@ export async function editCommentAsUser(
|
||||
|
||||
/** Delete an existing issue/PR comment. GitHub enforces permission (403 if not allowed). */
|
||||
export async function deleteCommentAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
@@ -216,14 +350,16 @@ export async function deleteCommentAsUser(
|
||||
|
||||
/** Merge a pull request as the linked GitHub user. GitHub enforces permission (403 if not allowed). */
|
||||
export async function mergePullRequestAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
pullNumber: number,
|
||||
method: "merge" | "squash" | "rebase" = "squash",
|
||||
hooks?: WriteHooks,
|
||||
): Promise<void> {
|
||||
const octokit = await requireOctokit(kv, githubUserId);
|
||||
await beforeWrite(hooks);
|
||||
try {
|
||||
await octokit.rest.pulls.merge({
|
||||
owner,
|
||||
@@ -238,13 +374,15 @@ export async function mergePullRequestAsUser(
|
||||
|
||||
/** Close a pull request as the linked GitHub user. GitHub enforces permission (403 if not allowed). */
|
||||
export async function closePullRequestAsUser(
|
||||
kv: KVNamespace,
|
||||
kv: TokenSource,
|
||||
githubUserId: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
pullNumber: number,
|
||||
hooks?: WriteHooks,
|
||||
): Promise<void> {
|
||||
const octokit = await requireOctokit(kv, githubUserId);
|
||||
await beforeWrite(hooks);
|
||||
try {
|
||||
await octokit.rest.pulls.update({
|
||||
owner,
|
||||
|
||||
+212
-4
@@ -1,7 +1,16 @@
|
||||
import { idempotencyStore, recordStatement, unrecordedGuard } from "../lib/idempotency";
|
||||
import { canUseD1, isMissingTable } from "../storage/d1";
|
||||
|
||||
interface StoredToken {
|
||||
userId: string;
|
||||
accessToken: string;
|
||||
expiresAt: number;
|
||||
/**
|
||||
* Valid only while D1 records it as the user's live token (see
|
||||
* `markLive`). Unset on tokens saved before that record existed, or
|
||||
* without the D1 table: those follow the revocation records alone.
|
||||
*/
|
||||
live?: boolean;
|
||||
}
|
||||
|
||||
async function hashToken(token: string): Promise<string> {
|
||||
@@ -12,11 +21,93 @@ async function hashToken(token: string): Promise<string> {
|
||||
.join("");
|
||||
}
|
||||
|
||||
/**
|
||||
* Tokens are revoked (on removal and rotation) in the idempotency store (D1)
|
||||
* as well as deleted from KV, whose deletes take a while to reach every
|
||||
* location: a revoked token must stop authenticating at once.
|
||||
*/
|
||||
function revokedTokenKey(tokenHash: string): string {
|
||||
return `token-revoked:${tokenHash}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* The user's live token, in D1. KV can hand a node a stale copy of the user's
|
||||
* token, so a removal or rotation working from that copy would miss the
|
||||
* newer token: dropping every live record of the user covers it whatever
|
||||
* copy was read.
|
||||
*/
|
||||
function liveTokenKey(userId: string, tokenHash: string): string {
|
||||
return `${liveTokenPrefix(userId)}${tokenHash}`;
|
||||
}
|
||||
|
||||
function liveTokenPrefix(userId: string): string {
|
||||
return `token-live:${userId}:`;
|
||||
}
|
||||
|
||||
/** Statement dropping the user's live records (all, or all but `keep`). */
|
||||
function dropLive(db: D1Database, userId: string, keep = ""): D1PreparedStatement {
|
||||
// A range over the prefix (`;` follows `:`), never a LIKE pattern.
|
||||
return db
|
||||
.prepare("DELETE FROM dedup_keys WHERE key >= ? AND key < ? AND key <> ?")
|
||||
.bind(liveTokenPrefix(userId), `token-live:${userId};`, keep);
|
||||
}
|
||||
|
||||
/**
|
||||
* Record `tokenHash` as the user's only live token, in one D1 batch. Returns
|
||||
* false where the D1 table does not exist yet (the token then follows the
|
||||
* revocation records alone).
|
||||
*/
|
||||
async function markLive(
|
||||
db: D1Database,
|
||||
userId: string,
|
||||
tokenHash: string,
|
||||
ttlSeconds: number,
|
||||
): Promise<boolean> {
|
||||
const now = Date.now();
|
||||
const key = liveTokenKey(userId, tokenHash);
|
||||
try {
|
||||
await db.batch([
|
||||
dropLive(db, userId, key),
|
||||
db
|
||||
.prepare(
|
||||
`INSERT INTO dedup_keys (key, claimed_at, expires_at) VALUES (?, ?, ?)
|
||||
ON CONFLICT(key) DO UPDATE SET
|
||||
claimed_at = excluded.claimed_at, expires_at = excluded.expires_at`,
|
||||
)
|
||||
.bind(key, now, now + ttlSeconds * 1000),
|
||||
]);
|
||||
return true;
|
||||
} catch (err) {
|
||||
if (isMissingTable(err)) return false;
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether a stored token may still be used (expiry aside). */
|
||||
async function usable(
|
||||
db: D1Database | undefined,
|
||||
kv: KVNamespace,
|
||||
t: StoredToken,
|
||||
tokenHash: string,
|
||||
): Promise<boolean> {
|
||||
if (!db) return true;
|
||||
const claims = idempotencyStore(db, kv);
|
||||
if (await claims.has(revokedTokenKey(tokenHash))) return false;
|
||||
if (t.live && !(await claims.has(liveTokenKey(t.userId, tokenHash)))) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
async function revokeToken(db: D1Database, kv: KVNamespace, t: StoredToken): Promise<void> {
|
||||
const ttl = Math.max(60, Math.ceil((t.expiresAt - Date.now()) / 1000));
|
||||
await idempotencyStore(db, kv).claim(revokedTokenKey(await hashToken(t.accessToken)), ttl);
|
||||
}
|
||||
|
||||
export async function saveToken(
|
||||
kv: KVNamespace,
|
||||
userId: string,
|
||||
accessToken: string,
|
||||
expiresInSeconds: number,
|
||||
db?: D1Database,
|
||||
): Promise<void> {
|
||||
const token: StoredToken = {
|
||||
userId,
|
||||
@@ -24,13 +115,36 @@ export async function saveToken(
|
||||
expiresAt: Date.now() + expiresInSeconds * 1000,
|
||||
};
|
||||
const ttl = Math.max(Math.floor(expiresInSeconds * 0.9), 60);
|
||||
const tokenHash = await hashToken(accessToken);
|
||||
// Rotation: the previous token's reverse index must stop resolving to this
|
||||
// user, or the old token would keep acting with the new one.
|
||||
// A failed read aborts the rotation: the old token must be revoked, and
|
||||
// "none" cannot be assumed.
|
||||
const previous = await kv.get<StoredToken>(`token:${userId}`, "json");
|
||||
const rotated = previous?.accessToken && previous.accessToken !== accessToken;
|
||||
// Revoked before it is overwritten: should this fail, the old token is
|
||||
// still on record and a retry revokes it then.
|
||||
if (rotated && db) await revokeToken(db, kv, previous);
|
||||
// The only live token from here on — whichever older one some node's KV
|
||||
// still shows.
|
||||
if (db && canUseD1(db) && (await markLive(db, userId, tokenHash, expiresInSeconds))) {
|
||||
token.live = true;
|
||||
}
|
||||
await kv.put(`token:${userId}`, JSON.stringify(token), { expirationTtl: ttl });
|
||||
|
||||
const tokenHash = await hashToken(accessToken);
|
||||
await kv.put(`token-reverse:${tokenHash}`, userId, { expirationTtl: ttl });
|
||||
if (rotated) await kv.delete(`token-reverse:${await hashToken(previous.accessToken)}`);
|
||||
}
|
||||
|
||||
export async function getToken(kv: KVNamespace, userId: string): Promise<string | null> {
|
||||
/** Where tokens are read from: KV alone, or KV with D1 (revocations honored). */
|
||||
export type TokenSource = KVNamespace | { KV: KVNamespace; DB?: D1Database };
|
||||
|
||||
function sourceOf(source: TokenSource): { kv: KVNamespace; db?: D1Database } {
|
||||
return "KV" in source ? { kv: source.KV, db: source.DB } : { kv: source };
|
||||
}
|
||||
|
||||
export async function getToken(source: TokenSource, userId: string): Promise<string | null> {
|
||||
const { kv, db } = sourceOf(source);
|
||||
const raw = await kv.get(`token:${userId}`, "json");
|
||||
if (!raw) return null;
|
||||
const t = raw as StoredToken;
|
||||
@@ -38,25 +152,119 @@ export async function getToken(kv: KVNamespace, userId: string): Promise<string
|
||||
await kv.delete(`token:${userId}`);
|
||||
return null;
|
||||
}
|
||||
// Revoked or no longer live, even where KV still holds it (writes and
|
||||
// deletes take a while to spread).
|
||||
if (!(await usable(db, kv, t, await hashToken(t.accessToken)))) return null;
|
||||
return t.accessToken;
|
||||
}
|
||||
|
||||
export async function removeToken(kv: KVNamespace, userId: string): Promise<void> {
|
||||
export async function removeToken(kv: KVNamespace, userId: string, db?: D1Database): Promise<void> {
|
||||
// First, and whatever KV shows here: a newer token than the copy read below
|
||||
// stops too.
|
||||
if (db && canUseD1(db)) {
|
||||
try {
|
||||
await dropLive(db, userId).run();
|
||||
} catch (err) {
|
||||
if (!isMissingTable(err)) throw err;
|
||||
}
|
||||
}
|
||||
const raw = await kv.get(`token:${userId}`, "json");
|
||||
if (raw) {
|
||||
const t = raw as StoredToken;
|
||||
if (db) await revokeToken(db, kv, t);
|
||||
const tokenHash = await hashToken(t.accessToken);
|
||||
await kv.delete(`token-reverse:${tokenHash}`);
|
||||
}
|
||||
await kv.delete(`token:${userId}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* The GitHub user whose *current* stored token is `accessToken`. The reverse
|
||||
* index only narrows the lookup; the stored token must match too, so a token
|
||||
* that was rotated, revoked or expired never authenticates (even while a stale
|
||||
* reverse key is still around).
|
||||
*/
|
||||
export async function findUserIdByToken(
|
||||
kv: KVNamespace,
|
||||
accessToken: string,
|
||||
db?: D1Database,
|
||||
): Promise<string | null> {
|
||||
const tokenHash = await hashToken(accessToken);
|
||||
return await kv.get(`token-reverse:${tokenHash}`, "text");
|
||||
if (db && (await idempotencyStore(db, kv).has(revokedTokenKey(tokenHash)))) return null;
|
||||
const userId = await kv.get(`token-reverse:${tokenHash}`, "text");
|
||||
if (!userId) return null;
|
||||
const stored = await kv.get<StoredToken>(`token:${userId}`, "json");
|
||||
if (!stored || Date.now() >= stored.expiresAt) return null;
|
||||
if ((await hashToken(stored.accessToken)) !== tokenHash) return null;
|
||||
if (!(await usable(db, kv, stored, tokenHash))) return null;
|
||||
return userId;
|
||||
}
|
||||
|
||||
export type LinkPlatform = "discord" | "telegram" | "feishu";
|
||||
|
||||
const LINK_TABLES: Record<LinkPlatform, { table: string; column: string }> = {
|
||||
discord: { table: "discord_links", column: "discord_user_id" },
|
||||
telegram: { table: "telegram_links", column: "telegram_user_id" },
|
||||
feishu: { table: "feishu_links", column: "feishu_user_id" },
|
||||
};
|
||||
|
||||
/** Statement linking a chat account to a GitHub account (for a batch). */
|
||||
export function linkStatement(
|
||||
db: D1Database,
|
||||
platform: LinkPlatform,
|
||||
chatUserId: string,
|
||||
githubUserId: string,
|
||||
): D1PreparedStatement {
|
||||
const { table, column } = LINK_TABLES[platform];
|
||||
return db
|
||||
.prepare(`INSERT OR REPLACE INTO ${table} (${column}, github_user_id) VALUES (?, ?)`)
|
||||
.bind(chatUserId, githubUserId);
|
||||
}
|
||||
|
||||
/** Statement removing a chat account's link (for a batch). */
|
||||
export function unlinkStatement(
|
||||
db: D1Database,
|
||||
platform: LinkPlatform,
|
||||
chatUserId: string,
|
||||
): D1PreparedStatement {
|
||||
const { table, column } = LINK_TABLES[platform];
|
||||
return db.prepare(`DELETE FROM ${table} WHERE ${column} = ?`).bind(chatUserId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Unlink as a bot command: the command's "done" record (`done`) commits in
|
||||
* the same D1 transaction, so an unlink that went through — even one whose
|
||||
* answer was lost — is never run again by a redelivery (which by then could
|
||||
* remove a link made since).
|
||||
*/
|
||||
export async function unlinkAsCommand(
|
||||
env: { DB: D1Database; KV: KVNamespace },
|
||||
platform: LinkPlatform,
|
||||
chatUserId: string,
|
||||
done?: { key: string; ttl: number },
|
||||
): Promise<void> {
|
||||
if (!done || !canUseD1(env.DB)) {
|
||||
await unlinkStatement(env.DB, platform, chatUserId).run();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await env.DB.batch([
|
||||
// Done already (another run took over this command after ours outlived
|
||||
// its claim): nothing is removed — the link may be a new one by now.
|
||||
unrecordedGuard(env.DB, done.key),
|
||||
unlinkStatement(env.DB, platform, chatUserId),
|
||||
recordStatement(env.DB, done.key, done.ttl),
|
||||
]);
|
||||
} catch (err) {
|
||||
// Recorded — committed here after all (its answer lost), or by another
|
||||
// run before: done either way.
|
||||
if (
|
||||
!(await idempotencyStore(env.DB, env.KV)
|
||||
.has(done.key)
|
||||
.catch(() => false))
|
||||
)
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
import { getInstallationToken, GITHUB_API_VERSION, GITHUB_USER_AGENT } from "./app-token";
|
||||
import {
|
||||
getInstallationToken,
|
||||
githubApiUrl,
|
||||
ENRICH_TIMEOUT_MS,
|
||||
GITHUB_API_VERSION,
|
||||
GITHUB_USER_AGENT,
|
||||
} from "./app-token";
|
||||
|
||||
export interface WorkflowJob {
|
||||
name?: string;
|
||||
@@ -24,10 +30,10 @@ export async function getWorkflowRunJobs(
|
||||
privateKey: string | undefined,
|
||||
installationId: number | undefined,
|
||||
): Promise<WorkflowJob[] | undefined> {
|
||||
if (!jobsUrl) return undefined;
|
||||
const url = githubApiUrl(jobsUrl);
|
||||
if (!url) return undefined;
|
||||
try {
|
||||
const token = await getInstallationToken(appId, privateKey, installationId);
|
||||
const url = new URL(jobsUrl);
|
||||
url.searchParams.set("per_page", "100");
|
||||
const headers: Record<string, string> = {
|
||||
Accept: "application/vnd.github+json",
|
||||
@@ -35,7 +41,12 @@ export async function getWorkflowRunJobs(
|
||||
"User-Agent": GITHUB_USER_AGENT,
|
||||
};
|
||||
if (token) headers.Authorization = `Bearer ${token}`;
|
||||
const res = await fetch(url.toString(), { headers });
|
||||
// No redirects: a token must never follow a response to another host.
|
||||
const res = await fetch(url.toString(), {
|
||||
headers,
|
||||
redirect: "manual",
|
||||
signal: AbortSignal.timeout(ENRICH_TIMEOUT_MS),
|
||||
});
|
||||
if (!res.ok) return undefined;
|
||||
const data = (await res.json()) as { jobs?: WorkflowJob[] };
|
||||
return data.jobs?.length ? data.jobs : undefined;
|
||||
|
||||
Reference in new issue
Block a user