feat(km-wiki-ingest): 新增無人值守 drainer runtime（cursor + cron + Gitea webhook）
Phase b：把 ingest 從手動直驅升級成真無人值守。drainer/ 是 workflow km_wiki_ingest_drain 的可部署執行體（Cloudflare Worker），復用已 live 的 arcrun-code/kbdb/graph、不改 cypher-executor。 - 兩觸發共用 processCard 核心：(A) cron drain（Gitea tree 列卡→游標後取一小批→ 逐卡處理→進游標→到底回捲；撐 5,083 檔）；(B) Gitea push webhook→只處理 delta。 - 游標存 KBDB 一個 ingest_cursor entry（走 API、零建表）。冪等：未改卡 entry (page_name+content_hash)/triplet(uri+hash) 皆 skip，只有新/改卡真寫。 - src/usercode.ts 由 workflow.yaml parse_card.code 抽出（單一真相源 card-to-envelope.mjs）。 - 維持非-auto：wrangler.toml 的 cron trigger 預設註解（不自走），drain 走 POST /drain。 scheduled handler 已在 code，leo 過閘後取消註解即開自走。README 記部署/secret/webhook 設定。 已部署 leo21c：arcrun-km-wiki-drainer.leo21c.workers.dev。以 wrangler dev（本地 miniflare + .dev.vars token，非 secret 寫入）對 live 服務驗證：cron drain 冪等全 skip、游標前進+回捲、 webhook delta 只寫變更卡、三模式命中新卡。GITEA_TOKEN secret 需 leo 於互動模式設（auto 擋 secret 寫入）。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HJiLCRUU2o3aSpPEzVCt2o
This commit is contained in:
kbdb-cc
@@ -0,0 +1,3 @@
|
||||
.dev.vars
|
||||
node_modules/
|
||||
.wrangler/
|
||||
@@ -0,0 +1,70 @@
|
||||
# arcrun-km-wiki-drainer —— km_wiki_ingest_drain 的無人值守 runtime
|
||||
|
||||
`workflow.yaml` 是宣告式 spec；**本 Worker 是它可真部署、有游標/觸發的執行體**。
|
||||
復用已 live 的零件與服務，**不改 cypher-executor**：
|
||||
|
||||
```
|
||||
Gitea 抓卡 md → arcrun-code 零件（sandbox inline JS 解析）→ /entries 冪等 upsert（embed:true）
|
||||
→ graph /triplets/ingest（server 側 per-source 冪等）
|
||||
```
|
||||
|
||||
- Live：`https://arcrun-km-wiki-drainer.leo21c.workers.dev`
|
||||
- 解析 JS＝`src/usercode.ts`（由 `workflow.yaml` 的 `parse_card.code` 抽出，單一真相源 `lib/card-to-envelope.mjs`）。
|
||||
|
||||
## 兩觸發模式（共用同一 `processCard` 核心）
|
||||
|
||||
| 模式 | 觸發 | 行為 |
|
||||
|---|---|---|
|
||||
| **Phase 0 cron drain** | cron tick（見「開自走」）| `drainBatch(BATCH_SIZE)`：Gitea tree 列卡 → 游標之後取一小批 → 逐卡 `processCard` → 進游標（到底**回捲**重掃）。壓 CF subrequest 下、撐得住 5,083 檔（每 tick 小批）。 |
|
||||
| **穩態 webhook** | Gitea push webhook → `POST /webhook` | 只處理本次 commit 動到的 `system-dev/wiki/cards/**.md`（delta），不重掃全庫。**Gitea→Cloudflare，非 GitHub Actions（不觸 D20）**。 |
|
||||
|
||||
**冪等**：未改卡的 entry（`page_name`+`content_hash`）與 triplet（`source.uri`+`content_hash`）皆 skip；只有新/改卡真寫。游標存 KBDB 一個 `ingest_cursor` entry（走 API、零建表）。
|
||||
|
||||
## HTTP 路由
|
||||
|
||||
| 方法 | 路徑 | 用途 |
|
||||
|---|---|---|
|
||||
| GET | `/` | 健康 |
|
||||
| GET | `/cursor` | 看游標狀態 |
|
||||
| POST | `/drain?batch=N` | 手動觸發一批 drain（= 一個 cron tick；驗收/補跑用） |
|
||||
| POST | `/webhook` | Gitea push webhook 入口（delta） |
|
||||
| POST | `/cursor/reset` | 游標歸零（重新全 drain） |
|
||||
|
||||
## ⚠️ 非-auto（預設）
|
||||
|
||||
`wrangler.toml` 的 `[triggers] crons` **預設註解掉**——不自走排程、不無人值守亂寫。
|
||||
drain 以 `POST /drain` 手動觸發（與 cron `scheduled` handler 同核心）。
|
||||
要開自走 Phase 0 backfill：leo 過閘後取消 `wrangler.toml` 那兩行註解再 deploy。
|
||||
|
||||
## 部署（wrangler 直推 leo21c，禁 acr update）
|
||||
|
||||
```bash
|
||||
cd registry/examples/km-wiki-ingest/drainer
|
||||
npm install
|
||||
npx wrangler deploy
|
||||
# ★ 必須：設 Gitea 讀取權（Leo/notes 為 private repo）——此步需 leo 在互動模式跑（secret 寫入）
|
||||
printf '%s' "$GITEA_TOKEN" | npx wrangler secret put GITEA_TOKEN
|
||||
# 選填：webhook 簽章驗證
|
||||
# printf '%s' "$WEBHOOK_SECRET" | npx wrangler secret put GITEA_WEBHOOK_SECRET
|
||||
```
|
||||
|
||||
## Gitea webhook 設定（穩態 delta）
|
||||
|
||||
`Leo/notes` → Settings → Webhooks → Add Webhook → Gitea：
|
||||
- Target URL：`https://arcrun-km-wiki-drainer.leo21c.workers.dev/webhook`
|
||||
- HTTP Method：POST，Content-Type：`application/json`
|
||||
- Trigger：Push events（可只限 `main`）
|
||||
- Secret：若設，同步 `GITEA_WEBHOOK_SECRET` secret（本 Worker 驗 `X-Gitea-Signature` HMAC-SHA256）
|
||||
- Gitea 的 ping event → Worker 回 `{ok:true,pong:true}`（測連通）
|
||||
|
||||
## config（wrangler.toml [vars]）
|
||||
|
||||
`REPO=Leo/notes REF=main OWNER=leo CARDS_ROOT=system-dev/wiki/cards BATCH_SIZE=2`
|
||||
`CODE_WORKER_URL / KBDB_URL / GRAPH_URL / GRAPH_KEY`（皆 leo21c live 服務）。
|
||||
|
||||
## 規模備註（5,083 檔）
|
||||
|
||||
- 列卡走 Gitea `git/trees?recursive=true`（一次全樹）；若 `truncated`，回應標記 `truncated:true`，
|
||||
屆時換 tree 分頁列法（現況 notes 卡數小、不截斷）。
|
||||
- 每 tick 只 `BATCH_SIZE` 張，游標續傳；到底回捲 → 全庫變更靠冪等於下一輪捕捉；
|
||||
即時變更走 webhook。
|
||||
+1528
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"name": "arcrun-km-wiki-drainer",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"description": "km_wiki_ingest_drain 的無人值守 runtime（cron drain + Gitea webhook delta）",
|
||||
"scripts": { "deploy": "wrangler deploy" },
|
||||
"devDependencies": {
|
||||
"@cloudflare/workers-types": "^4.20250408.0",
|
||||
"typescript": "^5.7.0",
|
||||
"wrangler": "^4.0.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
/**
|
||||
* arcrun km_wiki_ingest_drainer —— 無人值守卡片 ingest 編排 Worker（Phase b）
|
||||
*
|
||||
* 這是 workflow `km_wiki_ingest_drain` 的「具體 runtime」：workflow.yaml 是宣告式 spec，
|
||||
* 本 Worker 是可真部署、有游標/觸發的執行體。復用已 live 的零件與服務，不動 cypher-executor：
|
||||
* Gitea 抓卡 md → arcrun-code 零件（sandbox inline JS 解析）→ /entries 冪等 upsert（embed:true）
|
||||
* → graph /triplets/ingest（server 側 per-source 冪等）。
|
||||
*
|
||||
* 兩觸發模式共用同一 card-processing 核心（processCard）：
|
||||
* (A) Phase 0 cron drain：cron tick → drainBatch(BATCH_SIZE)：列卡（Gitea tree recursive）→
|
||||
* 游標（存 KBDB 一個 cursor entry）之後取一小批 → 逐卡 processCard → 進游標（到底回捲）。
|
||||
* 冪等：未改卡的 entry(content_hash)/triplet(uri+hash) 皆 skip，只有新/改卡真寫。
|
||||
* (B) 穩態 webhook：Gitea push webhook（非 GitHub Actions，不觸 D20）→ 只處理 delta（本次
|
||||
* commit 動到的 system-dev/wiki/cards/**.md）→ 逐卡 processCard。不重掃全庫。
|
||||
*
|
||||
* HTTP 路由（fetch）：
|
||||
* GET / 健康
|
||||
* GET /cursor 看游標狀態
|
||||
* POST /drain 手動觸發一批 drain（= 一個 cron tick；驗收/補跑用）
|
||||
* POST /webhook Gitea push webhook 入口（delta 模式）
|
||||
* POST /cursor/reset 游標歸零（重新全 drain 用）
|
||||
*/
|
||||
|
||||
import { CARD_TO_ENVELOPE_USERCODE } from './usercode';
|
||||
|
||||
export interface Env {
|
||||
REPO: string; // Leo/notes
|
||||
REF: string; // main
|
||||
OWNER: string; // leo（KBDB 租戶 owner_id）
|
||||
CARDS_ROOT: string; // system-dev/wiki/cards
|
||||
CODE_WORKER_URL: string; // https://arcrun-code.leo21c.workers.dev/
|
||||
KBDB_URL: string; // https://arcrun-kbdb.leo21c.workers.dev
|
||||
GRAPH_URL: string; // https://kbdb-graph-plugin.leo21c.workers.dev
|
||||
GRAPH_KEY: string; // leo
|
||||
BATCH_SIZE: string; // "2"
|
||||
GITEA_TOKEN: string; // secret：讀 repo
|
||||
GITEA_WEBHOOK_SECRET?: string; // secret（選填）：驗 webhook 簽章
|
||||
}
|
||||
|
||||
const CURSOR_TYPE = 'ingest_cursor';
|
||||
const cursorPageName = (env: Env) => `cursor:km_wiki_ingest_drain:${env.REPO}`;
|
||||
|
||||
// ── Gitea ────────────────────────────────────────────────────────────────────
|
||||
const giteaApi = (env: Env, path: string) => `https://git.uncle6.me/api/v1/repos/${env.REPO}${path}`;
|
||||
|
||||
/** 列出 CARDS_ROOT 下所有卡片（git tree recursive，排除 00-INDEX/.gitkeep），回 [{path, sha}] 已排序。 */
|
||||
async function listCards(env: Env): Promise<{ path: string; sha: string }[]> {
|
||||
const r = await fetch(giteaApi(env, `/git/trees/${encodeURIComponent(env.REF)}?recursive=true`), {
|
||||
headers: { Authorization: `token ${env.GITEA_TOKEN}` },
|
||||
});
|
||||
if (!r.ok) throw new Error(`gitea tree ${r.status}: ${(await r.text()).slice(0, 200)}`);
|
||||
const data = (await r.json()) as { tree?: { path: string; type: string; sha: string }[]; truncated?: boolean };
|
||||
const root = env.CARDS_ROOT.replace(/\/$/, '') + '/';
|
||||
const cards = (data.tree || [])
|
||||
.filter((e) => e.type === 'blob' && e.path.startsWith(root) && e.path.endsWith('.md'))
|
||||
.filter((e) => {
|
||||
const base = e.path.split('/').pop() || '';
|
||||
return base !== '.gitkeep' && !base.startsWith('00-INDEX');
|
||||
})
|
||||
.map((e) => ({ path: e.path, sha: e.sha }))
|
||||
.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
|
||||
// truncated：5,083 檔規模 Gitea tree 可能截斷 → 記錄以便換分頁列法（現況 notes 卡數小，不截斷）。
|
||||
(cards as unknown as { truncated?: boolean }).truncated = data.truncated === true;
|
||||
return cards;
|
||||
}
|
||||
|
||||
async function giteaRaw(env: Env, relPath: string): Promise<string> {
|
||||
const url = giteaApi(env, `/raw/${relPath.split('/').map(encodeURIComponent).join('/')}?ref=${encodeURIComponent(env.REF)}`);
|
||||
const r = await fetch(url, { headers: { Authorization: `token ${env.GITEA_TOKEN}` } });
|
||||
if (!r.ok) throw new Error(`gitea raw ${relPath} ${r.status}`);
|
||||
return r.text();
|
||||
}
|
||||
|
||||
// ── card-processing 核心 ──────────────────────────────────────────────────────
|
||||
async function parseCard(env: Env, md: string, relPath: string): Promise<any> {
|
||||
const r = await fetch(env.CODE_WORKER_URL, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
code: CARD_TO_ENVELOPE_USERCODE,
|
||||
input: { md, relPath, repo: env.REPO, opts: { budget: 40 } },
|
||||
limits: { timeout_ms: 3000, max_output_bytes: 4 * 1024 * 1024 },
|
||||
}),
|
||||
});
|
||||
const out = (await r.json()) as { success: boolean; data?: any; error?: string };
|
||||
if (!out.success) throw new Error(`code 零件解析失敗 ${relPath}: ${out.error}`);
|
||||
return out.data; // { entry, envelopes, meta, nodeCount, tripletCount }
|
||||
}
|
||||
|
||||
function metaHash(entry: any): string | undefined {
|
||||
try {
|
||||
return typeof entry.metadata_json === 'string' ? JSON.parse(entry.metadata_json).content_hash : undefined;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
async function upsertEntry(env: Env, entry: any): Promise<{ action: string; id?: string }> {
|
||||
const pageName = entry.page_name;
|
||||
const metadata_json = JSON.stringify(entry.metadata);
|
||||
const newHash = entry.metadata.content_hash;
|
||||
const lookup = await fetch(
|
||||
`${env.KBDB_URL}/entries?page_name=${encodeURIComponent(pageName)}&owner_id=${encodeURIComponent(env.OWNER)}`,
|
||||
);
|
||||
const existing = ((await lookup.json()) as { entries?: any[] }).entries?.[0];
|
||||
if (existing) {
|
||||
if (metaHash(existing) === newHash) return { action: 'skip', id: existing.id };
|
||||
const r = await fetch(`${env.KBDB_URL}/entries/${existing.id}`, {
|
||||
method: 'PATCH',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ content: entry.content, tags_json: JSON.stringify(entry.tags || []), metadata_json }),
|
||||
});
|
||||
return { action: 'patch', id: existing.id, ...(r.ok ? {} : { action: 'patch-failed' }) };
|
||||
}
|
||||
const r = await fetch(`${env.KBDB_URL}/entries`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
entry_type: entry.entry_type,
|
||||
content: entry.content,
|
||||
page_name: pageName,
|
||||
owner_id: env.OWNER,
|
||||
tags_json: JSON.stringify(entry.tags || []),
|
||||
source: entry.metadata.source,
|
||||
metadata_json,
|
||||
}),
|
||||
});
|
||||
const out = (await r.json()) as { entry?: { id?: string } };
|
||||
return { action: 'create', id: out.entry?.id };
|
||||
}
|
||||
|
||||
async function postEnvelopes(env: Env, envelopes: any[]): Promise<any[]> {
|
||||
const results = [];
|
||||
for (const envp of envelopes) {
|
||||
// envelopes 已由 code 節點剝除 _-鍵；此處再保險剝一次。
|
||||
const clean = Object.fromEntries(Object.entries(envp).filter(([k]) => !k.startsWith('_')));
|
||||
const r = await fetch(`${env.GRAPH_URL}/triplets/ingest?owner_id=${encodeURIComponent(env.OWNER)}`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json', 'X-Arcrun-API-Key': env.GRAPH_KEY },
|
||||
body: JSON.stringify(clean),
|
||||
});
|
||||
const text = await r.text();
|
||||
let body: any;
|
||||
try { body = JSON.parse(text); } catch { body = text; }
|
||||
results.push({ status: r.status, uri: (envp as any).source?.uri, triplets: (envp as any).triplets?.length, body });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
/** 一張卡的完整處理（冪等）：fetch → parse → upsert entry → post envelopes。 */
|
||||
async function processCard(env: Env, relPath: string): Promise<any> {
|
||||
const md = await giteaRaw(env, relPath);
|
||||
const plan = await parseCard(env, md, relPath);
|
||||
const entry = await upsertEntry(env, plan.entry);
|
||||
const envelopes = await postEnvelopes(env, plan.envelopes);
|
||||
const wrote = entry.action !== 'skip' || envelopes.some((e) => e.body && e.body.skipped === false);
|
||||
return {
|
||||
relPath,
|
||||
page_name: plan.entry.page_name,
|
||||
content_hash: (plan.entry.metadata.content_hash || '').slice(0, 12),
|
||||
entry: entry.action,
|
||||
envelopes: envelopes.map((e) => ({ status: e.status, triplets: e.triplets, skipped: e.body?.skipped, ingested: e.body?.ingested })),
|
||||
wrote,
|
||||
};
|
||||
}
|
||||
|
||||
// ── 游標（存 KBDB 一個 ingest_cursor entry；不 embed）─────────────────────────
|
||||
interface CursorState { last_path: string; cycle: number; processed_total: number; updated_at: number; }
|
||||
|
||||
async function getCursor(env: Env): Promise<{ id?: string; state: CursorState }> {
|
||||
const r = await fetch(
|
||||
`${env.KBDB_URL}/entries?page_name=${encodeURIComponent(cursorPageName(env))}&owner_id=${encodeURIComponent(env.OWNER)}`,
|
||||
);
|
||||
const e = ((await r.json()) as { entries?: any[] }).entries?.[0];
|
||||
if (e) {
|
||||
try { return { id: e.id, state: JSON.parse(e.content) }; } catch { /* fallthrough */ }
|
||||
return { id: e.id, state: { last_path: '', cycle: 0, processed_total: 0, updated_at: 0 } };
|
||||
}
|
||||
return { state: { last_path: '', cycle: 0, processed_total: 0, updated_at: 0 } };
|
||||
}
|
||||
|
||||
async function setCursor(env: Env, cur: { id?: string; state: CursorState }): Promise<void> {
|
||||
const content = JSON.stringify(cur.state);
|
||||
if (cur.id) {
|
||||
await fetch(`${env.KBDB_URL}/entries/${cur.id}`, {
|
||||
method: 'PATCH', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ content }),
|
||||
});
|
||||
} else {
|
||||
await fetch(`${env.KBDB_URL}/entries`, {
|
||||
method: 'POST', headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({ entry_type: CURSOR_TYPE, page_name: cursorPageName(env), owner_id: env.OWNER, content,
|
||||
metadata_json: JSON.stringify({ kind: 'ingest_cursor', embed: false }) }),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** cron / 手動觸發：drain 一小批。游標之後取 BATCH 張；到底回捲（continuous drain）。 */
|
||||
async function drainBatch(env: Env, batchSize: number): Promise<any> {
|
||||
const cards = await listCards(env);
|
||||
const truncated = (cards as unknown as { truncated?: boolean }).truncated;
|
||||
const cur = await getCursor(env);
|
||||
let idx = cards.findIndex((c) => c.path > cur.state.last_path);
|
||||
let wrapped = false;
|
||||
if (idx < 0) { idx = 0; wrapped = true; } // 到底 → 回捲重掃（靠冪等，未改卡 cheap skip）
|
||||
const batch = cards.slice(idx, idx + batchSize);
|
||||
const results = [];
|
||||
for (const c of batch) results.push(await processCard(env, c.path));
|
||||
const newLast = batch.length ? batch[batch.length - 1].path : cur.state.last_path;
|
||||
cur.state = {
|
||||
last_path: newLast,
|
||||
cycle: cur.state.cycle + (wrapped ? 1 : 0),
|
||||
processed_total: cur.state.processed_total + batch.length,
|
||||
updated_at: Math.floor(Date.now() / 1000),
|
||||
};
|
||||
await setCursor(env, cur);
|
||||
return {
|
||||
mode: 'cron-drain', total_cards: cards.length, truncated: !!truncated, wrapped,
|
||||
batch: batch.map((c) => c.path), results,
|
||||
wrote: results.filter((r) => r.wrote).length, skipped: results.filter((r) => !r.wrote).length,
|
||||
cursor: cur.state,
|
||||
};
|
||||
}
|
||||
|
||||
// ── webhook（Gitea push → delta）───────────────────────────────────────────────
|
||||
async function verifyGiteaSig(env: Env, raw: string, sig: string | null): Promise<boolean> {
|
||||
if (!env.GITEA_WEBHOOK_SECRET) return true; // 未設 secret → 不驗（leo 自有 repo）
|
||||
if (!sig) return false;
|
||||
const key = await crypto.subtle.importKey('raw', new TextEncoder().encode(env.GITEA_WEBHOOK_SECRET),
|
||||
{ name: 'HMAC', hash: 'SHA-256' }, false, ['sign']);
|
||||
const mac = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(raw));
|
||||
const hex = [...new Uint8Array(mac)].map((b) => b.toString(16).padStart(2, '0')).join('');
|
||||
return hex === sig;
|
||||
}
|
||||
|
||||
async function handleWebhook(env: Env, raw: string): Promise<any> {
|
||||
const payload = JSON.parse(raw) as { commits?: { added?: string[]; modified?: string[]; removed?: string[] }[] };
|
||||
const root = env.CARDS_ROOT.replace(/\/$/, '') + '/';
|
||||
const isCard = (p: string) => p.startsWith(root) && p.endsWith('.md') && !(p.split('/').pop() || '').startsWith('00-INDEX');
|
||||
const changed = new Set<string>();
|
||||
const removed = new Set<string>();
|
||||
for (const cm of payload.commits || []) {
|
||||
for (const p of [...(cm.added || []), ...(cm.modified || [])]) if (isCard(p)) changed.add(p);
|
||||
for (const p of cm.removed || []) if (isCard(p)) removed.add(p);
|
||||
}
|
||||
const results = [];
|
||||
for (const p of changed) results.push(await processCard(env, p));
|
||||
return {
|
||||
mode: 'webhook-delta', changed: [...changed], removed: [...removed],
|
||||
note: removed.size ? 'removed 卡的清理未實作（graph 有 deprecate 機制，另議）' : undefined,
|
||||
results, wrote: results.filter((r) => r.wrote).length,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Worker 入口 ────────────────────────────────────────────────────────────────
|
||||
export default {
|
||||
async scheduled(_event: ScheduledEvent, env: Env, ctx: ExecutionContext): Promise<void> {
|
||||
ctx.waitUntil(drainBatch(env, Number(env.BATCH_SIZE || '2')).then(() => {}).catch(() => {}));
|
||||
},
|
||||
|
||||
async fetch(req: Request, env: Env, _ctx: ExecutionContext): Promise<Response> {
|
||||
const url = new URL(req.url);
|
||||
const json = (o: unknown, s = 200) => new Response(JSON.stringify(o, null, 2), { status: s, headers: { 'content-type': 'application/json' } });
|
||||
try {
|
||||
if (req.method === 'GET' && url.pathname === '/') {
|
||||
return json({ ok: true, service: 'km_wiki_ingest_drainer', repo: env.REPO, cards_root: env.CARDS_ROOT, batch: env.BATCH_SIZE });
|
||||
}
|
||||
if (req.method === 'GET' && url.pathname === '/cursor') {
|
||||
return json(await getCursor(env));
|
||||
}
|
||||
if (req.method === 'POST' && url.pathname === '/cursor/reset') {
|
||||
const cur = await getCursor(env);
|
||||
cur.state = { last_path: '', cycle: 0, processed_total: 0, updated_at: Math.floor(Date.now() / 1000) };
|
||||
await setCursor(env, cur);
|
||||
return json({ ok: true, cursor: cur.state });
|
||||
}
|
||||
if (req.method === 'POST' && url.pathname === '/drain') {
|
||||
const n = Number(url.searchParams.get('batch') || env.BATCH_SIZE || '2');
|
||||
return json(await drainBatch(env, n));
|
||||
}
|
||||
if (req.method === 'POST' && url.pathname === '/webhook') {
|
||||
const raw = await req.text();
|
||||
const ok = await verifyGiteaSig(env, raw, req.headers.get('X-Gitea-Signature'));
|
||||
if (!ok) return json({ error: 'bad signature' }, 401);
|
||||
// Gitea ping event（測試）→ 回 pong
|
||||
if (req.headers.get('X-Gitea-Event') === 'ping') return json({ ok: true, pong: true });
|
||||
return json(await handleWebhook(env, raw));
|
||||
}
|
||||
return json({ error: 'not found' }, 404);
|
||||
} catch (e) {
|
||||
return json({ error: e instanceof Error ? e.message : String(e) }, 500);
|
||||
}
|
||||
},
|
||||
};
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ES2022",
|
||||
"moduleResolution": "Bundler",
|
||||
"lib": ["ES2022"],
|
||||
"types": ["@cloudflare/workers-types"],
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"noEmit": true,
|
||||
"resolveJsonModule": true
|
||||
},
|
||||
"include": ["src/**/*.ts"]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
name = "arcrun-km-wiki-drainer"
|
||||
main = "src/index.ts"
|
||||
compatibility_date = "2025-02-19"
|
||||
workers_dev = true
|
||||
|
||||
# ⚠️ 維持「非-auto」：預設不註冊 cron trigger（不自走排程、不無人值守亂寫）。
|
||||
# drain 以 POST /drain 手動觸發（與 scheduled handler 同核心）。
|
||||
# 要開自走 Phase 0 backfill：leo 過閘後，取消下面兩行註解再 deploy（scheduled handler 已在 code）。
|
||||
# [triggers]
|
||||
# crons = ["*/5 * * * *"]
|
||||
|
||||
[vars]
|
||||
REPO = "Leo/notes"
|
||||
REF = "main"
|
||||
OWNER = "leo"
|
||||
CARDS_ROOT = "system-dev/wiki/cards"
|
||||
CODE_WORKER_URL = "https://arcrun-code.leo21c.workers.dev/"
|
||||
KBDB_URL = "https://arcrun-kbdb.leo21c.workers.dev"
|
||||
GRAPH_URL = "https://kbdb-graph-plugin.leo21c.workers.dev"
|
||||
GRAPH_KEY = "leo"
|
||||
BATCH_SIZE = "2"
|
||||
# secret（wrangler secret put）：GITEA_TOKEN（必），GITEA_WEBHOOK_SECRET（選）
|
||||
Reference in New Issue
Block a user