fix(kbdb): D69 節流世代核對+標庫共用 D1 額度,補「挑哪一批」的統一 SelectionCriteria(Arcrun#85)
leo 逐行複核 fix/embed-backfill-d68 後點出的破口+二次裁決(票上全文見 Leo/Arcrun#85): 一、向量化優先序(今天寫的立刻/本週在跑的先跑/有查詢紀錄的庫優先/半年前慢慢跑)表達 不出來——策略要能從外面(工作流)指定,不能焊死在資料層。新增 embed.ts 的 `SelectionCriteria`(owner_id/source/library/since/until),backfillEmbeddings 與 reconcileEmbedGeneration 共用同一套形狀;「按庫」那一層現在有資料可用即可運作(見下)。 二、世代核對(reconcileEmbedGeneration)不打 AI 但逐筆寫 D1,47 萬筆候選 ≈ 4.7 倍 D1 100,000 rows/日免費額度,先前零保護。新增 actions/maintenance-quota.ts(單一 entries 列/日的共用計數器,精神同 execution-log.ts/embed.ts 既有慣例,不新增表)。 三、leo 二度裁決:「標庫」與「時間分層」其實是一件事,判定標準要從第一天同時容納兩者, 不能先做一半再回頭改。新增 actions/library-backfill.ts 的 backfillEntryLibraryTags—— 呼叫端(ingest/daemon/Arcrun#87)決定要貼哪個庫、用 page_names(Gitea 原稿卡名精準 點名,leo 定案的正解)或 source_prefix/page_name_prefix 過渡 fallback 篩選候選,base 只負責安全、節流地寫入。owner_id 刻意必填(leo 點出「補錯 owner 等於白做」——實查卡片 掛在 owner_id=bfezv28v,換成 'leo' 查卻是空的)。 D69:reconcile 與標庫 backfill 共用同一顆「今天還剩多少 D1 寫入額度」計數器(不共用的話 其中一個會把另一個的閘繞過去);新增 POST /entries/backfill-library + GET .../status, 擴充 POST /embed/backfill 與 /embed/reconcile 吃 library/since/until 參數。 測試:92 → 39 個新增/擴充案例覆蓋 since/until/library 篩選、reconcile 額度真的擋 (含「拿掉 cap 會變紅」反向驗證)、標庫 backfill 冪等/owner_id 必填/page_names 精準比對、 以及兩個操作共用同一顆額度計數器的跨模組驗證(雙向:先 reconcile 耗盡再標庫、反之亦然)。 kbdb 全套 192 個測試綠燈,tsc --noEmit 除既有 auth.test.ts 舊缺陷外無新增錯誤。 紅線:未併 main、未部署、未動任何實例的 is_embedded 旗標(只在本地 SQLite 測試治具跑過)。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -23,6 +23,7 @@ import {
|
||||
EmbedQueryFailedError,
|
||||
} from '../embed';
|
||||
import { migrateLegacyCredentialsForOwner } from '../actions/credential-legacy-migration';
|
||||
import { backfillEntryLibraryTags, libraryBackfillStatus } from '../actions/library-backfill';
|
||||
|
||||
export const entryRoutes = new Hono<{ Bindings: Bindings }>();
|
||||
|
||||
@@ -367,6 +368,64 @@ entryRoutes.patch('/deprecate-by-library', async (c) => {
|
||||
return c.json({ success: true, deprecated_count: count, vectors_deleted });
|
||||
});
|
||||
|
||||
// POST /entries/backfill-library — 標庫補存量(Arcrun#85 二次裁決/相關票 Arcrun#87,2026-08-11)。
|
||||
// body(必填 library + owner_id):{ library, owner_id, page_names?(string[],精準比對,
|
||||
// leo 定案的正解——見 actions/library-backfill.ts 檔頭「拿原稿遍歷」), entry_type?,
|
||||
// source_prefix?, page_name_prefix?(後兩者為過渡 fallback,精度不如 page_names),
|
||||
// since?, until?, limit?(1-500,預設100) }。
|
||||
// 冪等:只選「目前未標記 library」的候選;分批:單次 limit 上限,remaining>0 → 重複呼叫直到 0。
|
||||
// budget:與 /embed/reconcile 共用同一顆每日 D1 寫入額度(見 actions/maintenance-quota.ts)——
|
||||
// 兩者都是「多筆 D1 write、不打 AI」的背景維護操作,不共用額度的話補存量會把世代核對的閘繞過去。
|
||||
// base 對內容語意無知:不猜「這批該貼哪個庫」,呼叫端(ingest/#87)決定 library 與篩選條件;
|
||||
// owner_id 必填(同 /entries/deprecate-by-library 的既有防線——批次改一大片既有資料不准無租戶範圍地掃)。
|
||||
// 此路由必須在 '/:id' 之前註冊,否則 'backfill-library' 會被當成 id 參數。
|
||||
entryRoutes.post('/backfill-library', async (c) => {
|
||||
const body = (await c.req.json().catch(() => ({}))) as {
|
||||
library?: string;
|
||||
owner_id?: string;
|
||||
entry_type?: string;
|
||||
page_names?: string[];
|
||||
source_prefix?: string;
|
||||
page_name_prefix?: string;
|
||||
since?: number | string;
|
||||
until?: number | string;
|
||||
limit?: number | string;
|
||||
};
|
||||
const library = String(body.library ?? '').trim();
|
||||
const ownerId = String(body.owner_id ?? '').trim();
|
||||
if (!library || !ownerId) return c.json({ success: false, error: 'library 與 owner_id 必填' }, 400);
|
||||
try {
|
||||
const result = await backfillEntryLibraryTags(c.env.DB, c.env, {
|
||||
library,
|
||||
owner_id: ownerId,
|
||||
entry_type: body.entry_type || undefined,
|
||||
page_names: Array.isArray(body.page_names) && body.page_names.length > 0 ? body.page_names : undefined,
|
||||
source_prefix: body.source_prefix || undefined,
|
||||
page_name_prefix: body.page_name_prefix || undefined,
|
||||
since: body.since !== undefined ? Number(body.since) : undefined,
|
||||
until: body.until !== undefined ? Number(body.until) : undefined,
|
||||
limit: body.limit !== undefined ? Number(body.limit) : undefined,
|
||||
});
|
||||
return c.json({ success: true, ...result });
|
||||
} catch (e) {
|
||||
return c.json({ success: false, error: e instanceof Error ? e.message : String(e) }, 400);
|
||||
}
|
||||
});
|
||||
|
||||
// GET /entries/backfill-library/status?owner_id=&entry_type=&source_prefix=&page_name_prefix=&since=&until=
|
||||
// — 符合條件、目前未標記 library 的筆數(backfill 前後都能查,判斷還剩多少)。
|
||||
entryRoutes.get('/backfill-library/status', async (c) => {
|
||||
const status = await libraryBackfillStatus(c.env.DB, {
|
||||
owner_id: c.req.query('owner_id') || undefined,
|
||||
entry_type: c.req.query('entry_type') || undefined,
|
||||
source_prefix: c.req.query('source_prefix') || undefined,
|
||||
page_name_prefix: c.req.query('page_name_prefix') || undefined,
|
||||
since: c.req.query('since') ? Number(c.req.query('since')) : undefined,
|
||||
until: c.req.query('until') ? Number(c.req.query('until')) : undefined,
|
||||
});
|
||||
return c.json({ success: true, ...status });
|
||||
});
|
||||
|
||||
// PATCH /entries/:id
|
||||
entryRoutes.patch('/:id', async (c) => {
|
||||
const body = await c.req.json().catch(() => ({}));
|
||||
|
||||
Reference in New Issue
Block a user