feat: ingest 寫入端 + deprecate + get_source/refresh + wiki 合併 (issue #1 T3) (#2)

* chore(wiki): 導入 system-dev-template + 合併 wiki 到新位置

- system-dev/ 模板包進版控(VERSION/docs/scripts/wiki 骨架)
- 逐檔合併舊 .claude/wiki/ → system-dev/wiki/:
  - status/mistakes/decisions-summary 真資料覆蓋空範本
  - INDEX 新「多角度視圖」結構 + 舊決策/導航併入(過時詞「萬物皆 Block」改 API-as-Wall)
  - principles/TAXONOMY 為新位置獨有,保留
- 刪舊 .claude/wiki/(git 識別為 rename,內容完整搬移)
- 三層機敏防護 hooks + wiki 命令更新

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(sdd): 立 ingest-contract SDD + 搬入 ingest-candidate 契約 (T3.1+T3.8)

對應 issue #1(頂層 mira-dissolve T3)。

- contracts/ingest-candidate.json:ingest→graph 邊界契約(自頂層搬入)
- contracts/README.md:標明候選(輸入)≠已存(triplet)
- docs/3-specs/ingest-contract/design.md + tasks.md:
  - ensureTemplate 改 slot-diff 補丁(取代 early-return,免遷移腳本)
  - 補 KbdbClient.updateRecord(base PATCH /records/:id)
  - ingest 流程:驗證(422)→idempotency(uri+hash)→先 append 後 deprecate
  - triplet template 增 source_uri+content_hash slot 承載 idempotency
  - 跨 repo 協調點(3.6 圖工具併 KBDB MCP)明列需 arcrun 配合

總管已認可四個設計決定(issue #1 comment)。鐵律:零建表/零 SQL/零 migration。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(ingest): POST /triplets/ingest 寫入端 + deprecate-then-append (T3.2-3.5)

對應 issue #1 T3 B 段。

- templates: TRIPLET_SLOTS 加 status/superseded_by/source_uri/content_hash;
  ENTITY_SLOTS 加 gloss;recordToTriplet 映射新欄位(缺省 status=active 相容舊資料)
- kbdb-client: ensureTemplate 改 slot-diff 補丁(既有 template 走 PATCH /templates/:id
  補缺 slot,取代 early-return → 免遷移腳本);新增 updateRecord(PATCH /records/:id)
- triplet-ingest action(88 行純函式):Zod strict 鏡射 ingest-candidate 契約 →
  idempotency(uri+hash 同→no-op)→ 先 append 後 deprecate(無「全無 active」空窗)
- POST /triplets/ingest route:strict 驗證失敗 → 422(禁送 graph 領域欄位)
- queryTriplets 預設 active-only(traverse/search/neighbors 皆經此),
  includeDeprecated opt-out 供 rollback/考古
- 6 測試案全綠(vitest 16 passed);mock-client 同步 slot-diff + updateRecord

gates: zero SQL / zero migration / 無 D1·Vectorize·AI 綁定 / dry-run bundle 乾淨

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(graph): get_source + refresh 端點 + keyword 收斂 (T3.6-3.7)

對應 issue #1 T3 C 段(圖工具 HTTP API 備好,MCP 註冊薄殼待 arcrun)。

- get_source (3.7): graph-source.ts + GET /graph/source/:name —
  回節點的 active triplet 來源指標(uri/anchor/block_id/content_hash),去重。
  連帶加 source_anchor slot,ingest 從 source.anchor 帶入
- refresh (3.6/3.6b): graph-refresh.ts + POST /graph/refresh —
  純被動代轉 ingest(KBDB_INGEST_URL),只人發起、無排程/webhook(fan-out 紅線)。
  未設 URL → 誠實 forwarded:false,不假綠
- 3.6d: POST /search 移除公開 keyword 模式(重複 KBDB MCP),收斂 suggest-only;
  keywordSearch helper 留作 suggest 內部建構塊
- 3 新測試(get_source uri+anchor / active-only / refresh 未就緒誠實回報)

gates: vitest 19 passed / zero SQL / 無新綁定 / dry-run bundle 乾淨
待接:MCP 註冊薄殼併 arcrun u6u-mcp-server;refresh 端到端待 ingest(T4) 部署

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: richblack <leo21c@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
uncle6
2026-06-26 19:00:54 +08:00
committed by GitHub
parent 3a1faf19f4
commit 7a29dee357
44 changed files with 2773 additions and 96 deletions
+34
View File
@@ -0,0 +1,34 @@
// refreshT3.6 / T3.6b)— 代轉 ingest 重抓+萃某來源。
//
// 🚫 紅線:只能【人發起的 MCP 調用】觸發。禁掛排程/webhook 自動 refresh
// (否則變回 fan-out,踩 GitHub flag 紅線)。本端點純被動:收到一次調用 → 代轉一次。
// graph 自己不抓檔、不萃取(那是 ingest 純餵食器的職責);graph 只把 refresh 意圖
// 轉給 ingest 的端點,ingest 抓+萃完後再走 POST /triplets/ingest 回灌。
export type RefreshRequest = { uri: string; owner_id?: string };
export type RefreshResult = { forwarded: boolean; ingest_url?: string; note?: string };
/**
* 代轉 refresh 給 ingest 服務。ingestUrl 由 env 注入(KBDB_INGEST_URL)。
* 未設 → 誠實回 {forwarded:false}ingest repo T4 尚未就緒/未部署),不假裝成功。
*/
export async function refreshSource(
req: RefreshRequest,
ingestUrl: string | undefined,
): Promise<RefreshResult> {
if (!ingestUrl) {
return {
forwarded: false,
note: 'KBDB_INGEST_URL 未設:ingest 服務尚未就緒(T4 待部署),refresh 無對象可轉。',
};
}
const res = await fetch(ingestUrl.replace(/\/$/, '') + '/refresh', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ uri: req.uri, owner_id: req.owner_id }),
});
if (!res.ok) {
throw new Error(`[ingest] refresh ${req.uri}: ${res.status} ${res.statusText}`);
}
return { forwarded: true, ingest_url: ingestUrl };
}
+35
View File
@@ -0,0 +1,35 @@
// get_sourceT3.7)— 指回原文:給一個節點名,回它所有 triplet 的來源指標。
// 鐵律:走 base API、零 SQL。圖在插件層組裝。
// 用途:圖遍歷找到一筆知識後,回跳產生它的 canonical MDsource.uri + anchor)。
import type { KbdbClient } from '../lib/kbdb-client';
import { getNodeEdges } from './graph-nodes';
export type SourceRef = {
uri: string | null; // 來源穩定識別(github:owner/repo@path
anchor: string | null; // 檔內定位(heading slug / block id
block_id: string | null; // 向後相容:Logseq block id
content_hash: string | null; // 該批快照 hash
edge: { subject: string; predicate: string; object: string };
};
/** 給節點名,回觸及它的(active)triplet 的來源指標清單,去重同 uri+anchor。 */
export async function getSource(client: KbdbClient, node: string): Promise<SourceRef[]> {
const edges = await getNodeEdges(client, node); // 已 active-only(經 queryTriplets
const seen = new Set<string>();
const refs: SourceRef[] = [];
for (const t of edges) {
const key = `${t.source_uri ?? ''}#${t.source_anchor ?? ''}`;
if (seen.has(key)) continue;
seen.add(key);
refs.push({
uri: t.source_uri,
anchor: t.source_anchor,
block_id: t.source_block_id,
content_hash: t.content_hash,
edge: { subject: t.subject, predicate: t.predicate, object: t.object },
});
}
return refs;
}
+11
View File
@@ -18,6 +18,9 @@ export type CreateTripletData = {
bridge_score?: number;
subject_entity_type?: string;
object_entity_type?: string;
source_uri?: string;
content_hash?: string;
source_anchor?: string;
};
/** 建立三元組 → POST /recordstemplate=triplet)。 */
@@ -37,10 +40,14 @@ export async function createTriplet(
confidence: String(data.confidence ?? 1.0),
clusters_json: JSON.stringify(clusters),
bridge_score: String(bridgeScore),
status: 'active',
};
if (data.source_block_id) values.source_block_id = data.source_block_id;
if (data.subject_entity_type) values.subject_entity_type = data.subject_entity_type;
if (data.object_entity_type) values.object_entity_type = data.object_entity_type;
if (data.source_uri) values.source_uri = data.source_uri;
if (data.content_hash) values.content_hash = data.content_hash;
if (data.source_anchor) values.source_anchor = data.source_anchor;
const id = await client.createRecord(TPL_TRIPLET, values, data.owner_id);
return { id, subject: data.subject, predicate: data.predicate, object: data.object };
@@ -54,6 +61,7 @@ export type TripletFilters = {
offset?: number;
owner_id?: string;
entity_type?: string;
includeDeprecated?: boolean; // 預設只回 activerollback/考古才開(T3.5
};
/** 查三元組 → 取 template 全部 record,插件層 filterbase 無複合 slot 查詢)。 */
@@ -64,6 +72,9 @@ export async function queryTriplets(
const records = await client.listRecordsByTemplate(TPL_TRIPLET, filters.owner_id);
let triplets = records.map(recordToTriplet);
// active-onlydeprecated 不進圖遍歷/查詢(缺省 status 視為 active,相容舊資料)。
if (!filters.includeDeprecated) triplets = triplets.filter((t) => t.status === 'active');
if (filters.subject) triplets = triplets.filter((t) => t.subject === filters.subject);
if (filters.predicate) triplets = triplets.filter((t) => t.predicate === filters.predicate);
if (filters.object) triplets = triplets.filter((t) => t.object === filters.object);
+83
View File
@@ -0,0 +1,83 @@
// ingest 寫入端 — 收 ingest-candidate envelope,做 idempotency + deprecate-then-append。
// 契約:contracts/ingest-candidate.json。鐵律:走 base API、零 SQL。
// 取代策略:先 append 新批 active,後翻舊批 status=deprecated(中途失敗不留「全無 active」空窗)。
import { z } from '@hono/zod-openapi';
import type { KbdbClient } from '../lib/kbdb-client';
import { TPL_TRIPLET, ensurePluginTemplates, recordToTriplet } from '../lib/templates';
import { createTriplet } from './triplet-crud';
// Zod 鏡射契約:strict() = additionalProperties:false → 禁送欄位 422route 把 ZodError 轉 422)。
const NodeSchema = z.object({
name: z.string().min(1),
gloss: z.string().optional(),
entity_type: z.enum(['person', 'event', 'product', 'market', 'org']).optional(),
}).strict();
const EdgeSchema = z.object({
subject: z.string().min(1),
predicate: z.string().min(1),
object: z.string().min(1),
confidence: z.number().min(0).max(1).optional(),
}).strict();
export const IngestEnvelopeSchema = z.object({
source: z.object({
uri: z.string().min(1),
content_hash: z.string().min(1),
anchor: z.string().optional(),
commit: z.string().optional(),
block_id: z.string().optional(),
}).strict(),
extractor: z.object({
model: z.string().min(1),
tier: z.enum(['shallow', 'deep']),
extracted_at: z.number().int().optional(),
}).strict(),
nodes: z.array(NodeSchema).optional(),
triplets: z.array(EdgeSchema).min(1),
}).strict();
export type IngestEnvelope = z.infer<typeof IngestEnvelopeSchema>;
export type IngestResult = { skipped: boolean; ingested: number; deprecated: number };
/** 收 envelope → idempotency → 先 append 後 deprecate。回 {skipped,ingested,deprecated}。 */
export async function ingestEnvelope(
client: KbdbClient,
env: IngestEnvelope,
owner_id?: string,
): Promise<IngestResult> {
await ensurePluginTemplates(client);
// 同 source_uri 的現存 active tripletidempotency 分組 + 待 deprecate 對象)。
const all = (await client.listRecordsByTemplate(TPL_TRIPLET, owner_id)).map(recordToTriplet);
const priorActive = all.filter((t) => t.source_uri === env.source.uri && t.status === 'active');
// 同 hash → no-openvelope 已落地過)。
if (priorActive.some((t) => t.content_hash === env.source.content_hash)) {
return { skipped: true, ingested: 0, deprecated: 0 };
}
// 1) 先 append 新批 active。
for (const e of env.triplets) {
await createTriplet(client, {
subject: e.subject,
predicate: e.predicate,
object: e.object,
confidence: e.confidence,
source_block_id: env.source.block_id,
source_uri: env.source.uri,
content_hash: env.source.content_hash,
source_anchor: env.source.anchor,
owner_id,
});
}
// 2) 後翻舊批 status=deprecated(指向本批 source_uriappend 在前 → 無空窗)。
for (const old of priorActive) {
await client.updateRecord(old.id, { status: 'deprecated', superseded_by: env.source.content_hash });
}
return { skipped: false, ingested: env.triplets.length, deprecated: priorActive.length };
}
+21 -3
View File
@@ -113,12 +113,25 @@ export class KbdbClient {
// --- templates= 替代建表;插件要新類型只能建 template) ---
async ensureTemplate(name: string, slots: string[], description?: string): Promise<void> {
const existing = await this.req<{ id?: string } | { error: string }>(
const existing = await this.req<{ id?: string; slots?: string[] } | { error: string }>(
'GET',
`/templates/${encodeURIComponent(name)}`,
).catch(() => null);
if (existing && (existing as any).id) return;
await this.req('POST', '/templates', { name, slots, description, created_by: 'kbdb-graph' });
// 全新 template → 建。
if (!existing || !(existing as any).id) {
await this.req('POST', '/templates', { name, slots, description, created_by: 'kbdb-graph' });
return;
}
// 既有 template → 補缺 slot(不 early-return;否則 seed 後新增的 slot 永遠進不來)。
// 走 base PATCH /templates/:id 增 slot;既有環境免另跑遷移腳本即收斂。
const have = new Set((existing as any).slots ?? []);
const missing = slots.filter((s) => !have.has(s));
if (missing.length === 0) return;
await this.req('PATCH', `/templates/${encodeURIComponent((existing as any).id)}`, {
slots: [...have, ...missing],
});
}
// --- records= template 實例,填 slot ---
@@ -141,6 +154,11 @@ export class KbdbClient {
}
}
/** 翻 record 的 slot 值(base PATCH /records/:id)。deprecate(翻 status)與 rollback 都靠它。 */
async updateRecord(recordId: string, values: Record<string, string>): Promise<void> {
await this.req('PATCH', `/records/${encodeURIComponent(recordId)}`, { values });
}
async listRecordsByTemplate(template: string, owner_id?: string): Promise<BaseRecord[]> {
const { records } = await this.req<{ records: BaseRecord[] }>(
'GET',
+12 -1
View File
@@ -14,8 +14,13 @@ export const TRIPLET_SLOTS = [
'subject', 'predicate', 'object', 'source_block_id',
'confidence', 'clusters_json', 'bridge_score',
'subject_entity_type', 'object_entity_type',
// 取代/快照(T3.2):status=active|deprecatedsuperseded_by=取代它的新 record id
// source_uri+content_hash 承載 ingest idempotency(按 source_uri 分組 deprecate)。
// source_anchor 供 get_source 精準回跳原文(T3.7)。
'status', 'superseded_by', 'source_uri', 'content_hash', 'source_anchor',
];
export const ENTITY_SLOTS = ['canonical', 'aliases_json', 'entity_type', 'owner'];
// glossT3.2b):一句話描述,供「詞+gloss」語義 normalize 的 embedding 對象。
export const ENTITY_SLOTS = ['canonical', 'aliases_json', 'entity_type', 'owner', 'gloss'];
export const ENTITY_PENDING_SLOTS = [
'raw_name', 'candidate_entity_id', 'candidate_canonical', 'similarity',
];
@@ -41,6 +46,12 @@ export function recordToTriplet(rec: BaseRecord): Triplet {
bridge_score: parseInt(v.bridge_score ?? '0', 10),
subject_entity_type: (v.subject_entity_type as Triplet['subject_entity_type']) || null,
object_entity_type: (v.object_entity_type as Triplet['object_entity_type']) || null,
// 缺省視為 active(相容尚無 status slot 的舊資料)。
status: v.status === 'deprecated' ? 'deprecated' : 'active',
superseded_by: v.superseded_by || null,
source_uri: v.source_uri || null,
content_hash: v.content_hash || null,
source_anchor: v.source_anchor || null,
created_at: 0,
updated_at: 0,
};
+19
View File
@@ -6,6 +6,8 @@ import type { Bindings } from '../types';
import { traverseGraph, queryRelation } from '../actions/graph-traverse';
import { getNodeEdges, getNeighbors } from '../actions/graph-nodes';
import { findShortestPath } from '../actions/graph-path';
import { getSource } from '../actions/graph-source';
import { refreshSource } from '../actions/graph-refresh';
import { makeKbdbClient } from '../lib/kbdb-client';
const graphRoutes = new Hono<{ Bindings: Bindings }>();
@@ -46,4 +48,21 @@ graphRoutes.get('/path', async (c) => {
return c.json({ from, to, ...result });
});
// GET /graph/source/:name — get_sourceT3.7):回節點的原文來源指標(uri+anchor)
graphRoutes.get('/source/:name', async (c) => {
const name = decodeURIComponent(c.req.param('name'));
const sources = await getSource(makeKbdbClient(c.env), name);
return c.json({ node: name, sources, count: sources.length });
});
// POST /graph/refresh — 代轉 ingest 重抓+萃(T3.6/3.6b
// 🚫 只人發起 MCP 調用觸發,禁排程/webhook 自動(fan-out 紅線)。
graphRoutes.post('/refresh', async (c) => {
const body = await c.req.json().catch(() => ({}));
const uri = body?.uri;
if (!uri) return c.json({ error: 'uri required' }, 400);
const result = await refreshSource({ uri, owner_id: body?.owner_id }, c.env.KBDB_INGEST_URL);
return c.json(result);
});
export { graphRoutes };
+9 -16
View File
@@ -1,43 +1,36 @@
// 搜尋路由入口 — 僅驗證參數,呼叫 actions
//
// 插件層只做基本盤 keyword 搜尋(D1 LIKE,走 GET /entries/search)。
// T3.6d:移除 graph 的 keyword 搜尋【公開端點】——純關鍵字搜尋重複於 KBDB MCP 的 kbdb_search
// 歸 KBDB MCP,不由 graph 代理。graph 只保留圖專屬的 `suggest`(跨知識建議)。
// 注意:keywordSearch helper 仍保留,作為 suggest 的內部建構塊(非對外端點)。
// 語意搜尋 / embedding 屬基本盤 optional embed 模組,不在插件 → 已移除 /search/embed。
import { Hono } from 'hono';
import { z } from 'zod';
import type { Bindings, Variables } from '../types';
import { keywordSearch } from '../actions/search-query';
import { suggestKnowledge } from '../actions/search-suggest';
import { makeKbdbClient } from '../lib/kbdb-client';
const searchRoutes = new Hono<{ Bindings: Bindings; Variables: Variables }>();
const UnifiedSearchSchema = z.object({
const SuggestSchema = z.object({
query: z.string().min(1),
type: z.enum(['keyword', 'suggest']).optional().default('keyword'),
topK: z.number().min(1).max(20).optional(),
owner_id: z.string().optional(),
});
// 統一搜尋入口:POST /search
// 統一搜尋入口:POST /search — 僅 suggestkeyword 歸 KBDB MCPT3.6d
searchRoutes.post('/', async (c) => {
const parsed = UnifiedSearchSchema.safeParse(await c.req.json());
const parsed = SuggestSchema.safeParse(await c.req.json());
if (!parsed.success) return c.json({ error: parsed.error.flatten() }, 400);
const { query, type, topK, owner_id } = parsed.data;
const { query, topK, owner_id } = parsed.data;
// Namespace 讀取過濾:partner token 只能搜到自己 namespace 的資料
const namespace = c.get('namespace');
const effectiveOwner = namespace ?? owner_id;
const client = makeKbdbClient(c.env);
if (type === 'suggest') {
const result = await suggestKnowledge(client, query, topK, effectiveOwner);
return c.json(result);
}
const matches = await keywordSearch(client, query, { limit: topK, owner_id: effectiveOwner });
return c.json({ matches, count: matches.length, mode: 'keyword' });
const result = await suggestKnowledge(makeKbdbClient(c.env), query, topK, effectiveOwner);
return c.json(result);
});
export { searchRoutes };
+28
View File
@@ -2,6 +2,7 @@ import { createRoute, OpenAPIHono, z } from '@hono/zod-openapi';
import type { Bindings, Variables } from '../types';
import { createTriplet, queryTriplets } from '../actions/triplet-crud';
import { getTripletStats } from '../actions/triplet-stats';
import { ingestEnvelope, IngestEnvelopeSchema } from '../actions/triplet-ingest';
import { makeKbdbClient } from '../lib/kbdb-client';
const tripletRoutes = new OpenAPIHono<{ Bindings: Bindings; Variables: Variables }>();
@@ -97,4 +98,31 @@ tripletRoutes.openapi(createRouteDefinition, async (c) => {
return c.json({ ok: true }, 201);
});
// POST /triplets/ingest — 收 ingest-candidate envelopeidempotency + deprecate-then-append
const ingestRoute = createRoute({
method: 'post',
path: '/ingest',
request: {
body: { content: { 'application/json': { schema: IngestEnvelopeSchema } } },
},
responses: {
200: { description: 'Envelope ingested (or skipped if same content_hash)' },
422: { description: 'Invalid envelope (forbidden field or shape mismatch)' },
},
tags: ['Triplets'],
});
tripletRoutes.openapi(
ingestRoute,
async (c) => {
const env = c.req.valid('json');
const result = await ingestEnvelope(makeKbdbClient(c.env), env);
return c.json(result, 200);
},
// strict() 驗證失敗(如送禁止欄位 bridge_score)→ 422,不是預設 400。
(zres, c) => {
if (!zres.success) return c.json({ error: 'invalid envelope', issues: zres.error.issues }, 422);
},
);
export { tripletRoutes };
+8
View File
@@ -4,6 +4,7 @@ export type Bindings = {
// 插件不碰 DB/Vectorize/AI — 全走基本盤 APIAPI-as-Wall)。
// 語意搜尋/embedding 屬基本盤 optional embed 模組,不在插件。
KBDB_BASE_URL?: string; // 基本盤 arcrun/kbdb API 網址(leo: 可設定,先留空)
KBDB_INGEST_URL?: string; // ingest 服務網址(refresh 代轉對象;T4 就緒前留空)
KBDB_INTERNAL_TOKEN?: string;
ENVIRONMENT: string;
API_KEY?: string;
@@ -17,6 +18,8 @@ export type Variables = {
export type EntityType = 'person' | 'event' | 'product' | 'market' | 'org';
export type TripletStatus = 'active' | 'deprecated';
export type Triplet = {
id: string;
subject: string;
@@ -28,6 +31,11 @@ export type Triplet = {
bridge_score: number; // 跨越的 cluster 數量,Scout 發現指標
subject_entity_type: EntityType | null; // 主體 entity 類型(人格疊加局勢分析用)
object_entity_type: EntityType | null; // 客體 entity 類型
status: TripletStatus; // active(進圖遍歷)| deprecated(被取代,可查/可 rollback
superseded_by: string | null; // 取代它的新 record idactive 時為 null
source_uri: string | null; // ingest 來源穩定識別(idempotency 分組鍵 + get_source 指標)
content_hash: string | null; // 來源快照 hashidempotency 比對鍵)
source_anchor: string | null; // 檔內定位(heading slug / block id),get_source 精準回跳
created_at: number;
updated_at: number;
};