步驟6 執行統計回寫:每顆零件跑完回寫 success_rate(task 29,design.md「執行統計設計」)
- registry 新增 POST /analytics/record:ANALYTICS_KV 計數器(stats:{hash_id}:{version})
為唯一真相源,衍生值(success_rate/avg_duration_ms/call_count)回填 comp: 記錄
——查詢讀取端讀哪就寫哪,不開第二真相源。KV 無 CAS,誠實標註非原子。
- cypher-executor execution-evaluator 從 stub 改真實作:執行收尾(/cypher/execute
成功與 ExecutionError 路徑+webhook 路徑)對 trace 裡每顆 Component 節點
fire-and-forget 回寫,waitUntil 包、不增加執行同步延遲(仿 recordRecipeStats 慣例)。
成敗判定=trace error 或 output.success===false(makeHttpRunner 非 2xx 不 throw)。
- registry 位置沿 search-nodes 慣例:REGISTRY_BASE_URL 覆蓋,未設走 wasmWorkerUrl。
- 新增單測 13 個全綠;本地雙 wrangler dev 端到端實測:http_request 跑 5 次
(3 成功+2 失敗)→ success_rate 1→0.6、call_count 0→5,/cypher/search 同步可見。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -3,7 +3,7 @@ import { ExecutionError, WorkflowPaused } from '../types';
|
||||
import { GraphExecutor } from '../graph-executor';
|
||||
import { graphSchema } from '../lib/schemas';
|
||||
import { createComponentLoader } from '../lib/component-loader';
|
||||
import { writeEvaluation, updateComponentStats } from './execution-evaluator';
|
||||
import { recordComponentStats } from './execution-evaluator';
|
||||
import { parseTriplets } from './triplet-parser';
|
||||
import { searchNodes } from './search-nodes';
|
||||
import { buildExecutionGraph } from './graph-builder';
|
||||
@@ -68,18 +68,8 @@ export async function handleCypherExecute(
|
||||
const result = await executor.execute(parseResult.data as ExecutionGraph, context ?? {}, env.EXEC_CONTEXT);
|
||||
const duration_ms = Date.now() - start;
|
||||
|
||||
// 非同步記錄統計(Phase 7 補充 analytics,目前為 no-op)
|
||||
const componentId = graph.nodes.find(n => n.componentId)?.componentId ?? graphId;
|
||||
const runId = `${graphId}-${Date.now()}`;
|
||||
waitUntil(writeEvaluation(env, {
|
||||
run_id: runId,
|
||||
workflow_id: graphId,
|
||||
component_id: componentId,
|
||||
verdict: 'success',
|
||||
duration_ms,
|
||||
evaluated_at: Date.now(),
|
||||
}));
|
||||
waitUntil(updateComponentStats(env, componentId, 'success', duration_ms));
|
||||
// 非同步回寫每顆零件的執行統計(design.md「執行統計設計」;fire-and-forget 不阻擋回應)
|
||||
waitUntil(recordComponentStats(env, graph.nodes, result.trace));
|
||||
|
||||
return { success: true, data: result.data, trace: result.trace, duration_ms, graph };
|
||||
} catch (err) {
|
||||
@@ -101,19 +91,10 @@ export async function handleCypherExecute(
|
||||
}
|
||||
|
||||
const errMsg = err instanceof Error ? err.message : String(err);
|
||||
const componentId = graph.nodes.find(n => n.componentId)?.componentId ?? graphId;
|
||||
const runId = `${graphId}-${Date.now()}`;
|
||||
waitUntil(writeEvaluation(env, {
|
||||
run_id: runId,
|
||||
workflow_id: graphId,
|
||||
component_id: componentId,
|
||||
verdict: 'failed',
|
||||
duration_ms,
|
||||
error_message: errMsg.slice(0, 200),
|
||||
evaluated_at: Date.now(),
|
||||
}));
|
||||
waitUntil(updateComponentStats(env, componentId, 'failed', duration_ms));
|
||||
// 失敗路徑同樣回寫每顆零件統計:ExecutionError 帶完整 trace(失敗節點有 error、
|
||||
// 之前成功的節點照記成功);非 ExecutionError 無 trace 可歸因 → 不記(誠實:不瞎猜)。
|
||||
if (err instanceof ExecutionError) {
|
||||
waitUntil(recordComponentStats(env, graph.nodes, err.trace));
|
||||
const traceFormatted = err.trace.map(s => ({
|
||||
node: s.nodeId,
|
||||
status: s.error ? 'failed' : 'success',
|
||||
|
||||
@@ -1,36 +1,96 @@
|
||||
/**
|
||||
* Execution Analytics — 零件執行後的統計記錄
|
||||
* Execution Analytics — 零件執行後的統計回寫
|
||||
*
|
||||
* Phase 1 MVP:stub(不寫入任何外部服務)
|
||||
* Phase 7 補充:fire-and-forget POST 至 registry.arcrun.dev/analytics/record
|
||||
* SDD: system-dev/docs/3-specs/arcrun-core-mvp/design.md「執行統計設計」
|
||||
* 執行完成處(cypher-handlers / webhook-handlers 收尾)對本次用到的**每顆零件**
|
||||
* fire-and-forget POST registry `/analytics/record`——統計失敗不影響執行、不增加同步延遲
|
||||
* (呼叫端一律用 waitUntil 包,仿 recordRecipeStats / recordTelemetry 既有慣例)。
|
||||
*
|
||||
* 每顆零件的成敗判定來源=執行 trace(per-node):
|
||||
* - trace step 有 `error` → 失敗(runner throw)
|
||||
* - output 是物件且 `success === false` → 失敗(makeHttpRunner 對非 2xx 不 throw,回這種)
|
||||
* - 其餘 → 成功
|
||||
* FOREACH 重複執行同一節點 → trace 有幾筆就記幾次(每次真實執行都算一次樣本)。
|
||||
*/
|
||||
|
||||
import type { Bindings } from '../types';
|
||||
import type { GraphNode, TraceStep } from '../types';
|
||||
import { wasmWorkerUrl } from '../lib/component-loader';
|
||||
|
||||
export interface EvaluationRecord {
|
||||
run_id: string;
|
||||
workflow_id: string;
|
||||
/** 本模組需要的環境子集(傳整份 Bindings 也相容,仿 SearchNodesEnv 慣例)。 */
|
||||
export type AnalyticsEnv = {
|
||||
WORKER_SUBDOMAIN?: string;
|
||||
/** registry 位置覆蓋(可選;本地 wrangler dev / self-hosted 用)。未設 → wasmWorkerUrl('registry', WORKER_SUBDOMAIN)。 */
|
||||
REGISTRY_BASE_URL?: string;
|
||||
};
|
||||
|
||||
export interface ComponentVerdict {
|
||||
component_id: string;
|
||||
verdict: 'success' | 'failed' | 'timeout';
|
||||
success: boolean;
|
||||
duration_ms: number;
|
||||
error_message?: string;
|
||||
evaluated_at: number;
|
||||
}
|
||||
|
||||
/** 記錄執行結果(MVP:no-op,Phase 7 補充 analytics)*/
|
||||
export async function writeEvaluation(
|
||||
_env: Bindings,
|
||||
_record: EvaluationRecord,
|
||||
): Promise<void> {
|
||||
// Phase 7: POST to registry.arcrun.dev/analytics/record
|
||||
/** 從執行 trace 導出每顆零件的成敗(只算 type=Component 且有 componentId 的節點)。 */
|
||||
export function componentVerdictsFromTrace(
|
||||
nodes: GraphNode[],
|
||||
trace: TraceStep[],
|
||||
): ComponentVerdict[] {
|
||||
const componentByNodeId = new Map<string, string>();
|
||||
for (const n of nodes) {
|
||||
if (n.type === 'Component' && n.componentId) componentByNodeId.set(n.id, n.componentId);
|
||||
}
|
||||
|
||||
const verdicts: ComponentVerdict[] = [];
|
||||
for (const step of trace) {
|
||||
const componentId = componentByNodeId.get(step.nodeId);
|
||||
if (!componentId) continue;
|
||||
|
||||
const out = step.output;
|
||||
const outputSaysFailed =
|
||||
typeof out === 'object' && out !== null && !Array.isArray(out) &&
|
||||
(out as Record<string, unknown>).success === false;
|
||||
|
||||
verdicts.push({
|
||||
component_id: componentId,
|
||||
success: !step.error && !outputSaysFailed,
|
||||
duration_ms: Math.max(0, Number(step.duration_ms) || 0),
|
||||
});
|
||||
}
|
||||
return verdicts;
|
||||
}
|
||||
|
||||
/** 更新零件統計(MVP:no-op,Phase 7 補充)*/
|
||||
export async function updateComponentStats(
|
||||
_env: Bindings,
|
||||
_componentId: string,
|
||||
_verdict: 'success' | 'failed' | 'timeout',
|
||||
_durationMs: number,
|
||||
/**
|
||||
* 對本次執行用到的每顆零件回寫統計到 registry(design.md「Analytics Record」)。
|
||||
* 永不 throw;呼叫端用 waitUntil 包,不阻擋主流程。
|
||||
*/
|
||||
export async function recordComponentStats(
|
||||
env: AnalyticsEnv,
|
||||
nodes: GraphNode[],
|
||||
trace: TraceStep[],
|
||||
): Promise<void> {
|
||||
// Phase 7: update ANALYTICS_KV via registry worker
|
||||
try {
|
||||
const base = (
|
||||
env.REGISTRY_BASE_URL ??
|
||||
(env.WORKER_SUBDOMAIN ? wasmWorkerUrl('registry', env.WORKER_SUBDOMAIN) : undefined)
|
||||
)?.replace(/\/$/, '');
|
||||
if (!base) return;
|
||||
|
||||
const verdicts = componentVerdictsFromTrace(nodes, trace);
|
||||
if (verdicts.length === 0) return;
|
||||
|
||||
await Promise.all(
|
||||
verdicts.map(v =>
|
||||
fetch(`${base}/analytics/record`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
canonical_id: v.component_id,
|
||||
success: v.success,
|
||||
duration_ms: v.duration_ms,
|
||||
}),
|
||||
}).catch(() => undefined), // 統計失敗不影響執行
|
||||
),
|
||||
);
|
||||
} catch {
|
||||
// fire-and-forget:不拋錯,不影響主流程
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,6 +4,8 @@ import { GraphExecutor } from '../graph-executor';
|
||||
import { graphSchema } from '../lib/schemas';
|
||||
import { createComponentLoader } from '../lib/component-loader';
|
||||
import { recordTelemetry } from '../lib/telemetry';
|
||||
import { recordComponentStats } from './execution-evaluator';
|
||||
import type { GraphNode, TraceStep } from '../types';
|
||||
|
||||
/**
|
||||
* kbdb-base §7.1+§7.5.h:一條工作流執行結束後,把這次用到的 recipe 各記一次成功/失敗到 KBDB 市場星數。
|
||||
@@ -96,6 +98,17 @@ export async function executeWebhookGraph(
|
||||
// kbdb-base §7.1:整體成功 → 用到的 recipe 各記成功一次。
|
||||
recordRecipeStats(env, executor.usedRecipeKeys, true, Date.now(), ctx);
|
||||
|
||||
// arcrun-core-mvp「執行統計設計」:對用到的每顆零件回寫執行結果(fire-and-forget)。
|
||||
{
|
||||
const statsPromise = recordComponentStats(
|
||||
env,
|
||||
(parsed.data as ExecutionGraph).nodes as GraphNode[],
|
||||
result.trace as TraceStep[],
|
||||
);
|
||||
if (ctx?.waitUntil) ctx.waitUntil(statsPromise);
|
||||
else void statsPromise;
|
||||
}
|
||||
|
||||
return { success: true, data: result.data, duration_ms };
|
||||
} catch (err) {
|
||||
const duration_ms = Date.now() - start;
|
||||
@@ -117,6 +130,18 @@ export async function executeWebhookGraph(
|
||||
recordRecipeStats(env, executor.usedRecipeKeys, false, Date.now(), ctx);
|
||||
}
|
||||
|
||||
// 零件統計失敗路徑:ExecutionError 帶完整 trace(失敗節點有 error、先前成功節點照記成功);
|
||||
// paused 非失敗不記;非 ExecutionError 無 trace 可歸因 → 不記。
|
||||
if (!isPaused && err instanceof ExecutionError) {
|
||||
const statsPromise = recordComponentStats(
|
||||
env,
|
||||
(parsed.data as ExecutionGraph).nodes as GraphNode[],
|
||||
err.trace,
|
||||
);
|
||||
if (ctx?.waitUntil) ctx.waitUntil(statsPromise);
|
||||
else void statsPromise;
|
||||
}
|
||||
|
||||
if (err instanceof ExecutionError) {
|
||||
const traceFormatted = err.trace.map(s => ({
|
||||
node: s.nodeId,
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
// 單元測試:execution-evaluator — 從 trace 導出每顆零件成敗 + 回寫 registry
|
||||
// SDD: system-dev/docs/3-specs/arcrun-core-mvp/design.md「執行統計設計」
|
||||
|
||||
import { describe, it, expect, vi, afterEach } from 'vitest';
|
||||
import { componentVerdictsFromTrace, recordComponentStats } from '../src/actions/execution-evaluator';
|
||||
import type { GraphNode, TraceStep } from '../src/types';
|
||||
|
||||
const NODES: GraphNode[] = [
|
||||
{ id: 'input', type: 'Input' },
|
||||
{ id: 'fetch', type: 'Component', componentId: 'http_request' },
|
||||
{ id: 'transform', type: 'Component', componentId: 'code' },
|
||||
{ id: 'output', type: 'Output' },
|
||||
];
|
||||
|
||||
function step(nodeId: string, over: Partial<TraceStep> = {}): TraceStep {
|
||||
return { nodeId, type: 'Component', input: {}, output: { ok: true }, duration_ms: 10, ...over };
|
||||
}
|
||||
|
||||
describe('componentVerdictsFromTrace', () => {
|
||||
it('只算 Component 節點;Input/Output 跳過', () => {
|
||||
const verdicts = componentVerdictsFromTrace(NODES, [
|
||||
step('input', { type: 'Input' }),
|
||||
step('fetch'),
|
||||
step('output', { type: 'Output' }),
|
||||
]);
|
||||
expect(verdicts).toEqual([{ component_id: 'http_request', success: true, duration_ms: 10 }]);
|
||||
});
|
||||
|
||||
it('trace 有 error → 該零件記失敗', () => {
|
||||
const verdicts = componentVerdictsFromTrace(NODES, [
|
||||
step('fetch', { error: 'boom', output: null }),
|
||||
]);
|
||||
expect(verdicts).toEqual([{ component_id: 'http_request', success: false, duration_ms: 10 }]);
|
||||
});
|
||||
|
||||
it('output.success === false → 記失敗(makeHttpRunner 對非 2xx 不 throw)', () => {
|
||||
const verdicts = componentVerdictsFromTrace(NODES, [
|
||||
step('fetch', { output: { success: false, status: 500, error: 'oops' } }),
|
||||
]);
|
||||
expect(verdicts[0].success).toBe(false);
|
||||
});
|
||||
|
||||
it('FOREACH 同節點多筆 trace → 每次執行各記一次樣本', () => {
|
||||
const verdicts = componentVerdictsFromTrace(NODES, [
|
||||
step('fetch'),
|
||||
step('fetch', { error: 'x', output: null }),
|
||||
step('fetch'),
|
||||
]);
|
||||
expect(verdicts).toHaveLength(3);
|
||||
expect(verdicts.map(v => v.success)).toEqual([true, false, true]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('recordComponentStats', () => {
|
||||
afterEach(() => vi.unstubAllGlobals());
|
||||
|
||||
it('對每顆零件各發一次 POST /analytics/record(fire-and-forget)', async () => {
|
||||
const calls: Array<{ url: string; body: Record<string, unknown> }> = [];
|
||||
vi.stubGlobal('fetch', vi.fn(async (url: string, init: RequestInit) => {
|
||||
calls.push({ url: String(url), body: JSON.parse(String(init.body)) });
|
||||
return new Response('{}', { status: 200 });
|
||||
}));
|
||||
|
||||
await recordComponentStats(
|
||||
{ REGISTRY_BASE_URL: 'http://registry.local' },
|
||||
NODES,
|
||||
[step('fetch'), step('transform', { error: 'bad', output: null })],
|
||||
);
|
||||
|
||||
expect(calls).toHaveLength(2);
|
||||
expect(calls[0].url).toBe('http://registry.local/analytics/record');
|
||||
expect(calls[0].body).toEqual({ canonical_id: 'http_request', success: true, duration_ms: 10 });
|
||||
expect(calls[1].body).toEqual({ canonical_id: 'code', success: false, duration_ms: 10 });
|
||||
});
|
||||
|
||||
it('registry 打不到也不 throw(統計失敗不影響執行)', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn(async () => { throw new Error('network down'); }));
|
||||
await expect(
|
||||
recordComponentStats({ REGISTRY_BASE_URL: 'http://registry.local' }, NODES, [step('fetch')]),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it('無 REGISTRY_BASE_URL 也無 WORKER_SUBDOMAIN → 靜默略過不打', async () => {
|
||||
const fetchSpy = vi.fn();
|
||||
vi.stubGlobal('fetch', fetchSpy);
|
||||
await recordComponentStats({}, NODES, [step('fetch')]);
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('未設 REGISTRY_BASE_URL → 用 wasmWorkerUrl 慣例組 registry URL', async () => {
|
||||
const calls: string[] = [];
|
||||
vi.stubGlobal('fetch', vi.fn(async (url: string) => {
|
||||
calls.push(String(url));
|
||||
return new Response('{}', { status: 200 });
|
||||
}));
|
||||
await recordComponentStats({ WORKER_SUBDOMAIN: 'uncle6-me' }, NODES, [step('fetch')]);
|
||||
expect(calls[0]).toBe('https://arcrun-registry.uncle6-me.workers.dev/analytics/record');
|
||||
});
|
||||
});
|
||||
@@ -137,7 +137,8 @@ export async function searchComponents(
|
||||
|
||||
// ── 內部工具函數 ──────────────────────────────────────────────────────────────
|
||||
|
||||
function computeScore(v: Record<string, unknown>): number {
|
||||
// export:recordAnalytics 選「最優版本」要與 getComponent 同判準(單一評分真相)
|
||||
export function computeScore(v: Record<string, unknown>): number {
|
||||
const successRate = parseFloat(String(v.success_rate ?? '1'));
|
||||
const avgDuration = parseFloat(String(v.avg_duration_ms ?? '10'));
|
||||
const callCount = parseInt(String(v.call_count ?? '0'), 10);
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
// recordAnalytics — 零件執行結果回寫(POST /analytics/record 的實作)
|
||||
// SDD: system-dev/docs/3-specs/arcrun-core-mvp/design.md「執行統計設計」
|
||||
//
|
||||
// 真相源=ANALYTICS_KV 計數器(key = stats:{hash_id}:{version},見 src/types.ts 註記):
|
||||
// { total_runs, success_runs, total_ms }
|
||||
// 讀取端(queryComponents / GET /components/*)讀的是 comp: 記錄上的
|
||||
// success_rate / avg_duration_ms / call_count 欄位——所以每次記錄後把衍生值
|
||||
// 回填 comp: 記錄(唯一寫入者是本函式,衍生值不是第二個真相源)。
|
||||
//
|
||||
// 尺度註記:design.md:499 寫 success_rate = success_runs / total_runs * 100(百分比顯示)。
|
||||
// 本 repo comp: 記錄的 success_rate 既有尺度是 0..1(預設 1,computeScore 直接相乘),
|
||||
// 為不破壞既有讀取端與未測零件的預設值,落庫維持 0..1;*100 只是等價的顯示換算。
|
||||
//
|
||||
// 誠實限制:CF KV 無 CAS/原子遞增,read-modify-write 併發下偶有丟計數;
|
||||
// 統計用途可接受(design 的「樂觀鎖」在 KV 上做不到,不假裝做到了)。
|
||||
|
||||
import type { Bindings } from '../types';
|
||||
import { computeScore } from './queryComponents';
|
||||
|
||||
export interface AnalyticsRecordInput {
|
||||
canonical_id: string; // 也接受 cmp_xxxxxxxx hash_id
|
||||
version?: string; // 不給 → 記到查詢會回的最優版本(getComponent 同判準)
|
||||
success: boolean;
|
||||
duration_ms: number;
|
||||
}
|
||||
|
||||
export interface AnalyticsRecordResult {
|
||||
ok: boolean;
|
||||
error?: string;
|
||||
canonical_id?: string;
|
||||
version?: string;
|
||||
total_runs?: number;
|
||||
success_runs?: number;
|
||||
success_rate?: number; // 0..1,與 comp: 記錄同尺度
|
||||
avg_duration_ms?: number;
|
||||
}
|
||||
|
||||
interface StatsCounters {
|
||||
total_runs: number;
|
||||
success_runs: number;
|
||||
total_ms: number;
|
||||
}
|
||||
|
||||
export async function recordAnalytics(
|
||||
input: AnalyticsRecordInput,
|
||||
env: Bindings,
|
||||
): Promise<AnalyticsRecordResult> {
|
||||
// 1. 解析 hash_id(與 queryComponents.resolveHashId 同規則)
|
||||
const hashId = input.canonical_id.startsWith('cmp_')
|
||||
? input.canonical_id
|
||||
: await env.SUBMISSIONS_KV.get(`idx:${input.canonical_id}`);
|
||||
if (!hashId) {
|
||||
return { ok: false, error: `零件 ${input.canonical_id} 不在索引` };
|
||||
}
|
||||
|
||||
// 2. 找目標版本記錄:指定 version 就用它;沒指定 → 最優版本(與 getComponent 同判準:score 最高)
|
||||
const list = await env.SUBMISSIONS_KV.list({ prefix: `comp:${hashId}:` });
|
||||
let targetKey: string | null = null;
|
||||
let targetRecord: Record<string, unknown> | null = null;
|
||||
let bestScore = -Infinity;
|
||||
|
||||
for (const key of list.keys) {
|
||||
const raw = await env.SUBMISSIONS_KV.get(key.name);
|
||||
if (!raw) continue;
|
||||
let v: Record<string, unknown>;
|
||||
try { v = JSON.parse(raw); } catch { continue; }
|
||||
if (v.status === 'tombstone') continue;
|
||||
|
||||
if (input.version) {
|
||||
if (String(v.version) === input.version) {
|
||||
targetKey = key.name;
|
||||
targetRecord = v;
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const score = computeScore(v);
|
||||
if (score > bestScore) {
|
||||
bestScore = score;
|
||||
targetKey = key.name;
|
||||
targetRecord = v;
|
||||
}
|
||||
}
|
||||
|
||||
if (!targetKey || !targetRecord) {
|
||||
return { ok: false, error: `零件 ${input.canonical_id} 無可用版本記錄` };
|
||||
}
|
||||
|
||||
const version = String(targetRecord.version ?? 'v1');
|
||||
const canonicalId = String(targetRecord.canonical_id ?? input.canonical_id);
|
||||
|
||||
// 3. 更新計數器(真相源,ANALYTICS_KV stats:{hash_id}:{version})
|
||||
const statsKey = `stats:${hashId}:${version}`;
|
||||
let counters: StatsCounters = { total_runs: 0, success_runs: 0, total_ms: 0 };
|
||||
const rawStats = await env.ANALYTICS_KV.get(statsKey);
|
||||
if (rawStats) {
|
||||
try {
|
||||
const parsed = JSON.parse(rawStats) as Partial<StatsCounters>;
|
||||
counters = {
|
||||
total_runs: Number(parsed.total_runs) || 0,
|
||||
success_runs: Number(parsed.success_runs) || 0,
|
||||
total_ms: Number(parsed.total_ms) || 0,
|
||||
};
|
||||
} catch { /* 損毀計數器 → 重新起算 */ }
|
||||
}
|
||||
counters.total_runs += 1;
|
||||
counters.success_runs += input.success ? 1 : 0;
|
||||
counters.total_ms += Math.max(0, Number(input.duration_ms) || 0);
|
||||
await env.ANALYTICS_KV.put(statsKey, JSON.stringify(counters));
|
||||
|
||||
// 4. 衍生值回填 comp: 記錄(讀取端讀的地方)
|
||||
const successRate = counters.success_runs / counters.total_runs;
|
||||
const avgDurationMs = Math.round(counters.total_ms / counters.total_runs);
|
||||
targetRecord.success_rate = successRate;
|
||||
targetRecord.avg_duration_ms = avgDurationMs;
|
||||
targetRecord.call_count = counters.total_runs;
|
||||
await env.SUBMISSIONS_KV.put(targetKey, JSON.stringify(targetRecord));
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
canonical_id: canonicalId,
|
||||
version,
|
||||
total_runs: counters.total_runs,
|
||||
success_runs: counters.success_runs,
|
||||
success_rate: successRate,
|
||||
avg_duration_ms: avgDurationMs,
|
||||
};
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import validateContractRoute from './routes/validateContract';
|
||||
import componentsRoute from './routes/components';
|
||||
import queryRoute from './routes/query';
|
||||
import initRoute from './routes/init';
|
||||
import analyticsRoute from './routes/analytics';
|
||||
|
||||
const app = new Hono<{ Bindings: Bindings }>();
|
||||
app.use('*', cors());
|
||||
@@ -25,4 +26,7 @@ app.route('/components', componentsRoute); // POST /components
|
||||
// === 初始化端點(建立 tpl-component template)===
|
||||
app.route('/init', initRoute);
|
||||
|
||||
// === 執行統計回寫(cypher-executor 執行收尾 fire-and-forget 打進來)===
|
||||
app.route('/analytics', analyticsRoute); // POST /analytics/record
|
||||
|
||||
export default app;
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
// POST /analytics/record — 零件執行結果回寫端點
|
||||
// SDD: system-dev/docs/3-specs/arcrun-core-mvp/design.md「執行統計設計」
|
||||
// 呼叫方:cypher-executor 執行收尾(fire-and-forget,統計失敗不影響執行)
|
||||
|
||||
import { Hono } from 'hono';
|
||||
import type { Bindings } from '../types';
|
||||
import { recordAnalytics } from '../actions/recordAnalytics';
|
||||
|
||||
const app = new Hono<{ Bindings: Bindings }>();
|
||||
|
||||
app.post('/record', async c => {
|
||||
const body = await c.req.json().catch(() => null) as {
|
||||
canonical_id?: unknown;
|
||||
version?: unknown;
|
||||
success?: unknown;
|
||||
duration_ms?: unknown;
|
||||
} | null;
|
||||
|
||||
if (!body || typeof body.canonical_id !== 'string' || body.canonical_id.trim() === '') {
|
||||
return c.json({ ok: false, error: 'canonical_id 必填' }, 400);
|
||||
}
|
||||
if (typeof body.success !== 'boolean') {
|
||||
return c.json({ ok: false, error: 'success 必須為 boolean' }, 400);
|
||||
}
|
||||
|
||||
const result = await recordAnalytics({
|
||||
canonical_id: body.canonical_id.trim(),
|
||||
version: typeof body.version === 'string' && body.version !== '' ? body.version : undefined,
|
||||
success: body.success,
|
||||
duration_ms: typeof body.duration_ms === 'number' ? body.duration_ms : 0,
|
||||
}, c.env);
|
||||
|
||||
if (!result.ok) return c.json(result, 404);
|
||||
return c.json(result);
|
||||
});
|
||||
|
||||
export default app;
|
||||
@@ -0,0 +1,99 @@
|
||||
// 單元測試:recordAnalytics — 執行統計回寫
|
||||
// SDD: system-dev/docs/3-specs/arcrun-core-mvp/design.md「執行統計設計」
|
||||
|
||||
import { describe, it, expect, beforeEach } from 'vitest';
|
||||
import { recordAnalytics } from '../src/actions/recordAnalytics';
|
||||
import type { Bindings } from '../src/types';
|
||||
|
||||
// 最小 KV mock(get/put/list,In-memory)
|
||||
function makeKv() {
|
||||
const store = new Map<string, string>();
|
||||
return {
|
||||
store,
|
||||
async get(key: string) { return store.get(key) ?? null; },
|
||||
async put(key: string, value: string) { store.set(key, value); },
|
||||
async list({ prefix }: { prefix: string }) {
|
||||
return { keys: [...store.keys()].filter(k => k.startsWith(prefix)).map(name => ({ name })) };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe('recordAnalytics', () => {
|
||||
let submissions: ReturnType<typeof makeKv>;
|
||||
let analytics: ReturnType<typeof makeKv>;
|
||||
let env: Bindings;
|
||||
|
||||
beforeEach(() => {
|
||||
submissions = makeKv();
|
||||
analytics = makeKv();
|
||||
env = { SUBMISSIONS_KV: submissions, ANALYTICS_KV: analytics } as unknown as Bindings;
|
||||
|
||||
// 種一顆零件(indexOnlyComponent 的記錄形狀)
|
||||
submissions.store.set('idx:http_request', 'cmp_abc12345');
|
||||
submissions.store.set('comp:cmp_abc12345:v1', JSON.stringify({
|
||||
component_hash_id: 'cmp_abc12345',
|
||||
canonical_id: 'http_request',
|
||||
display_name: 'HTTP Request',
|
||||
version: 'v1',
|
||||
success_rate: 1,
|
||||
avg_duration_ms: 0,
|
||||
call_count: 0,
|
||||
visibility: 'public',
|
||||
status: 'active',
|
||||
}));
|
||||
});
|
||||
|
||||
it('N 次成功+M 次失敗 → success_rate = success_runs / total_runs,計數器與 comp 記錄同步', async () => {
|
||||
// 3 成功 + 2 失敗
|
||||
for (const success of [true, true, true, false, false]) {
|
||||
const r = await recordAnalytics({ canonical_id: 'http_request', success, duration_ms: 100 }, env);
|
||||
expect(r.ok).toBe(true);
|
||||
}
|
||||
|
||||
// 真相源:ANALYTICS_KV 計數器
|
||||
const counters = JSON.parse(analytics.store.get('stats:cmp_abc12345:v1')!);
|
||||
expect(counters).toEqual({ total_runs: 5, success_runs: 3, total_ms: 500 });
|
||||
|
||||
// 讀取端:comp 記錄被回填衍生值
|
||||
const record = JSON.parse(submissions.store.get('comp:cmp_abc12345:v1')!);
|
||||
expect(record.success_rate).toBeCloseTo(3 / 5);
|
||||
expect(record.avg_duration_ms).toBe(100);
|
||||
expect(record.call_count).toBe(5);
|
||||
});
|
||||
|
||||
it('接受 cmp_ hash_id 直接記錄', async () => {
|
||||
const r = await recordAnalytics({ canonical_id: 'cmp_abc12345', success: true, duration_ms: 50 }, env);
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.canonical_id).toBe('http_request');
|
||||
expect(r.total_runs).toBe(1);
|
||||
});
|
||||
|
||||
it('指定 version 時記到該版本', async () => {
|
||||
submissions.store.set('comp:cmp_abc12345:v2', JSON.stringify({
|
||||
component_hash_id: 'cmp_abc12345',
|
||||
canonical_id: 'http_request',
|
||||
version: 'v2',
|
||||
success_rate: 1, avg_duration_ms: 0, call_count: 0,
|
||||
visibility: 'public', status: 'active',
|
||||
}));
|
||||
const r = await recordAnalytics({ canonical_id: 'http_request', version: 'v2', success: false, duration_ms: 30 }, env);
|
||||
expect(r.ok).toBe(true);
|
||||
expect(r.version).toBe('v2');
|
||||
expect(analytics.store.has('stats:cmp_abc12345:v2')).toBe(true);
|
||||
expect(analytics.store.has('stats:cmp_abc12345:v1')).toBe(false);
|
||||
});
|
||||
|
||||
it('不在索引的零件回 ok:false(誠實 404,不假綠)', async () => {
|
||||
const r = await recordAnalytics({ canonical_id: 'no_such_component', success: true, duration_ms: 1 }, env);
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.error).toContain('不在索引');
|
||||
});
|
||||
|
||||
it('tombstone 版本不被記錄', async () => {
|
||||
submissions.store.set('comp:cmp_abc12345:v1', JSON.stringify({
|
||||
component_hash_id: 'cmp_abc12345', canonical_id: 'http_request', version: 'v1', status: 'tombstone',
|
||||
}));
|
||||
const r = await recordAnalytics({ canonical_id: 'http_request', success: true, duration_ms: 1 }, env);
|
||||
expect(r.ok).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -171,9 +171,13 @@
|
||||
- [x] 28.3 `/webhooks/:token/trigger` 路由已補上 `waitUntil(writeExecutionVerdict(...))`
|
||||
- _Requirements: 7.2_
|
||||
|
||||
- [ ] 29. registry Worker analytics 端點
|
||||
- [ ] 29.1 新增 `POST /analytics/record` 路由,原子更新 `ANALYTICS_KV`
|
||||
- [ ] 29.2 `GET /components` 回傳加入 `total_runs`、`success_rate`、`avg_duration_ms`
|
||||
- [x] 29. registry Worker analytics 端點(2026-07-31,CP arcrun-usable 步驟 6)
|
||||
- [x] 29.1 新增 `POST /analytics/record` 路由,更新 `ANALYTICS_KV` 計數器(`stats:{hash_id}:{version}`)
|
||||
- 註:KV 無 CAS,「原子更新」在 KV 上做不到——read-modify-write,併發偶有丟計數,統計用途可接受(誠實限制)
|
||||
- 衍生值(success_rate 0..1/avg_duration_ms/call_count)回填 `comp:` 記錄(查詢讀取端),計數器是唯一真相源
|
||||
- [x] 29.2 `GET /components/:id`、`/components/search` 回傳的 `success_rate`、`avg_duration_ms`、`call_count` 隨執行更新(欄位既存,本次讓它有真資料)
|
||||
- [x] 29.3 cypher-executor 執行收尾對用到的**每顆零件**回寫(`execution-evaluator.ts` 從 stub 改真實作;`/cypher/execute` 與 webhook 路徑都掛,waitUntil fire-and-forget;成敗判定=trace error 或 output.success===false)
|
||||
- 實測(本地 wrangler dev ×2,registry 8788+cypher 8787+REGISTRY_BASE_URL 指本地):同一零件 http_request 跑 5 次(3 成功+2 失敗 404)→ `GET /components/http_request` success_rate 1→0.6、call_count 0→5;`/cypher/search` 節點回 `success_rate: 0.6`
|
||||
- _Requirements: 7.3, 7.6_
|
||||
|
||||
- [x] 30. `author` 欄位已加入 contract.yaml 規格
|
||||
|
||||
Reference in New Issue
Block a user