docs(arcrun): SDD llm-interface — AI 操盤手使用體驗 first-class
設計動機:3 天 mira dogfood 累積 14 個痛點,7 個純粹是 LI 缺失。
arcrun 過去設計集中在「人」(u6u-gui / docs),AI 對 arcrun 的可用性
沒被當第一公民。
SDD 三件套(matrix/arcrun/.agents/specs/llm-interface/):
requirements.md
- personas(Claude Code 主力 / 用戶私人 agent / SDK 使用者)
- 範圍涵蓋 5 系統(cypher-executor / registry / u6u-mcp / u6u-gui / kbdb)
- 10 個 FR:onboarding / CRUD 對等 / dry-run / 結構化 trace /
可程式化 error / feedback tool / implicit telemetry /
skill blocks / examples / weekly closed loop
- 5 個 NFR:相容 / 多 transport / error contract 穩定 /
feedback exportable / coverage 量化
design.md
- 5 層 LI 模型:AGENTS.md / arcrun-mcp / Skills / Examples / Telemetry
- 25 個 MCP tool 完整清單分 5 類
- error_code enum v1
- coverage matrix(GUI 動作 vs MCP / 31 cypher-executor 路由 vs LI)
- 完整 AGENTS.md 模板
- u6u-mcp → arcrun-mcp migration plan(90 天 deprecation)
- weekly_review workflow YAML 範本
tasks.md
- 5 個 milestone(M1 收 data / M2 gap-fill / M3 skill+examples /
M4 closed loop / M5 rename)
- 估算 23 個工作日 (~5 週)
- M1 是硬前置(不收 data 改了也不知道對沒)
Audit 基準(用 4 個並行 Explore agent 整理):
- cypher-executor: 31 HTTP 路由,9 個 AI-essential
- u6u-mcp: 15 tool,缺 update/delete/history/validate/feedback
- u6u-gui: 8 個人類動作可對等 LI / 3 個視覺類不需
- kbdb: 50 路由 13 group,LI 走 abstracted tool 不直接 expose
同步更新 .claude/rules/04-current-progress.md SDD 索引。
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,616 @@
|
|||||||
|
# Design: LI (LLM Interface) for arcrun
|
||||||
|
|
||||||
|
> v0.1 — 2026-05-16
|
||||||
|
> 對應 requirements.md(同目錄)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 設計哲學
|
||||||
|
|
||||||
|
| 過去 (UI 時代) | 現在 (LI 時代) |
|
||||||
|
|---|---|
|
||||||
|
| 使用者是人,要學軟體 | 使用者是 AI,要被軟體 onboard |
|
||||||
|
| 操作靠視覺、滑鼠 | 操作靠 MCP tool call |
|
||||||
|
| 錯誤訊息是技術 stack trace | 錯誤訊息是「下一步該做什麼」 |
|
||||||
|
| 文件是 long-form 教學 | 文件是結構化 schema + 可程式查 |
|
||||||
|
| 用戶教學是 onboarding 課程 | 「教學」是 MCP 工具自己會回 hint |
|
||||||
|
| 回饋靠 helpdesk 工單 | 回饋是 MCP tool(AI 直接 call) |
|
||||||
|
|
||||||
|
**原則**:
|
||||||
|
1. **discoverable** — AI 不靠 grep / 不靠人就能知道有什麼
|
||||||
|
2. **idempotent** — 同樣輸入兩次結果一樣,可預測
|
||||||
|
3. **dry-run by default** — preview 是預設、commit 是 explicit
|
||||||
|
4. **structured errors** — error 是 JSON 含 next_actions,不是字串
|
||||||
|
5. **closed loop** — AI 卡住的 data 自動回流,平台 self-improving
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 系統架構
|
||||||
|
|
||||||
|
```
|
||||||
|
┌────────────────────────────────────────────────────────────────┐
|
||||||
|
│ AI agent (Claude / Cursor / ...) │
|
||||||
|
└──────────────────┬──────────────────────────────┬───────────────┘
|
||||||
|
│ │
|
||||||
|
│ ① Read AGENTS.md │ ② MCP tool calls
|
||||||
|
▼ ▼
|
||||||
|
┌────────────────────────┐ ┌──────────────────────────────────┐
|
||||||
|
│ AGENTS.md │ │ arcrun-mcp (擴張 u6u-mcp) │
|
||||||
|
│ (repo + KBDB block) │ │ 25 tools, 5 categories │
|
||||||
|
└────────────────────────┘ └──────────────┬───────────────────┘
|
||||||
|
│
|
||||||
|
┌────────┬─────────┬────────┴────────┬──────────┐
|
||||||
|
▼ ▼ ▼ ▼ ▼
|
||||||
|
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌──────────┐
|
||||||
|
│ cypher- │ │registry │ │ kbdb │ │ Skills │ │ Examples │
|
||||||
|
│executor │ │ │ │ │ │ (KBDB) │ │ (git+KV) │
|
||||||
|
│ 31 路由 │ │ │ │ 50 路由 │ │ │ │ │
|
||||||
|
└─────────┘ └─────────┘ └─────────┘ └─────────┘ └──────────┘
|
||||||
|
▲
|
||||||
|
│ ③ AI 部署完 / 卡住 call feedback
|
||||||
|
│
|
||||||
|
┌─────────────┴──────────────┐
|
||||||
|
│ report_feedback / agent-telemetry │
|
||||||
|
│ (寫進 KBDB) │
|
||||||
|
└─────────────┬──────────────┘
|
||||||
|
│ ④ 每週聚合
|
||||||
|
▼
|
||||||
|
┌──────────────────────────────┐
|
||||||
|
│ agent_feedback_weekly_review │
|
||||||
|
│ (arcrun workflow, dogfood) │
|
||||||
|
│ → arcrun-roadmap KBDB block │
|
||||||
|
└──────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 五層 LI 模型
|
||||||
|
|
||||||
|
### Layer 0:Onboarding(AGENTS.md)
|
||||||
|
|
||||||
|
**位置**:
|
||||||
|
- `arcrun/AGENTS.md`(repo 根,git 可讀)
|
||||||
|
- KBDB `type=agent-onboarding` block(MCP `get_onboarding_doc()` 拉)
|
||||||
|
|
||||||
|
**結構(控 5-8K tokens)**:
|
||||||
|
1. **30 字 What** — 「arcrun 是用 YAML 把零件串成 workflow 的平台」
|
||||||
|
2. **5 個核心概念** — component / cypher binding YAML / FOREACH / paused-resume / api_key
|
||||||
|
3. **連線**:「在你的 MCP config 加 `https://mcp.arcrun.dev/mcp`,bearer `ak_xxx`」
|
||||||
|
4. **第一個 workflow(minimal example)** — 10 行 YAML,含部署 + 觸發
|
||||||
|
5. **5 個 URL 慣例與為什麼**(workers.dev vs arcrun.dev / KBDB endpoints)
|
||||||
|
6. **常見錯誤 5 個 + 怎麼讀**
|
||||||
|
7. **不確定的時候**:明確列「先 call list_X 再下手」
|
||||||
|
8. **回報機制**:完成任何 workflow deploy / 卡住,**必須** call `report_feedback`
|
||||||
|
|
||||||
|
**部署機制**:
|
||||||
|
- repo `arcrun/AGENTS.md` 是 source of truth
|
||||||
|
- GH Actions 監聽該檔變動 → 自動 PATCH KBDB onboarding block
|
||||||
|
- MCP `get_onboarding_doc()` 從 KBDB 讀(最新)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Layer 1:arcrun-mcp(主戰場)
|
||||||
|
|
||||||
|
#### 1.1 命名與部署
|
||||||
|
|
||||||
|
- **Server URL**: `https://mcp.arcrun.dev/mcp`(新增 route,對應 `mcp.finally.click` 既有 worker)
|
||||||
|
- **過渡**:u6u-mcp Worker rename `arcrun-mcp`,雙路由(finally.click + arcrun.dev)並存 90 天
|
||||||
|
- **Tool prefix**: 統一 `arcrun_*`(既有 `u6u_*` 加 alias 對應,舊名 deprecation 公告)
|
||||||
|
|
||||||
|
#### 1.2 Tool 完整清單(25 個,5 類)
|
||||||
|
|
||||||
|
##### A. Onboarding & Discovery(5 tools)
|
||||||
|
|
||||||
|
| Tool | 對應 HTTP | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `arcrun_get_onboarding` | KBDB GET `/blocks?type=agent-onboarding` | 拉 AGENTS.md 內容 |
|
||||||
|
| `arcrun_list_components` | registry GET `/components` | 所有可用零件 + 1 行 desc |
|
||||||
|
| `arcrun_get_component_contract` | registry GET `/components/:id` | input/output schema + gherkin tests |
|
||||||
|
| `arcrun_list_recipes` | cypher GET `/recipes` + `/auth-recipes` | API recipe 一覽 |
|
||||||
|
| `arcrun_search_examples` | KBDB POST `/search`(type=workflow-example)| 用 use case 搜範例 |
|
||||||
|
|
||||||
|
##### B. Workflow CRUD(7 tools)
|
||||||
|
|
||||||
|
| Tool | 對應 HTTP | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `arcrun_list_workflows` | cypher GET `/webhooks/named` | 用戶現有 workflow |
|
||||||
|
| `arcrun_get_workflow` | cypher GET `/webhooks/named/:name` | 拿 YAML / graph |
|
||||||
|
| `arcrun_validate_yaml` | cypher POST `/validate` | dry-run 校驗,不部署 |
|
||||||
|
| `arcrun_push_workflow` | cypher POST `/webhooks/named` | 部署(也含 validate) |
|
||||||
|
| `arcrun_delete_workflow` | cypher DELETE `/webhooks/named/:name` | 刪 |
|
||||||
|
| `arcrun_preview_workflow` | **新增** POST `/preview` | 不寫 KV、模擬 execute |
|
||||||
|
| `arcrun_diff_workflow` | **新增** POST `/webhooks/named/:name/diff` | 既有 vs 新 YAML 差異 |
|
||||||
|
|
||||||
|
##### C. Execution & Trace(5 tools)
|
||||||
|
|
||||||
|
| Tool | 對應 HTTP | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `arcrun_run_workflow` | cypher POST `/webhooks/named/:name/trigger` | 觸發 + 回 execution_id |
|
||||||
|
| `arcrun_get_execution_trace` | **新增** GET `/executions/:id` | 結構化 trace(per-node status) |
|
||||||
|
| `arcrun_list_recent_executions` | **新增** GET `/workflows/:name/executions?limit=10` | 最近 N 次 |
|
||||||
|
| `arcrun_list_paused_executions` | **新增** GET `/executions/paused` | 卡 callback 的 |
|
||||||
|
| `arcrun_resume_execution` | cypher POST `/workflows/resume` | 手動 resume |
|
||||||
|
|
||||||
|
##### D. Component & Recipe Management(4 tools)
|
||||||
|
|
||||||
|
| Tool | 對應 HTTP | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `arcrun_search_components` | registry POST search | 語意搜尋 |
|
||||||
|
| `arcrun_publish_component` | registry POST publish | 上 WASM |
|
||||||
|
| `arcrun_create_recipe` | cypher POST `/recipes` | 建 API recipe |
|
||||||
|
| `arcrun_create_auth_recipe` | cypher POST `/auth-recipes` | 建 auth recipe |
|
||||||
|
|
||||||
|
##### E. Feedback & Skills(4 tools)
|
||||||
|
|
||||||
|
| Tool | 對應 HTTP | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `arcrun_report_feedback` | KBDB POST `/blocks`(type=agent-feedback)| AI 主動回報 |
|
||||||
|
| `arcrun_list_skills` | KBDB GET `/blocks?type=agent-skill` | 列可用 playbook |
|
||||||
|
| `arcrun_get_skill` | KBDB GET `/blocks/:id` | 拿 skill 內容 |
|
||||||
|
| `arcrun_publish_skill` | KBDB POST `/blocks`(type=agent-skill)| AI 把學到的存回去 |
|
||||||
|
|
||||||
|
#### 1.3 Tool contract 統一規範
|
||||||
|
|
||||||
|
每個 tool 都遵守:
|
||||||
|
|
||||||
|
**Input**:
|
||||||
|
- Zod schema declarative
|
||||||
|
- 每個參數有 `.describe('...')`(給 AI 看的)
|
||||||
|
- optional 標 default
|
||||||
|
|
||||||
|
**Output(成功)**:
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
ok: true,
|
||||||
|
data: T,
|
||||||
|
hints?: string[], // optional next-step suggestion,e.g.「你剛部署了,可 call run_workflow 測試」
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Output(失敗)**:
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
ok: false,
|
||||||
|
error_code: 'enum_value',
|
||||||
|
human_message: string,
|
||||||
|
next_actions: string[], // 可程式化的下一步
|
||||||
|
detail?: unknown, // 原始錯誤(debug)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1.4 error_code enum(v1,可加不可刪)
|
||||||
|
|
||||||
|
| error_code | 對應狀況 |
|
||||||
|
|---|---|
|
||||||
|
| `auth_missing` | api_key 沒帶 |
|
||||||
|
| `auth_invalid` | api_key 無效 |
|
||||||
|
| `not_found` | workflow / block 不存在 |
|
||||||
|
| `validation_failed` | YAML / schema 不過 |
|
||||||
|
| `component_not_found` | 零件 ID 沒對應 |
|
||||||
|
| `component_not_in_whitelist` | 零件存在但 cypher-executor 不認 |
|
||||||
|
| `paused_awaiting_resume` | workflow 在等 callback |
|
||||||
|
| `rate_limited` | 太頻繁 |
|
||||||
|
| `internal_error` | 平台 bug(含 detail) |
|
||||||
|
| `dependency_unavailable` | 下游服務(claude / mira daemon / KBDB)掛 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Layer 2:Skill blocks(KBDB-native playbook)
|
||||||
|
|
||||||
|
#### 2.1 Schema
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type AgentSkill = {
|
||||||
|
id: string;
|
||||||
|
type: 'agent-skill';
|
||||||
|
page_name: `skill-${slug}`;
|
||||||
|
content: string; // markdown playbook
|
||||||
|
tags_json: string[]; // ['watcher', 'debug', 'rag', ...]
|
||||||
|
source: 'manual' | 'auto-extracted';
|
||||||
|
metadata_json: {
|
||||||
|
when_to_use: string;
|
||||||
|
example_use_case: string;
|
||||||
|
contributed_by?: string; // agent user_agent
|
||||||
|
success_count?: number;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2.2 種子 skill(先建 5 個,以 mira 經驗為本)
|
||||||
|
|
||||||
|
| skill page_name | 用途 |
|
||||||
|
|---|---|
|
||||||
|
| `skill-build_watcher_workflow` | cron 掃資料 → 觸發處理(mira_feed_watcher pattern) |
|
||||||
|
| `skill-debug_paused_workflow` | claude_api / 任何 async callback paused 怎麼追 |
|
||||||
|
| `skill-migrate_http_to_trigger_workflow` | self-fetch 換 trigger_workflow(剛踩過的) |
|
||||||
|
| `skill-rag_with_arcrun` | 用 KBDB + claude_api 做 RAG |
|
||||||
|
| `skill-add_new_wasm_component` | 從 TinyGo 寫到 deploy 全流程 |
|
||||||
|
|
||||||
|
#### 2.3 自動萃取機制(後期)
|
||||||
|
|
||||||
|
`agent_feedback_weekly_review` 跑出來的 Top patterns,LLM 包成 skill draft,leo review 後 publish。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Layer 3:Examples(可搜尋的範例庫)
|
||||||
|
|
||||||
|
#### 3.1 存放
|
||||||
|
|
||||||
|
- `arcrun/registry/examples/{slug}/`
|
||||||
|
- `workflow.yaml`
|
||||||
|
- `description.md`(解決什麼問題、怎麼 trigger、預期結果)
|
||||||
|
- `tags.json`(`["webhook", "llm", "cron", ...]`)
|
||||||
|
- CI build 出 `examples-index.json` 推 KBDB(type=workflow-example,內容是 YAML + tags)
|
||||||
|
|
||||||
|
#### 3.2 搜尋
|
||||||
|
|
||||||
|
- `arcrun_search_examples(use_case)` 走 KBDB `/search`(既有 semantic)
|
||||||
|
- 命中 → 回 YAML + description + 「你可以基於這個改 X」hint
|
||||||
|
|
||||||
|
#### 3.3 第一批範例(10 個)
|
||||||
|
|
||||||
|
| slug | 用途 |
|
||||||
|
|---|---|
|
||||||
|
| `webhook-to-slack` | 簡單轉發 |
|
||||||
|
| `cron-watcher` | mira_feed_watcher 簡化版 |
|
||||||
|
| `llm-classify` | claude_api 分類 |
|
||||||
|
| `rag-search-answer` | 從 KBDB 找 context → 回答 |
|
||||||
|
| `email-summary` | gmail → claude → telegram |
|
||||||
|
| `pdf-to-blocks` | convert → ingest |
|
||||||
|
| `github-issue-bot` | webhook → claude → 留 comment |
|
||||||
|
| `daily-digest` | cron → 多源聚合 → 推送 |
|
||||||
|
| `parallel-fanout` | 一份輸入分發多 workflow |
|
||||||
|
| `error-retry` | try_catch + wait + retry |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Layer 4:Telemetry & Feedback Loop
|
||||||
|
|
||||||
|
#### 4.1 兩條線
|
||||||
|
|
||||||
|
```
|
||||||
|
explicit feedback (AI call) implicit telemetry (platform auto-log)
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
KBDB block type= KBDB block type=
|
||||||
|
agent-feedback agent-telemetry
|
||||||
|
│ │
|
||||||
|
└──────────────┬───────────────────────┘
|
||||||
|
▼
|
||||||
|
agent_feedback_weekly_review
|
||||||
|
(arcrun workflow, 週一 9am cron)
|
||||||
|
▼
|
||||||
|
LLM 聚合 + 分類
|
||||||
|
▼
|
||||||
|
KBDB block type=arcrun-roadmap
|
||||||
|
(leo 收到 + 寫進 mira 河道)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4.2 agent-feedback schema
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type AgentFeedback = {
|
||||||
|
type: 'agent-feedback';
|
||||||
|
content: string; // description 原文
|
||||||
|
source: 'mcp-tool-call';
|
||||||
|
user_id: string; // 用戶 namespace
|
||||||
|
metadata_json: {
|
||||||
|
workflow_name?: string;
|
||||||
|
issue_type: 'doc_unclear' | 'tool_missing' | 'error_unhelpful'
|
||||||
|
| 'unexpected_behavior' | 'feature_request' | 'success_story';
|
||||||
|
retry_count?: number;
|
||||||
|
blocked: boolean;
|
||||||
|
suggested_fix?: string;
|
||||||
|
agent_user_agent: string; // 'claude-code/1.x' etc
|
||||||
|
};
|
||||||
|
tags_json: ['agent-feedback', `issue:${issue_type}`];
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4.3 agent-telemetry schema
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type AgentTelemetry = {
|
||||||
|
type: 'agent-telemetry';
|
||||||
|
source: 'cypher-executor';
|
||||||
|
metadata_json: {
|
||||||
|
event_type: 'deploy_success' | 'deploy_fail' | 'run_success'
|
||||||
|
| 'run_fail' | 'validation_error' | 'mcp_tool_call';
|
||||||
|
workflow_name?: string;
|
||||||
|
component_id?: string;
|
||||||
|
error_code?: string;
|
||||||
|
duration_ms: number;
|
||||||
|
api_key_hash: string; // 雜湊,不存原值
|
||||||
|
agent_user_agent: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
寫入點:
|
||||||
|
- `webhook-handlers.executeWebhookGraph` 完成(成功 / 失敗都記)
|
||||||
|
- `routes/webhooks-named.ts` push(deploy 記)
|
||||||
|
- `routes/cypher.ts` validate 失敗(validation_error 記)
|
||||||
|
- arcrun-mcp 每個 tool call wrap 記(mcp_tool_call)
|
||||||
|
|
||||||
|
#### 4.4 隱私
|
||||||
|
|
||||||
|
- api_key 一律 SHA-256 截前 16 字元(不可逆,可聚合)
|
||||||
|
- workflow content 不 log(只記 name)
|
||||||
|
- 用戶看自己的 telemetry:MCP `arcrun_my_telemetry(limit)`,只回自己 hash
|
||||||
|
|
||||||
|
#### 4.5 weekly review workflow
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: agent_feedback_weekly_review
|
||||||
|
description: 每週一聚合 AI 回饋 + telemetry,產出 Top 痛點
|
||||||
|
|
||||||
|
flow:
|
||||||
|
- "weekly_cron >> ON_SUCCESS >> fetch_feedback"
|
||||||
|
- "fetch_feedback >> ON_SUCCESS >> fetch_telemetry"
|
||||||
|
- "fetch_telemetry >> ON_SUCCESS >> aggregate"
|
||||||
|
- "aggregate >> ON_SUCCESS >> llm_summarize"
|
||||||
|
- "llm_summarize >> ON_SUCCESS >> publish_roadmap_block"
|
||||||
|
- "publish_roadmap_block >> ON_SUCCESS >> notify_telegram"
|
||||||
|
|
||||||
|
config:
|
||||||
|
weekly_cron:
|
||||||
|
component: cron
|
||||||
|
cron_expr: "0 9 * * 1" # 週一 9 am UTC
|
||||||
|
fetch_feedback:
|
||||||
|
component: kbdb_get
|
||||||
|
type: 'agent-feedback'
|
||||||
|
created_after: "{{ now - 7d }}"
|
||||||
|
limit: 500
|
||||||
|
fetch_telemetry:
|
||||||
|
component: kbdb_get
|
||||||
|
type: 'agent-telemetry'
|
||||||
|
created_after: "{{ now - 7d }}"
|
||||||
|
limit: 5000
|
||||||
|
aggregate:
|
||||||
|
component: filter # 或寫個 `group_by` 邏輯零件
|
||||||
|
items: "{{fetch_feedback.blocks}}"
|
||||||
|
group_by: 'metadata_json.issue_type'
|
||||||
|
llm_summarize:
|
||||||
|
component: claude_api
|
||||||
|
prompt: |
|
||||||
|
你是 arcrun 平台的回饋分析師。下面是這週 AI agent 的所有回饋 +
|
||||||
|
telemetry 失敗事件。請:
|
||||||
|
1. 列 Top 5 痛點(含證據引用)
|
||||||
|
2. 為每個痛點建議具體改動(文件 / MCP tool / 錯誤訊息)
|
||||||
|
3. 評估嚴重程度(blocked AI vs 略不便)
|
||||||
|
|
||||||
|
Feedback:{{aggregate}}
|
||||||
|
Telemetry 失敗事件:{{fetch_telemetry.blocks}}
|
||||||
|
publish_roadmap_block:
|
||||||
|
component: kbdb_create_block
|
||||||
|
type: 'arcrun-roadmap'
|
||||||
|
page_name: 'roadmap-week-{{date.iso_week}}'
|
||||||
|
content: "{{llm_summarize.data.text}}"
|
||||||
|
notify_telegram:
|
||||||
|
component: telegram
|
||||||
|
chat_id: "{{ leo_chat_id }}"
|
||||||
|
text: "Arcrun 週報出爐:{{publish_roadmap_block.data.id}}"
|
||||||
|
```
|
||||||
|
|
||||||
|
**這個 workflow 是 dogfood inception**:arcrun 自己跑分析 arcrun 自己。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Coverage Matrix(人類 vs AI)
|
||||||
|
|
||||||
|
> 這份表是 NFR-5「覆蓋率可量化」的具體實踐。每個 release 重新檢視。
|
||||||
|
|
||||||
|
### 4.1 人類 GUI 能做的事 × LI 對應
|
||||||
|
|
||||||
|
| 人類在 u6u-gui 做的 | LI MCP 工具 | 對等? |
|
||||||
|
|---|---|---|
|
||||||
|
| 看現有 workflow 列表 | `arcrun_list_workflows` | ✅ |
|
||||||
|
| 點開 workflow 看 YAML | `arcrun_get_workflow` | ✅ |
|
||||||
|
| 編輯 workflow YAML | `arcrun_push_workflow`(含 update) | ✅ |
|
||||||
|
| 執行 workflow | `arcrun_run_workflow` | ✅ |
|
||||||
|
| 看執行結果 | `arcrun_get_execution_trace` | ✅(新增) |
|
||||||
|
| 翻零件庫 | `arcrun_list_components` + search | ✅ |
|
||||||
|
| 看零件 contract | `arcrun_get_component_contract` | ✅ |
|
||||||
|
| 上傳 WASM 零件 | `arcrun_publish_component` | ✅ |
|
||||||
|
| 設 credential | `arcrun_create_recipe` + (手動 push credential) | ⚠️ 部分(credential 提交需安全考量,AI 不全自動) |
|
||||||
|
| 翻歷史執行 | `arcrun_list_recent_executions` | ✅(新增) |
|
||||||
|
| 看 paused 工作流 | `arcrun_list_paused_executions` | ✅(新增) |
|
||||||
|
| 手動 resume | `arcrun_resume_execution` | ✅ |
|
||||||
|
| 視覺化 canvas 拖拉 | — | ❌(純視覺,LI 不複製) |
|
||||||
|
| 看排版 prototype | — | ❌(同上) |
|
||||||
|
| Flip UI/Logic view | — | ❌(同上) |
|
||||||
|
| Action log(操作軌跡) | `arcrun_my_telemetry` | ✅(含更多資料)|
|
||||||
|
|
||||||
|
**Gap**:3 個視覺類動作 LI 不需要對等;其他全等。
|
||||||
|
|
||||||
|
### 4.2 cypher-executor 31 路由 × LI 暴露
|
||||||
|
|
||||||
|
| 路由 | LI 暴露? | 工具 |
|
||||||
|
|---|---|---|
|
||||||
|
| `/health`、`/docs`、`/openapi.json` | ❌ infra | — |
|
||||||
|
| `/execute` | ✅ | `arcrun_run_workflow`(內部) |
|
||||||
|
| `/cypher/search`、`/cypher/execute` | ✅ | discovery 內部用 |
|
||||||
|
| `/workflows/resume` | ✅ | `arcrun_resume_execution` |
|
||||||
|
| `/webhooks/named*` | ✅ | CRUD 對應 5 個 tools |
|
||||||
|
| `/webhooks/*` (anonymous) | ❌ deprecated | — |
|
||||||
|
| `/credentials*` | ⚠️ | 只 list + delete,POST 走人類流程(安全考量) |
|
||||||
|
| `/recipes*`、`/auth-recipes*` | ✅ | 4 個 tools |
|
||||||
|
| `/validate` | ✅ | `arcrun_validate_yaml` |
|
||||||
|
| `/auth/*`、`/register`、`/me*` | ❌ admin | — |
|
||||||
|
|
||||||
|
**新增需建路由**:`/preview`、`/executions/:id`、`/workflows/:name/executions`、`/executions/paused`、`/webhooks/named/:name/diff`、`/my-telemetry`(共 6 個)
|
||||||
|
|
||||||
|
### 4.3 KBDB 50 路由 × LI 暴露
|
||||||
|
|
||||||
|
LI **不直接 expose KBDB 50 個路由**。AI 透過 arcrun-mcp 的 abstracted tool(`get_skill` / `list_skills` / `report_feedback` 等)間接用 KBDB。
|
||||||
|
|
||||||
|
例外(值得直接 expose 的):
|
||||||
|
- `arcrun_kbdb_search(query)` — 走 `/search` semantic
|
||||||
|
- `arcrun_kbdb_get_block(id)` — 看 block 內容(debug 用)
|
||||||
|
|
||||||
|
KBDB schema 設計 / triplets / records 這些屬於另一個 SDD(KBDB MCP),不在本 LI 範圍。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. AGENTS.md 模板
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Arcrun for AI Agents
|
||||||
|
|
||||||
|
## What is Arcrun (30 sec)
|
||||||
|
Arcrun lets you compose Cloudflare Workers (WASM components) into workflows via YAML.
|
||||||
|
You write YAML, push to KV, trigger via webhook or cron. Each component is a
|
||||||
|
TinyGo/AssemblyScript WASM with stdin/stdout JSON I/O.
|
||||||
|
|
||||||
|
## Connect (one step)
|
||||||
|
Add to your MCP config:
|
||||||
|
\`\`\`json
|
||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"arcrun": {
|
||||||
|
"url": "https://mcp.arcrun.dev/mcp",
|
||||||
|
"headers": {"Authorization": "Bearer $ARCRUN_API_KEY"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
\`\`\`
|
||||||
|
Get your ak_ key at https://arcrun.dev/me.
|
||||||
|
|
||||||
|
## Your first workflow (5 min)
|
||||||
|
1. `arcrun_list_components()` → see what's available
|
||||||
|
2. Write 3-line YAML in chat (assistant does this)
|
||||||
|
3. `arcrun_validate_yaml(yaml)` → confirm
|
||||||
|
4. `arcrun_push_workflow(yaml)` → deploy
|
||||||
|
5. `arcrun_run_workflow(name, input)` → test
|
||||||
|
6. `arcrun_get_execution_trace(id)` → see results
|
||||||
|
|
||||||
|
## Core concepts (5 things)
|
||||||
|
- **Component**: WASM Worker. Use existing or `arcrun_publish_component`.
|
||||||
|
- **Cypher binding YAML**: triplets `A >> relation >> B` define graph.
|
||||||
|
- **FOREACH**: `>> 對每個 X >> next_node` iterates lists.
|
||||||
|
- **Paused-resume**: claude_api etc pause workflow; resumes via callback.
|
||||||
|
- **api_key**: ak_xxx, all calls need this in MCP auth.
|
||||||
|
|
||||||
|
## URL conventions (why this matters)
|
||||||
|
- `cypher.arcrun.dev` = orchestration API (you push here)
|
||||||
|
- `arcrun-{kebab}.{user}.workers.dev` = component workers (workers.dev avoids CF self-fetch)
|
||||||
|
- `kbdb-*.arcrun.dev` = KBDB component workers (your data lives here)
|
||||||
|
|
||||||
|
## Common errors
|
||||||
|
| error_code | What to do |
|
||||||
|
|---|---|
|
||||||
|
| `component_not_found` | call `list_components`, check spelling |
|
||||||
|
| `validation_failed` | read `next_actions` field, fix YAML |
|
||||||
|
| `paused_awaiting_resume` | normal for claude_api; wait or `get_execution_trace` |
|
||||||
|
| `auth_invalid` | ak_ wrong, re-fetch from /me |
|
||||||
|
|
||||||
|
## Unsure? Do this:
|
||||||
|
- Don't know what's available → `list_components` / `list_workflows` / `search_examples`
|
||||||
|
- Don't know syntax → `get_skill('skill-build_X')` for playbooks
|
||||||
|
- Hit weird behavior → **MUST** call `report_feedback(issue_type='unexpected_behavior', ...)`
|
||||||
|
|
||||||
|
## Feedback (required)
|
||||||
|
After every workflow deploy or debug session, call:
|
||||||
|
\`\`\`
|
||||||
|
arcrun_report_feedback({
|
||||||
|
workflow_name: '...',
|
||||||
|
issue_type: 'success_story' | 'doc_unclear' | 'tool_missing' | 'error_unhelpful' | 'feature_request',
|
||||||
|
description: '...',
|
||||||
|
blocked: false,
|
||||||
|
retry_count: N,
|
||||||
|
suggested_fix: '...' // optional
|
||||||
|
})
|
||||||
|
\`\`\`
|
||||||
|
|
||||||
|
Even "success" stories help — they tell us what's working.
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Migration plan(u6u-mcp → arcrun-mcp)
|
||||||
|
|
||||||
|
### 6.1 不破壞
|
||||||
|
|
||||||
|
- u6u-mcp Worker rename `arcrun-mcp`(wrangler name 改)
|
||||||
|
- 兩個 hostname 並存:`mcp.finally.click` + `mcp.arcrun.dev`(route 加一條)
|
||||||
|
- 舊 tool 名(`u6u_*`)保留,加 alias 對應新名(`arcrun_*`)
|
||||||
|
- 90 天 deprecation 期,舊 tool call → 回 warning hint「`u6u_X` → 改用 `arcrun_X`」
|
||||||
|
|
||||||
|
### 6.2 新增 tools(按 phase)
|
||||||
|
|
||||||
|
Phase 1(gap-fill):
|
||||||
|
- `arcrun_validate_yaml`(既有 `/validate` 已存,包 MCP 即可)
|
||||||
|
- `arcrun_get_execution_trace`(需 cypher-executor 新加 `/executions/:id`)
|
||||||
|
- `arcrun_list_recent_executions`(新 endpoint)
|
||||||
|
- `arcrun_list_paused_executions`(新 endpoint)
|
||||||
|
- `arcrun_report_feedback`(新 tool)
|
||||||
|
- `arcrun_get_onboarding`(KBDB read)
|
||||||
|
|
||||||
|
Phase 2(advanced):
|
||||||
|
- `arcrun_preview_workflow`(新 endpoint,沒 KV side-effect)
|
||||||
|
- `arcrun_diff_workflow`
|
||||||
|
- `arcrun_my_telemetry`
|
||||||
|
- `arcrun_search_examples`(KBDB write + search 已存)
|
||||||
|
- `arcrun_list_skills` / `arcrun_get_skill` / `arcrun_publish_skill`
|
||||||
|
|
||||||
|
Phase 3(auto-loop):
|
||||||
|
- `agent_feedback_weekly_review` workflow
|
||||||
|
- `arcrun-roadmap` block 生成
|
||||||
|
- LLM extract skill 自動化
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 開發順序與里程碑
|
||||||
|
|
||||||
|
### Milestone 1:可量測(1 週)
|
||||||
|
- 寫 AGENTS.md v1
|
||||||
|
- 加 implicit telemetry 寫入點(cypher-executor)
|
||||||
|
- 加 `arcrun_report_feedback` MCP tool
|
||||||
|
- 建 `agent-feedback` / `agent-telemetry` KBDB template
|
||||||
|
- 開始收 data
|
||||||
|
|
||||||
|
### Milestone 2:gap-fill(1 週)
|
||||||
|
- u6u-mcp 補上 6 個 Phase 1 tools
|
||||||
|
- 新 endpoints 在 cypher-executor 加
|
||||||
|
- 每個 tool 結構化 error contract
|
||||||
|
|
||||||
|
### Milestone 3:skill + example(1 週)
|
||||||
|
- 種子 5 個 skill blocks
|
||||||
|
- 種子 10 個 example workflows
|
||||||
|
- `search_examples` 跑通
|
||||||
|
|
||||||
|
### Milestone 4:closed loop(半週)
|
||||||
|
- weekly_review workflow 部署
|
||||||
|
- 第一份 arcrun-roadmap block 產出
|
||||||
|
- leo 收到第一份週報
|
||||||
|
|
||||||
|
### Milestone 5:rename + cleanup(1 週)
|
||||||
|
- u6u-mcp → arcrun-mcp 公開
|
||||||
|
- 舊 tool 加 deprecation warning
|
||||||
|
- AGENTS.md 同步 KBDB
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 開放決策(待 leo)
|
||||||
|
|
||||||
|
| 決策 | 推薦 | 理由 |
|
||||||
|
|---|---|---|
|
||||||
|
| MCP server URL | `mcp.arcrun.dev`(route 並存 `mcp.finally.click` 90 天) | 品牌統一 |
|
||||||
|
| 擴張 u6u-mcp vs 建新 | 擴張 + rename | 不 fork,零移轉痛 |
|
||||||
|
| AGENTS.md 位置 | `arcrun/AGENTS.md` + KBDB 自動同步 | 兩面都拿到 |
|
||||||
|
| feedback 是否要 auth | 要(驗 ak_ 存在即可,不查餘額) | 防 spam |
|
||||||
|
| LI 是否包含 KBDB MCP | 部分(abstracted),KBDB MCP 另立 SDD | 範圍清楚 |
|
||||||
|
| Telemetry 保留多久 | 90 天 hot + 1 年 cold archive | 平衡 cost 與 review |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 跨 SDD 連動
|
||||||
|
|
||||||
|
| 其他 SDD | 連動點 |
|
||||||
|
|---|---|
|
||||||
|
| `credential-primitives-wasm` | LI 不重做 auth,只用 `auth_static_key` 等既有零件 |
|
||||||
|
| `recipe-system` | LI 暴露 recipe CRUD tool 對應 |
|
||||||
|
| `component-registry-canon` | LI `list_components` / `get_contract` 走 registry |
|
||||||
|
| `resumable-workflow` | LI `list_paused_executions` / `resume` 用 |
|
||||||
|
| `u6u-platform-evolution` | LI 是 evolution 之一,未來分 user-tier 時要考慮 |
|
||||||
|
| **mira-app** (polaris/mira) | LI 是 mira dogfood 痛點轉化的產物,roadmap 含對 mira 的回饋 |
|
||||||
@@ -0,0 +1,242 @@
|
|||||||
|
# Requirements: LI (LLM Interface) for arcrun
|
||||||
|
|
||||||
|
> 把 arcrun 平台對「AI 操盤手」的完整使用面,當成 first-class product 設計。
|
||||||
|
> 對比:以前做網站要 user-friendly(UI),現在 AI 是主要用戶(LI)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
arcrun 是 n8n-like workflow 平台,平台本身的 end-user 有兩類:
|
||||||
|
- **人類**:透過 u6u-gui canvas / arcrun.dev landing 操作
|
||||||
|
- **AI agent**:透過 MCP / API 操作(Claude Code、Cursor、Codex、自製 agent 等)
|
||||||
|
|
||||||
|
過去設計集中在「人」(UI / docs)。AI 對 arcrun 的「可用性」沒被當第一公民。
|
||||||
|
|
||||||
|
### 證據:3 天 mira dogfood 累積的 14 個痛點(baseline)
|
||||||
|
|
||||||
|
| 編號 | 痛點 | 性質 |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | 新零件 deploy 要手動 dashboard 點 workers.dev | platform DX |
|
||||||
|
| 2 | CF 同 zone self-fetch 死鎖 | platform infra |
|
||||||
|
| 3 | `http_request` 4xx 回 success=true cascade | error semantics |
|
||||||
|
| 4 | interpolateString array stringified | dev experience |
|
||||||
|
| 5 | `kbdb_upsert_block` 沒在 `WASM_HTTP_RUNNER_IDS` 白名單 | discovery |
|
||||||
|
| 6 | `resumeFromPaused` paused_node_id 沒 namespace | implicit behavior |
|
||||||
|
| 7 | `acr validate` 不認「對每個 X」 | validator stale |
|
||||||
|
| 8 | cron 從零寫起 | feature gap |
|
||||||
|
| 9 | CF Pages 沒接 auto-deploy | infra setup |
|
||||||
|
| 10 | watcher self-fetch 死鎖(剛解) | architecture |
|
||||||
|
| 11 | skill 改要寫 python script | tooling gap |
|
||||||
|
| 12 | 不知道有什麼零件可用 | **LI gap** |
|
||||||
|
| 13 | 不知道現有 workflow 怎麼長 | **LI gap** |
|
||||||
|
| 14 | trace / log 看不到結構化的 | **LI gap** |
|
||||||
|
|
||||||
|
**14 個裡有 7 個(編號 3-7, 12-14)純粹是 LI 缺失**。如果 LI 完整,等於把 50% 的踩坑時間還給開發者。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 目標用戶(personas)
|
||||||
|
|
||||||
|
### P1:Claude Code / Cursor 等 IDE-embedded AI(主力)
|
||||||
|
|
||||||
|
特徵:
|
||||||
|
- 跟 leo 一起 pair programming
|
||||||
|
- 有 file access、shell 能力
|
||||||
|
- 透過 MCP / curl / SDK 跟 arcrun 對話
|
||||||
|
|
||||||
|
需求:
|
||||||
|
- 知道 arcrun 全貌(不靠人類解釋)
|
||||||
|
- 知道目前用戶有什麼資產(workflow / component / template / credential)
|
||||||
|
- 寫 YAML 不靠猜
|
||||||
|
- 部署前能 dry-run
|
||||||
|
- 部署後能看執行結果
|
||||||
|
- 卡住能 introspect + ask
|
||||||
|
|
||||||
|
### P2:用戶私人 agent(次要,未來)
|
||||||
|
|
||||||
|
特徵:
|
||||||
|
- 在 mira / 用戶 app 內跑
|
||||||
|
- 透過 MCP 對 arcrun
|
||||||
|
- 沒有檔案系統,只有 HTTP
|
||||||
|
|
||||||
|
需求:基本同 P1,但**完全靠 MCP**(沒 grep / file read fallback)
|
||||||
|
|
||||||
|
### P3:自製腳本 / SDK 使用者
|
||||||
|
|
||||||
|
不在本 SDD 範圍。SDK 既有規格走 `.agents/specs/arcrun/sdk-and-website/`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 範圍(系統涵蓋)
|
||||||
|
|
||||||
|
LI 不是「新建一個 service」,是**跨 5 個既有系統的橫向 layer**:
|
||||||
|
|
||||||
|
| 系統 | 既有狀態 | LI 涵蓋的部分 |
|
||||||
|
|---|---|---|
|
||||||
|
| **arcrun cypher-executor** | 31 HTTP 路由 | 哪些對外、哪些 AI 該看得到 |
|
||||||
|
| **arcrun registry** | component 管理 Worker | discovery layer |
|
||||||
|
| **u6u-mcp** | 15 MCP tools(HTTP→MCP 薄包裝) | 主擴張面(gap-fill) |
|
||||||
|
| **u6u-gui** | 人類 canvas IDE | 取其 endpoint 觀念,不取其 UI 元素 |
|
||||||
|
| **kbdb** | 50 個 HTTP 路由 | KBDB 該不該直接給 AI、用什麼姿勢 |
|
||||||
|
| **arcrun CLI (acr)** | Node CLI | LI 不依賴 CLI(CLI 是人的工具) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 非範圍(這個 SDD 不處理)
|
||||||
|
|
||||||
|
- 不處理 GUI 設計 / 視覺
|
||||||
|
- 不處理 SDK(Python/JS)API 設計
|
||||||
|
- 不處理 user OAuth flow
|
||||||
|
- 不重新設計 KBDB schema(只決定 LI 該包什麼 API)
|
||||||
|
- 不重新設計 cypher binding 語法
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 功能需求(FR)
|
||||||
|
|
||||||
|
### FR-1:AI 一條指令就能上手 arcrun
|
||||||
|
|
||||||
|
- 提供 `AGENTS.md`(onboarding doc),AI 載入就能用
|
||||||
|
- MCP server URL 寫進 doc,「直接 connect」一步搞定
|
||||||
|
- 不需讀 SDD / 不需 grep codebase / 不需問人
|
||||||
|
|
||||||
|
### FR-2:完整 CRUD + Discovery 對等
|
||||||
|
|
||||||
|
人類在 u6u-gui 能做的 8 個動作(list_workflows / get / update / execute / search_components / get_component / list_templates / list_credentials),**MCP 至少同等覆蓋**,不能有「人類能做但 AI 不能做」的 gap。
|
||||||
|
|
||||||
|
### FR-3:Dry-run 是預設行為
|
||||||
|
|
||||||
|
- `validate_workflow(yaml)` MCP tool(取既有 `/validate` 路由)
|
||||||
|
- `preview_workflow(yaml, input)` 不寫入 KV,模擬執行
|
||||||
|
- AI 養成習慣「先 dry-run 再 push」
|
||||||
|
|
||||||
|
### FR-4:可診斷的 trace
|
||||||
|
|
||||||
|
- `get_execution_trace(execution_id)` 回結構化 JSON:每個 node 的 status / input / output / error / duration
|
||||||
|
- `list_paused_executions()` 列卡住的執行(callback 沒回時 debug 用)
|
||||||
|
- 不靠 `wrangler tail` 純文字
|
||||||
|
|
||||||
|
### FR-5:錯誤訊息是「給 AI 看的下一步」
|
||||||
|
|
||||||
|
所有 MCP tool error response 必須含:
|
||||||
|
- `error_code`(穩定字串 enum,可程式化 catch)
|
||||||
|
- `human_message`(描述)
|
||||||
|
- `next_actions`(陣列,可選:「call X」/「執行 Y」/「修改 Z」)
|
||||||
|
|
||||||
|
範例:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error_code": "component_not_in_whitelist",
|
||||||
|
"human_message": "零件 'filter' 不在用戶可用清單",
|
||||||
|
"next_actions": [
|
||||||
|
"call list_components() 看完整可用清單",
|
||||||
|
"若該零件需平台啟用,告訴用戶執行 `acr enable filter`",
|
||||||
|
"若是自製零件,先 push 零件再 push workflow"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### FR-6:AI 可以回報問題
|
||||||
|
|
||||||
|
- `report_feedback(workflow_name, issue_type, description, ...)` MCP tool
|
||||||
|
- 結構化 enum issue_type(防自由文字難聚合)
|
||||||
|
- 寫進 KBDB 成 `agent-feedback` block,可被定期 review
|
||||||
|
- AI 規範「順利 / 卡住 / 不確定 都該 call」(透過 AGENTS.md 強制)
|
||||||
|
|
||||||
|
### FR-7:Implicit telemetry 不依賴 AI 自覺
|
||||||
|
|
||||||
|
- cypher-executor 每次 deploy / execute / fail 自動寫 `agent-telemetry` block
|
||||||
|
- 含 client_user_agent(哪個 AI 用的)、error_message、duration
|
||||||
|
- 不依賴 AI 主動 call,平台自己收
|
||||||
|
|
||||||
|
### FR-8:Skill blocks 可重用
|
||||||
|
|
||||||
|
- 「pattern / playbook」存 KBDB type=`agent-skill`
|
||||||
|
- MCP `list_skills(tag)` / `get_skill(id)` 給 AI 查
|
||||||
|
- 範例:`build_watcher_workflow` / `debug_paused_workflow` / `migrate_http_to_trigger_workflow`
|
||||||
|
|
||||||
|
### FR-9:範例庫可搜尋
|
||||||
|
|
||||||
|
- `search_examples(use_case)` 給 use case → 回相似 workflow YAML
|
||||||
|
- 範例存 `registry/examples/` git + 一份 index 在 KV
|
||||||
|
- 進階:semantic search(既有 KBDB `/search/embed`)
|
||||||
|
|
||||||
|
### FR-10:定期 review 機制(自動化)
|
||||||
|
|
||||||
|
- `agent_feedback_weekly_review` workflow(**arcrun 自己跑**)
|
||||||
|
- 每週一聚合 feedback + telemetry,產出 Top N 痛點 + 建議
|
||||||
|
- 寫進 `arcrun-roadmap` KBDB block
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 非功能需求(NFR)
|
||||||
|
|
||||||
|
### NFR-1:向下相容
|
||||||
|
|
||||||
|
LI 是新層,不破壞既有:
|
||||||
|
- u6u-mcp 既有 15 tool 全保留(廢棄走標準 deprecation)
|
||||||
|
- u6u-gui 不動
|
||||||
|
- cypher-executor 既有路由不改 contract
|
||||||
|
|
||||||
|
### NFR-2:transport 不鎖死
|
||||||
|
|
||||||
|
MCP 主,但 HTTP 同等可用(讓沒 MCP 客戶端的 agent 也可用)。每個 MCP tool 都對應一個 HTTP endpoint。
|
||||||
|
|
||||||
|
### NFR-3:error contract 穩定
|
||||||
|
|
||||||
|
`error_code` enum 是 public API,加新值是 minor,移除值是 major。版本化。
|
||||||
|
|
||||||
|
### NFR-4:feedback 數據可外部 export
|
||||||
|
|
||||||
|
`agent-feedback` / `agent-telemetry` 走 KBDB `/blocks` 標準 API,任何 AI / 人都能拉。不鎖死在 dashboard。
|
||||||
|
|
||||||
|
### NFR-5:覆蓋率可量化
|
||||||
|
|
||||||
|
- 「人類在 u6u-gui 能做但 MCP 不能做」清單必須能列出
|
||||||
|
- 每次 LI 改動後,這份清單往 0 收斂
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 成功指標
|
||||||
|
|
||||||
|
| 指標 | 量測 | 目標 |
|
||||||
|
|---|---|---|
|
||||||
|
| 新 AI agent 上手時間(從 zero 到第一個 workflow 部署) | 手動實驗 | < 10 min |
|
||||||
|
| 一次 workflow 部署需要的 MCP call 次數 | telemetry | < 5 calls |
|
||||||
|
| AI 回報「文件不清楚」週次數 | feedback aggregation | 持續下降 |
|
||||||
|
| 「MCP 缺工具」issue 類型佔比 | feedback type ratio | < 10% |
|
||||||
|
| Implicit telemetry 與 explicit feedback 比例 | 計數 | 9:1 不算問題(implicit 量大正常) |
|
||||||
|
| Coverage gap(人類能做 vs AI 能做) | 手動審查 | 0 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 風險與假設
|
||||||
|
|
||||||
|
### 假設
|
||||||
|
- MCP 是主要 AI 接觸點(不是 SDK / CLI)
|
||||||
|
- AI 願意按 AGENTS.md 規範 call feedback tool
|
||||||
|
- KBDB 能承載 telemetry 量級(短期:< 1000 events / day)
|
||||||
|
|
||||||
|
### 風險
|
||||||
|
|
||||||
|
| 風險 | 應對 |
|
||||||
|
|---|---|
|
||||||
|
| AI 不照 AGENTS.md 規範用 | implicit telemetry 不依賴自覺,仍能收 data |
|
||||||
|
| MCP tool 設計錯誤越改越亂 | 每個 tool 有 contract test,error_code enum 版本化 |
|
||||||
|
| feedback 雜訊太多無法 review | review workflow 用 LLM 聚合,不靠人讀原始 |
|
||||||
|
| u6u-mcp 重構搞壞既有 | 走 deprecation,舊 tool 留 90 天 |
|
||||||
|
| KBDB telemetry 量爆掉 | sample rate / 老資料自動 archive |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 開放問題(待 leo 決策)
|
||||||
|
|
||||||
|
1. **deployment 名稱**:MCP server URL 用 `mcp.arcrun.dev` 還是沿用 `mcp.finally.click`?
|
||||||
|
- 推薦:`mcp.arcrun.dev`(品牌一致,符合 mira CLAUDE.md §1.7)
|
||||||
|
2. **新 MCP vs 擴張 u6u-mcp**:在現有 u6u-mcp 加 tool,還是建 `arcrun-mcp` 取代?
|
||||||
|
- 推薦:先擴張 u6u-mcp(避免 fork 痛),長期可改名
|
||||||
|
3. **AGENTS.md 放哪**:`arcrun/AGENTS.md` repo 根?還是 KBDB block?或兩個地方都?
|
||||||
|
- 推薦:repo `arcrun/AGENTS.md` + KBDB 自動同步 block(讓沒 git access 的 AI 也能讀)
|
||||||
|
4. **feedback 寫入是否需 auth**:AI report bug 要不要 api_key?
|
||||||
|
- 推薦:要(防 spam),但 tier 低(只要驗 ak_ 存在)
|
||||||
@@ -0,0 +1,244 @@
|
|||||||
|
# Tasks: LI (LLM Interface) for arcrun
|
||||||
|
|
||||||
|
> SDD: design.md + requirements.md(同目錄)
|
||||||
|
> 進度標記:`[ ]` pending / `[🔄]` doing / `[x]` done / `[⏸]` blocked
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone 1:可量測(先收 data)
|
||||||
|
|
||||||
|
目標:1 週內把「平台自己收 AI 用得好不好」的數據管道接通。
|
||||||
|
|
||||||
|
### M1.1 AGENTS.md v1
|
||||||
|
- [ ] 寫 `arcrun/AGENTS.md`(按 design.md §5 模板)
|
||||||
|
- [ ] CI hook:repo `AGENTS.md` 變動 → 自動同步 KBDB block
|
||||||
|
- [ ] `arcrun_get_onboarding` MCP tool(讀 KBDB block)
|
||||||
|
|
||||||
|
### M1.2 Implicit telemetry 收集
|
||||||
|
- [ ] 建 KBDB template `agent-telemetry`(slots: event_type, workflow_name, error_code, duration_ms, api_key_hash, agent_user_agent)
|
||||||
|
- [ ] cypher-executor `webhook-handlers.executeWebhookGraph` 末尾加 telemetry 寫入(成功 / 失敗都記)
|
||||||
|
- [ ] cypher-executor `routes/webhooks-named.ts` push 加 deploy 事件
|
||||||
|
- [ ] cypher-executor `routes/cypher.ts` validate 失敗 → validation_error 事件
|
||||||
|
- [ ] api_key SHA-256 截 16 字元 helper
|
||||||
|
- [ ] 隱私 check:workflow content 不 log,只 name
|
||||||
|
|
||||||
|
### M1.3 Explicit feedback tool
|
||||||
|
- [ ] 建 KBDB template `agent-feedback`(slots: issue_type, workflow_name, retry_count, blocked, suggested_fix, agent_user_agent)
|
||||||
|
- [ ] u6u-mcp 加 tool `arcrun_report_feedback`
|
||||||
|
- [ ] Zod schema 鎖死 issue_type enum
|
||||||
|
- [ ] 寫入時 `user_id` 從 partner-auth middleware 拿
|
||||||
|
- [ ] 寫入時 tag 自動補(`agent-feedback`, `issue:{type}`)
|
||||||
|
|
||||||
|
### M1.4 驗收
|
||||||
|
- [ ] 用 Claude Code 跑 mira 開發 1-2 天,自然累積 telemetry + feedback
|
||||||
|
- [ ] 用 `curl kbdb-get.arcrun.dev type=agent-telemetry` 確認有 data
|
||||||
|
- [ ] 用 `curl kbdb-get.arcrun.dev type=agent-feedback` 確認 enum 有效
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone 2:gap-fill(補 MCP 工具)
|
||||||
|
|
||||||
|
目標:人類 GUI 能做的,AI 透過 MCP 都能做。
|
||||||
|
|
||||||
|
### M2.1 新增 cypher-executor 路由
|
||||||
|
|
||||||
|
- [ ] `GET /executions/:id` — 回結構化 trace(讀 EXEC_CONTEXT KV)
|
||||||
|
- 既有資料:graph-executor.ts trace array,需確認 KV 持久化
|
||||||
|
- [ ] `GET /workflows/:name/executions?limit=10` — 最近 N 次執行 ID + 摘要
|
||||||
|
- 需 ANALYTICS_KV 或新 index
|
||||||
|
- [ ] `GET /executions/paused` — 列當前 paused executions
|
||||||
|
- 走 EXEC_CONTEXT KV scan `paused:*` prefix
|
||||||
|
- [ ] `POST /preview` — dry-run,不寫 KV
|
||||||
|
- 複用 GraphExecutor,env.EXEC_CONTEXT 改 in-memory mock
|
||||||
|
- [ ] `POST /webhooks/named/:name/diff` — 新舊 YAML diff
|
||||||
|
- [ ] `GET /my-telemetry?limit=N` — 用戶自己看 telemetry
|
||||||
|
|
||||||
|
### M2.2 MCP tools(包既有 + 新增 endpoint)
|
||||||
|
|
||||||
|
- [ ] `arcrun_validate_yaml` — 包 `/validate`
|
||||||
|
- [ ] `arcrun_get_execution_trace`
|
||||||
|
- [ ] `arcrun_list_recent_executions`
|
||||||
|
- [ ] `arcrun_list_paused_executions`
|
||||||
|
- [ ] `arcrun_resume_execution` — 包 `/workflows/resume`
|
||||||
|
- [ ] `arcrun_list_workflows` — 既有但確認
|
||||||
|
- [ ] `arcrun_get_workflow`
|
||||||
|
- [ ] `arcrun_delete_workflow`
|
||||||
|
- [ ] `arcrun_preview_workflow`
|
||||||
|
- [ ] `arcrun_diff_workflow`
|
||||||
|
- [ ] `arcrun_list_recipes` / `arcrun_create_recipe`
|
||||||
|
- [ ] `arcrun_list_auth_recipes` / `arcrun_create_auth_recipe`
|
||||||
|
- [ ] `arcrun_my_telemetry`
|
||||||
|
|
||||||
|
### M2.3 Error contract 統一
|
||||||
|
|
||||||
|
- [ ] 定義 `error_code` enum v1(design.md §3.1.4)
|
||||||
|
- [ ] u6u-mcp 所有 tool 統一 error wrap(helper function)
|
||||||
|
- [ ] cypher-executor 所有 route 統一 error response(含 error_code + next_actions)
|
||||||
|
- [ ] 寫測試:每個 error_code 至少一個 case
|
||||||
|
|
||||||
|
### M2.4 驗收
|
||||||
|
|
||||||
|
- [ ] 模擬 zero-knowledge AI(新 conversation)按 AGENTS.md 部署一個 hello workflow
|
||||||
|
- [ ] 量測:從 `list_components` 到 `run_workflow` 成功總 MCP call < 5
|
||||||
|
- [ ] 比較人類 GUI 路徑,clickwise 對等
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone 3:skill blocks + examples
|
||||||
|
|
||||||
|
目標:AI 寫第一個 workflow 不靠猜,有範本和 playbook。
|
||||||
|
|
||||||
|
### M3.1 種子 skill blocks(5 個)
|
||||||
|
|
||||||
|
- [ ] `skill-build_watcher_workflow` — cron + 過濾 + trigger 模式
|
||||||
|
- [ ] `skill-debug_paused_workflow` — claude_api callback 流程 + 怎麼追
|
||||||
|
- [ ] `skill-migrate_http_to_trigger_workflow` — 從 self-fetch 換 trigger_workflow
|
||||||
|
- [ ] `skill-rag_with_arcrun` — KBDB search + claude_api 組裝
|
||||||
|
- [ ] `skill-add_new_wasm_component` — TinyGo 寫 + push + 註冊白名單
|
||||||
|
|
||||||
|
### M3.2 MCP tools
|
||||||
|
|
||||||
|
- [ ] `arcrun_list_skills(tag?)`
|
||||||
|
- [ ] `arcrun_get_skill(id)`
|
||||||
|
- [ ] `arcrun_publish_skill` — AI 把學到的回存
|
||||||
|
|
||||||
|
### M3.3 種子 examples(10 個)
|
||||||
|
|
||||||
|
- [ ] `webhook-to-slack`
|
||||||
|
- [ ] `cron-watcher`
|
||||||
|
- [ ] `llm-classify`
|
||||||
|
- [ ] `rag-search-answer`
|
||||||
|
- [ ] `email-summary`
|
||||||
|
- [ ] `pdf-to-blocks`
|
||||||
|
- [ ] `github-issue-bot`
|
||||||
|
- [ ] `daily-digest`
|
||||||
|
- [ ] `parallel-fanout`
|
||||||
|
- [ ] `error-retry`
|
||||||
|
|
||||||
|
每個包 `workflow.yaml` + `description.md` + `tags.json`,放 `arcrun/registry/examples/{slug}/`。
|
||||||
|
|
||||||
|
### M3.4 examples 索引 + 搜尋
|
||||||
|
|
||||||
|
- [ ] CI build 範例 → KBDB block type=`workflow-example`(含 YAML + tags + description)
|
||||||
|
- [ ] `arcrun_search_examples(use_case)` MCP tool(走 KBDB `/search`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone 4:closed loop
|
||||||
|
|
||||||
|
目標:data 收得到 → 平台自己消化產出 roadmap。
|
||||||
|
|
||||||
|
### M4.1 Weekly review workflow
|
||||||
|
|
||||||
|
- [ ] 寫 `polaris/mira/arcrun/agent_feedback_weekly_review.yaml`(依 design.md §4.5 範本)
|
||||||
|
- [ ] cron `0 9 * * 1`(週一早 9 UTC)
|
||||||
|
- [ ] `acr push`
|
||||||
|
- [ ] 手動觸發測試一次
|
||||||
|
|
||||||
|
### M4.2 LLM 聚合 prompt
|
||||||
|
|
||||||
|
- [ ] 寫 prompt:把 feedback + telemetry 餵 Claude → 產出 Top 5 痛點 + 建議
|
||||||
|
- [ ] 結果格式固定:markdown sections(痛點 / 證據 / 建議 / 嚴重度)
|
||||||
|
- [ ] 存 KBDB block type=`arcrun-roadmap`
|
||||||
|
|
||||||
|
### M4.3 通知
|
||||||
|
|
||||||
|
- [ ] notify_telegram 節點:推給 leo
|
||||||
|
- [ ] 同時寫進 mira 河道(讓 leo 在熟悉介面看)
|
||||||
|
|
||||||
|
### M4.4 驗收
|
||||||
|
|
||||||
|
- [ ] 跑滿 1 週 → 收到第一份 roadmap
|
||||||
|
- [ ] leo review 後挑 1-2 個 issue 修補
|
||||||
|
- [ ] 跑第二週 → 確認該 issue 從 top list 消失
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone 5:rename + cleanup
|
||||||
|
|
||||||
|
目標:完成 LI 品牌化,u6u-mcp 退場。
|
||||||
|
|
||||||
|
### M5.1 部署
|
||||||
|
|
||||||
|
- [ ] u6u-mcp Worker 改 wrangler name → `arcrun-mcp`
|
||||||
|
- [ ] 加 route `mcp.arcrun.dev/*`
|
||||||
|
- [ ] 舊 route `mcp.finally.click/*` 保留 90 天
|
||||||
|
|
||||||
|
### M5.2 Tool rename
|
||||||
|
|
||||||
|
- [ ] 每個 `u6u_*` tool 加 alias `arcrun_*`
|
||||||
|
- [ ] 舊 tool call → response 含 `deprecation_warning` 欄
|
||||||
|
- [ ] AGENTS.md 全用新名
|
||||||
|
|
||||||
|
### M5.3 文件
|
||||||
|
|
||||||
|
- [ ] `arcrun/AGENTS.md` 最終版
|
||||||
|
- [ ] `u6u-mcp/README.md` 標 deprecated,指向新位置
|
||||||
|
- [ ] `matrix/arcrun/.agents/specs/llm-interface/design.md` 更新「實際部署狀態」附錄
|
||||||
|
|
||||||
|
### M5.4 觀察 90 天
|
||||||
|
|
||||||
|
- [ ] 監控舊 tool name 使用率
|
||||||
|
- [ ] 0 使用 → 移除 alias
|
||||||
|
- [ ] 寫一篇 retrospective:LI 做完前後 AI 使用 arcrun 的 time-to-first-workflow 對比
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backlog(暫不排)
|
||||||
|
|
||||||
|
### B.1 KBDB MCP 獨立 SDD
|
||||||
|
- LI 範圍只包 KBDB 的 `agent-*` template
|
||||||
|
- 完整 KBDB AI 介面(type=note/page/triplet/template/record 等)另立 SDD `kbdb-llm-interface`
|
||||||
|
- 跟 mira KM 系統互動最密
|
||||||
|
|
||||||
|
### B.2 多 agent 隔離
|
||||||
|
- 多 AI 共用同 ak_xxx 時,telemetry 區隔 agent_user_agent
|
||||||
|
- 進階:每個 AI 子 namespace(mira / cursor / 自製 agent)
|
||||||
|
|
||||||
|
### B.3 AGENTS.md i18n
|
||||||
|
- v1 純中文(leo + 自家用)
|
||||||
|
- v2 英文版(給開源用戶)
|
||||||
|
|
||||||
|
### B.4 自動 skill 萃取
|
||||||
|
- weekly_review 產出的 pattern 自動包成 skill draft
|
||||||
|
- leo review approve → publish
|
||||||
|
|
||||||
|
### B.5 SDK 對等(python-sdk / js-sdk)
|
||||||
|
- SDK 提供和 MCP 同樣的 25 個 method
|
||||||
|
- 給「不想用 MCP 的人」也能 AI-friendly
|
||||||
|
- 走 sdk-and-website SDD 範圍
|
||||||
|
|
||||||
|
### B.6 GUI side 補 LI 看板
|
||||||
|
- u6u-gui 加 `/li-dashboard` 顯示用戶自己的 telemetry / feedback
|
||||||
|
- 不阻擋 LI 推出(leo 先看 KBDB 原始 block 即可)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 依賴關係
|
||||||
|
|
||||||
|
```
|
||||||
|
M1 (data 收集)
|
||||||
|
↓
|
||||||
|
M2 (MCP gap-fill)
|
||||||
|
↓
|
||||||
|
M3 (skill + examples) ← 可與 M2 並行後段
|
||||||
|
↓
|
||||||
|
M4 (closed loop) ←─── 需 M1 data 累積 1-2 週
|
||||||
|
↓
|
||||||
|
M5 (rename)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 工估算
|
||||||
|
|
||||||
|
| Milestone | 工 | 阻擋項 |
|
||||||
|
|---|---|---|
|
||||||
|
| M1 | 5 個工作日 | 無 |
|
||||||
|
| M2 | 5 個工作日 | M1 完(telemetry 先就位才好驗證 M2 改動) |
|
||||||
|
| M3 | 5 個工作日 | M2 完(tool 介面定型才寫 skill) |
|
||||||
|
| M4 | 3 個工作日 | M1 data 累積 1 週 |
|
||||||
|
| M5 | 5 個工作日 | M2-M4 完 |
|
||||||
|
| **總** | **23 個工作日 (~5 週)** | |
|
||||||
|
|
||||||
|
實際視 leo 排程,可邊用邊改、不必一氣呵成。**M1 是硬前置**——資料不收,改了也不知道改對沒。
|
||||||
@@ -56,6 +56,7 @@
|
|||||||
| 子系統 | SDD |
|
| 子系統 | SDD |
|
||||||
|--------|-----|
|
|--------|-----|
|
||||||
| **主要(正在動)** Credential Primitives WASM 改寫 | `.agents/specs/arcrun/credential-primitives-wasm/` |
|
| **主要(正在動)** Credential Primitives WASM 改寫 | `.agents/specs/arcrun/credential-primitives-wasm/` |
|
||||||
|
| **LI (LLM Interface)** — AI 操盤手使用體驗(2026-05-16 新建,mira dogfood 痛點轉化) | `.agents/specs/llm-interface/` |
|
||||||
| arcrun 總進度 | `.agents/specs/arcrun/arcrun.md` |
|
| arcrun 總進度 | `.agents/specs/arcrun/arcrun.md` |
|
||||||
| Auth Recipe 系統(schema、預建 20 個服務) | `.agents/specs/arcrun/auth-recipe.md` |
|
| Auth Recipe 系統(schema、預建 20 個服務) | `.agents/specs/arcrun/auth-recipe.md` |
|
||||||
| Landing Page | `.agents/specs/arcrun/landing-page.md` |
|
| Landing Page | `.agents/specs/arcrun/landing-page.md` |
|
||||||
|
|||||||
Reference in New Issue
Block a user