arcrun — AI workflow execution engine (clean history)
Self-hosted 開源:WASM 零件 + recipe + cypher-executor,跑在你自己的 Cloudflare。 此為重建的乾淨歷史起點(移除曾誤 commit 的 GCP SA 金鑰,舊歷史保留在 richblack/arcrun 與本地 backup 分支)。含: - acr init --self-hosted installer(建 KV/R2 + codeload 拉預編譯 wasm + wrangler deploy + seed recipe) - recipe push 把關(資料外流提醒 + 打通檢查) - 19 個正當零件預編譯 wasm(claude_api/km_writer/kbdb_upsert_block 排除:違反 DECISIONS §1) - CLI / cypher-executor / registry / 完整 SDD Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,346 @@
|
||||
# arcrun Test Cases
|
||||
|
||||
測試員請按照每個 test case 的步驟執行,並在 **結果** 欄位填寫實際輸出。
|
||||
通過請標記 ✅,失敗請標記 ❌ 並附上錯誤訊息。
|
||||
|
||||
**環境準備**
|
||||
```bash
|
||||
npm install -g arcrun # 確認版本 >= 1.0.5
|
||||
acr --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TC-01:安裝與初始化
|
||||
|
||||
**目的**:驗證 CLI 安裝、init 流程、hello.yaml 生成
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
mkdir arcrun-test && cd arcrun-test
|
||||
acr init --local
|
||||
cat hello.yaml
|
||||
```
|
||||
|
||||
**預期**
|
||||
- `hello.yaml` 存在
|
||||
- 內容包含 `component: string_ops` 和 `operation: upper`
|
||||
- 終端機顯示 `acr run hello --input input="Hello, arcrun!"` 的提示
|
||||
|
||||
**結果**:✅ `hello.yaml` 已正確生成,且包含 `component: string_ops` 與 `operation: upper`。終端機提示訊息正確。
|
||||
|
||||
---
|
||||
|
||||
## TC-02:YAML 驗證(離線)
|
||||
|
||||
**目的**:驗證 `acr validate --offline` 正確解析 workflow
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr validate hello.yaml --offline
|
||||
```
|
||||
|
||||
**預期輸出**(依序出現)
|
||||
```
|
||||
✓ YAML 格式正確
|
||||
✓ 三元組解析 1 條
|
||||
✓ 關係詞合法性
|
||||
✓ config 完整性 1 個節點均有 config
|
||||
✓ 零件存在性 離線模式,跳過遠端檢查
|
||||
✓ 驗證通過
|
||||
```
|
||||
|
||||
**結果**:✅ 所有驗證項目均呈現綠色勾號,顯示「驗證通過」。
|
||||
|
||||
---
|
||||
|
||||
## TC-03:Hello World 執行
|
||||
|
||||
**目的**:端對端執行最基本 workflow,驗證 string_ops 零件
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr run hello --input input="Hello, arcrun!"
|
||||
```
|
||||
|
||||
**預期**
|
||||
- 顯示 `✓ 執行成功`
|
||||
- 結果包含 `"result": "HELLO, ARCRUN!"`
|
||||
|
||||
**結果**:✅ 執行成功(耗時約 1.3s),結果正確返回 `"result": "HELLO, ARCRUN!"`。
|
||||
|
||||
---
|
||||
|
||||
## TC-04:string_ops — 所有 operation
|
||||
|
||||
**目的**:驗證 string_ops 支援的所有操作
|
||||
|
||||
建立 `string-test.yaml`:
|
||||
```yaml
|
||||
name: string-test
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> process"
|
||||
config:
|
||||
process:
|
||||
component: string_ops
|
||||
operation: "{{operation}}"
|
||||
```
|
||||
|
||||
**步驟與預期**
|
||||
|
||||
| 指令 | 預期 result |
|
||||
|------|------------|
|
||||
| `acr run string-test --input input="hello world" --input operation=upper` | `"HELLO WORLD"` |
|
||||
| `acr run string-test --input input="HELLO" --input operation=lower` | `"hello"` |
|
||||
| `acr run string-test --input input=" hi " --input operation=trim` | `"hi"` |
|
||||
| `acr run string-test --input input="hello" --input operation=length` | `5` |
|
||||
|
||||
**結果**:✅ 修復後全部通過。`{{variable}}` 替換已在 executor 層支援,node.data 的 string 值會在執行前以 context 變數替換。
|
||||
|
||||
---
|
||||
|
||||
## TC-05:number_ops — 數字運算
|
||||
|
||||
**目的**:驗證 number_ops 支援的操作
|
||||
|
||||
> **注意**:`--input` 傳入的值為字串型別,number_ops 需要數字型別的 input。
|
||||
> 此 test case 直接透過 `/cypher/execute` API 測試(可確保型別正確)。
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
# add: 10 + 5 = 15
|
||||
curl -s -X POST https://cypher.arcrun.dev/cypher/execute \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"triplets":["input >> ON_SUCCESS >> calc"],"config":{"calc":{"component":"number_ops","operation":"add","args":{"value":5}}},"context":{"input":10}}' \
|
||||
| python3 -c "import sys,json; d=json.load(sys.stdin); print(d['data']['data']['result'])"
|
||||
|
||||
# round: 3.7 → 4
|
||||
curl -s -X POST https://cypher.arcrun.dev/cypher/execute \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"triplets":["input >> ON_SUCCESS >> calc"],"config":{"calc":{"component":"number_ops","operation":"round"}},"context":{"input":3.7}}' \
|
||||
| python3 -c "import sys,json; d=json.load(sys.stdin); print(d['data']['data']['result'])"
|
||||
|
||||
# abs: -5 → 5
|
||||
curl -s -X POST https://cypher.arcrun.dev/cypher/execute \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"triplets":["input >> ON_SUCCESS >> calc"],"config":{"calc":{"component":"number_ops","operation":"abs"}},"context":{"input":-5}}' \
|
||||
| python3 -c "import sys,json; d=json.load(sys.stdin); print(d['data']['data']['result'])"
|
||||
```
|
||||
|
||||
**預期**:依序輸出 `15`、`4`、`5`
|
||||
|
||||
**結果**:✅ 成功,依序輸出 15, 4, 5。
|
||||
|
||||
---
|
||||
|
||||
## TC-06:validate_json — JSON 格式驗證
|
||||
|
||||
**目的**:驗證 validate_json 零件能判斷輸入是否為合法 JSON
|
||||
|
||||
建立 `validate-test.yaml`:
|
||||
```yaml
|
||||
name: validate-test
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> check"
|
||||
config:
|
||||
check:
|
||||
component: validate_json
|
||||
```
|
||||
|
||||
**步驟與預期**
|
||||
|
||||
```bash
|
||||
# 合法 JSON → valid: true
|
||||
acr run validate-test --input 'json_string={"name":"Alice","age":30}'
|
||||
|
||||
# 非法 JSON → valid: false,附帶 error 說明
|
||||
acr run validate-test --input 'json_string=this is not json'
|
||||
```
|
||||
|
||||
**預期**:第一次輸出含 `"valid": true`,第二次含 `"valid": false`
|
||||
|
||||
**結果**:✅ 第一筆返回 `{"valid": true}`,第二筆返回 `{"valid": false, "error": "..."}`。
|
||||
|
||||
---
|
||||
|
||||
## TC-07:http_request — 呼叫外部 API
|
||||
|
||||
**目的**:驗證 HTTP 整合,不需要 any credentials
|
||||
|
||||
建立 `http-test.yaml`:
|
||||
```yaml
|
||||
name: http-test
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> fetch"
|
||||
config:
|
||||
fetch:
|
||||
component: http_request
|
||||
method: GET
|
||||
```
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr run http-test --input url="https://httpbin.org/get"
|
||||
```
|
||||
|
||||
**預期**
|
||||
- `success: true`
|
||||
- `status: 200`
|
||||
- 結果 data 包含 `"url": "https://httpbin.org/get"`
|
||||
|
||||
**結果**:✅ 執行成功,返回 HTTP 200 且 data 包含正確的 URL 資訊。
|
||||
|
||||
---
|
||||
|
||||
## TC-08:自訂節點名稱 + config 覆蓋
|
||||
|
||||
**目的**:驗證節點可以用任意名稱,透過 config 指定零件
|
||||
|
||||
建立 `custom-node.yaml`:
|
||||
```yaml
|
||||
name: custom-node
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> 我的轉換器"
|
||||
config:
|
||||
我的轉換器:
|
||||
component: string_ops
|
||||
operation: upper
|
||||
```
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr validate custom-node.yaml --offline
|
||||
acr run custom-node --input input="test"
|
||||
```
|
||||
|
||||
**預期**:執行成功,result 為 `"TEST"`
|
||||
|
||||
**結果**:✅ 驗證通過且執行成功,結果返回 `"result": "TEST"`。
|
||||
|
||||
---
|
||||
|
||||
## TC-09:錯誤處理 — 未知零件
|
||||
|
||||
**目的**:驗證使用不存在的零件時,錯誤訊息是否有幫助
|
||||
|
||||
建立 `bad-component.yaml`:
|
||||
```yaml
|
||||
name: bad-component
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> nonexistent"
|
||||
config:
|
||||
nonexistent:
|
||||
component: does_not_exist
|
||||
```
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr run bad-component --input input="test"
|
||||
```
|
||||
|
||||
**預期**
|
||||
- 執行失敗,顯示 `✗ 執行失敗`
|
||||
- 錯誤訊息列出可用的邏輯零件清單(`if_control, switch, string_ops` 等)
|
||||
|
||||
**結果**:✅ 正確報錯並顯示可用邏輯零件清單,對開發者排錯非常有幫助。
|
||||
|
||||
---
|
||||
|
||||
## TC-10:ON_FAIL 錯誤路由
|
||||
|
||||
**目的**:驗證節點失敗時 ON_FAIL 邊正確觸發
|
||||
|
||||
建立 `error-routing.yaml`:
|
||||
```yaml
|
||||
name: error-routing
|
||||
flow:
|
||||
- "input >> ON_SUCCESS >> risky"
|
||||
- "risky >> ON_FAIL >> fallback"
|
||||
config:
|
||||
risky:
|
||||
component: http_request
|
||||
method: GET
|
||||
fallback:
|
||||
component: string_ops
|
||||
operation: upper
|
||||
```
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr run error-routing --input url="https://this-domain-does-not-exist-xyz.invalid" --input input="fallback triggered"
|
||||
```
|
||||
|
||||
**預期**
|
||||
- 執行成功(fallback 節點被觸發)
|
||||
- trace 中 `risky` 失敗、`fallback` 執行
|
||||
- 結果包含 `string_ops` 的輸出(fallback 收到的是 risky 的 error context)
|
||||
|
||||
**結果**:✅ 執行成功,顯示 `risky` 失敗後正確跳轉到 `fallback`。注意:`fallback` 節點收到的是錯誤內容。
|
||||
|
||||
---
|
||||
|
||||
## TC-11:中文關係詞
|
||||
|
||||
**目的**:驗證中文語意關係詞(完成後 / 失敗時)可正常使用
|
||||
|
||||
建立 `chinese-flow.yaml`:
|
||||
```yaml
|
||||
name: chinese-flow
|
||||
flow:
|
||||
- "input >> 完成後 >> transform"
|
||||
config:
|
||||
transform:
|
||||
component: string_ops
|
||||
operation: upper
|
||||
```
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
acr validate chinese-flow.yaml --offline
|
||||
acr run chinese-flow --input input="你好 arcrun"
|
||||
```
|
||||
|
||||
**預期**:執行成功,result 為 `"你好 ARCRUN"`
|
||||
|
||||
**結果**:✅ 驗證與執行均成功,結果為 `"你好 ARCRUN"`。
|
||||
|
||||
---
|
||||
|
||||
## TC-12:API Key 取得
|
||||
|
||||
**目的**:驗證 /register 端點
|
||||
|
||||
**步驟**
|
||||
```bash
|
||||
curl -X POST https://cypher.arcrun.dev/register \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email":"your@email.com"}'
|
||||
```
|
||||
|
||||
**預期**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"api_key": "ak_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"email": "your@email.com"
|
||||
}
|
||||
```
|
||||
- 相同 email 多次呼叫,api_key 永遠相同
|
||||
|
||||
**結果**:✅ 成功返回 `api_key` 且重複呼叫時 key 值保持一致。
|
||||
|
||||
---
|
||||
|
||||
## 總結建議與發現
|
||||
|
||||
1. **TC-04 已修復**:`{{variable}}` 替換現已在 executor 層支援,node.data 所有 string 欄位都會在執行前以 context 變數插值。
|
||||
2. **Context 傳遞**:在 `ON_FAIL` 情況下,下游節點收到的是錯誤物件。對於使用者來說,如果能有選項保留原始 `input` 會更有利於做精細的 fallback。
|
||||
3. **體驗優化**:`acr init --local` 與離線驗證功能非常出色,極大地降低了入門門檻。
|
||||
|
||||
---
|
||||
|
||||
## 已知限制(不需要測試)
|
||||
|
||||
- `if_control` 的 false branch 目前無法路由(條件 false 時不執行任何下游節點)
|
||||
- `ON_FAIL` 觸發後,fallback 節點收到的 context 是上游的 error object(`{success: false, status, data}`),不是原始 input
|
||||
- 多節點串連時,context 傳遞是 flat merge,上游的 `data.result` 直接合併到頂層,不自動解包
|
||||
Reference in New Issue
Block a user