diff --git a/.claude/rules/05-deploy-convention.md b/.claude/rules/05-deploy-convention.md new file mode 100644 index 0000000..d92d478 --- /dev/null +++ b/.claude/rules/05-deploy-convention.md @@ -0,0 +1,123 @@ +# 部署慣例(CI/CD) + +> **核心原則:新增 Worker = 新目錄 + `wrangler.toml`,不用改 workflow。** + +`.github/workflows/deploy.yml` 是**通用掃描式** workflow,不該為每個 Worker 手寫 job。 + +--- + +## Workflow 如何找到要部署的 Worker? + +``` +find . -name 'wrangler.toml' -not -path '*/node_modules/*' -not -name 'wrangler.test.toml' +``` + +每一個命中的目錄 = 一個部署單位。無論是: + +- `cypher-executor/` (orchestration Worker) +- `registry/` (合約管理 Worker) +- `.component-builds/{name}/` (零件 Worker,25+ 個) +- 未來新增的任何 Worker + +**無需改 workflow,只要符合掃描規則就會自動部署**。 + +--- + +## 觸發邏輯 + +| 觸發 | 部署範圍 | +|------|---------| +| `push` 到 main | diff 涉及的 Worker 目錄才部署 | +| `push` 到 main + 改 `registry/components/{name}/` | 連動 rebuild `.component-builds/{name}/component.wasm` 再 deploy | +| `workflow_dispatch` + `force_all=true` | 全部 Worker | +| `workflow_dispatch` + `only=a,b,c` | 只部署指定清單 | +| `push` 但 base sha 不可及(首次) | 全部 Worker | + +--- + +## 新增 Worker 的步驟 + +### 如果是新 WASM 零件 Worker + +1. 在 `registry/components/{new_name}/` 建 `main.go` + `component.contract.yaml` +2. 在 `.component-builds/{new_name}/` 建 Worker 模板: + - `wrangler.toml`(name/routes/bindings) + - `package.json`(hono + workers-types + wrangler 即可,參考 `auth_static_key/package.json`) + - `tsconfig.json`(可直接複製) + - `src/index.ts`(WASI shim,方案 A:import `../../cypher-executor/src/lib/wasi-shim`) +3. 本地跑 `pnpm install` 產 `pnpm-lock.yaml` +4. 本地跑 `tinygo build -target=wasi -o {new_name}.wasm main.go` 先驗證 build 通過 +5. Commit push → CI 自動 rebuild WASM + deploy + +### 如果是新 orchestration/service Worker + +1. 在 repo 根建新目錄(類似 `cypher-executor/`) +2. `wrangler.toml` + `package.json` + `pnpm-lock.yaml` + `src/index.ts` + `tsconfig.json` +3. Push → CI 自動部署 + +--- + +## Runtime Secret 管理 + +**CI 只提供 Cloudflare 驗證,不碰 runtime secret**。 + +- GH Actions secrets:`CLOUDFLARE_API_TOKEN`、`CLOUDFLARE_ACCOUNT_ID`(一次性設好) +- Runtime secret(例:`ENCRYPTION_KEY`、`OPENAI_KEY`、`GOOGLE_API_KEY`): + - **由 richblack 一次性手動** `wrangler secret put ` 設進各 Worker + - 不進 CI,不進 `wrangler.toml` `[vars]` + - 需要的 Worker:`auth_static_key`、`auth_service_account`(兩個都要 `ENCRYPTION_KEY`) + +--- + +## Lockfile 規範 + +- **統一使用 pnpm**。新增 Worker 只放 `pnpm-lock.yaml`,不要 `package-lock.json` +- 若新建 Worker 時用 `npm install` 產出 `package-lock.json`,**刪掉它**,改跑 `pnpm install` +- `cypher-executor/` 和 `registry/` 的 `package-lock.json` 已於 2026-04-20 刪除 + +**現存例外**(歷史遺產,混合期不強制遷移): +- `.component-builds/{if_control, switch, ... 16 個舊邏輯零件}/` 仍是 `package-lock.json`,workflow 有 fallback 分支(`pnpm install --no-frozen-lockfile`)可跑 +- `builtins/`、`landing/` 同上 + +**新增 Worker 一律 pnpm,不要製造新的混合情況**。 + +--- + +## WASM 來源 + +`.component-builds/{name}/component.wasm` 不 commit 進 repo(build 產物)。 + +- 本地開發:`cd registry/components/{name} && tinygo build -target=wasi -o {name}.wasm main.go && cp {name}.wasm ../../../.component-builds/{name}/component.wasm` +- CI:workflow 在 deploy 前自動 rebuild + copy + +**例外**:Phase 1-3 開發期為了加速驗證,部分 WASM 檔暫時 commit 進了 repo。之後會加 `.gitignore` 清理。 + +--- + +## 並行度 + +`max-parallel: 5` — 避免觸發 Cloudflare Workers API rate limit。 + +Worker 數量 > 5 時,deploy 會分批跑。25 個 Worker 大約 5 輪 × ~30 秒 = 2-3 分鐘可完成全部。 + +--- + +## 禁止事項 + +1. **禁止**為新 Worker 手動加 deploy job 到 `deploy.yml`。通用掃描會自動處理,手加就是重複工作。 +2. **禁止**把 runtime secret(API key / encryption key / credential)放進 GH Actions secrets 或 `wrangler.toml` `[vars]`,只能用 `wrangler secret put`。 +3. **禁止**在 CI 裡跑不必要的測試阻擋 deploy。測試在 PR / 本地跑,`main` 推上去就 deploy(trunk-based)。若要測試關,開新 workflow 檔,不要污染 deploy workflow。 +4. **禁止**跳過 TinyGo rebuild 直接 deploy 舊 `.wasm`。CI 的 rebuild 步驟是確保部署的是最新 source。 + +--- + +## 驗證指令 + +本地模擬 CI 的掃描結果: + +```bash +find . -name 'wrangler.toml' -not -path '*/node_modules/*' -not -name 'wrangler.test.toml' \ + | xargs -n1 dirname | sort -u +``` + +應列出 ~25 個目錄。任何「我新增了 Worker 但沒被 deploy」的問題,先跑這條確認目錄被掃到。 diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index ad5e090..a8bbeee 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -1,65 +1,317 @@ name: Deploy Workers +# 通用 deploy workflow:掃描 repo 內所有含 wrangler.toml 的目錄 → matrix fanout 部署 +# 新增 Worker = 新目錄 + wrangler.toml + (src/index.ts | component.wasm 等) + pnpm-lock.yaml +# 不用改本檔。詳見 .claude/rules/05-deploy-convention.md。 + on: push: branches: [main] - paths: - - 'cypher-executor/**' - - 'registry/**' - - 'credentials/**' - workflow_dispatch: # 允許手動觸發,部署全部三個 + # 不設 paths:由 discover job 用 git diff 過濾,push 都會觸發讓 workflow 判斷 + workflow_dispatch: + inputs: + force_all: + description: "Deploy all Workers regardless of diff" + type: boolean + default: false + only: + description: "Comma-separated Worker dirs to deploy (overrides diff; empty = auto)" + type: string + default: "" + +concurrency: + group: deploy-${{ github.ref }} + cancel-in-progress: false jobs: - deploy-cypher-executor: - name: Deploy cypher-executor + # ── Job 1:掃描所有 wrangler.toml 目錄,輸出 deploy matrix ────────────── + # 分兩層: + # tier1 = .component-builds/*(零件 Worker,互不相依,全部平行) + # tier2 = 其他(cypher-executor/registry/builtins,可能透過 service binding 相依於 tier1) + # tier1 全綠後才啟動 tier2,避免 service binding target 未存在造成首次部署失敗。 + discover: + name: Discover Workers runs-on: ubuntu-latest + outputs: + tier1: ${{ steps.emit.outputs.tier1 }} + tier2: ${{ steps.emit.outputs.tier2 }} + tier1_count: ${{ steps.emit.outputs.tier1_count }} + tier2_count: ${{ steps.emit.outputs.tier2_count }} steps: - uses: actions/checkout@v5 - - uses: actions/setup-node@v5 with: - node-version: '20' - - name: Install deps - working-directory: cypher-executor - run: npm install --legacy-peer-deps - - name: Deploy - working-directory: cypher-executor - run: npx wrangler deploy - env: - CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} - CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + # diff 判斷需要前一個 commit + fetch-depth: 2 - deploy-registry: - name: Deploy registry - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v5 - - uses: actions/setup-node@v5 - with: - node-version: '20' - - name: Install deps - working-directory: registry - run: npm install --legacy-peer-deps - - name: Deploy - working-directory: registry - run: npx wrangler deploy + - name: Enumerate & filter + id: emit env: - CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} - CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + FORCE_ALL: ${{ github.event.inputs.force_all }} + ONLY: ${{ github.event.inputs.only }} + EVENT: ${{ github.event_name }} + run: | + set -euo pipefail - deploy-credentials: - name: Deploy credentials + # 所有含 wrangler.toml 的 Worker 目錄,排除: + # - node_modules/ + # - wrangler.test.toml(測試用) + # - Pages 專案(含 pages_build_output_dir,另有 build pipeline,不適用本 workflow) + mapfile -t all_dirs < <( + find . -type f -name 'wrangler.toml' \ + -not -path '*/node_modules/*' \ + -not -name 'wrangler.test.toml' \ + | while read f; do + # 排除 Pages 專案(wrangler pages deploy 與 wrangler deploy 流程不同) + if grep -q 'pages_build_output_dir' "$f"; then + continue + fi + dirname "$f" + done \ + | sort -u \ + | sed 's|^\./||' + ) + + echo "Found ${#all_dirs[@]} worker dirs" + for d in "${all_dirs[@]}"; do echo " - $d"; done + + # 決定要部署哪些 + declare -a targets=() + + if [[ "$EVENT" == "workflow_dispatch" && -n "$ONLY" ]]; then + # 手動觸發 + 指定清單 + IFS=',' read -ra req <<< "$ONLY" + for r in "${req[@]}"; do + r="${r// /}" + for d in "${all_dirs[@]}"; do + if [[ "$d" == "$r" ]]; then + targets+=("$d") + fi + done + done + elif [[ "$EVENT" == "workflow_dispatch" && "$FORCE_ALL" == "true" ]]; then + targets=("${all_dirs[@]}") + elif [[ "$EVENT" == "push" ]]; then + # diff 過濾:哪些 worker 目錄有變動? + # 也要連動 registry/components/{name}/ — 改 main.go 應該 redeploy .component-builds/{name}/ + base_sha="${{ github.event.before }}" + head_sha="${{ github.sha }}" + # 若 base 為 0000...(首次 push)或 base 不可及,fallback 為全部 + if [[ -z "$base_sha" || "$base_sha" == "0000000000000000000000000000000000000000" ]]; then + echo "No base sha, deploy all" + targets=("${all_dirs[@]}") + else + # 取得 diff 的檔案路徑 + mapfile -t changed < <(git diff --name-only "$base_sha" "$head_sha" || true) + echo "Changed files:" + for f in "${changed[@]}"; do echo " $f"; done + + for d in "${all_dirs[@]}"; do + # 判斷:若 d 下任何檔案變動,或 d 是 .component-builds/{name} 且 registry/components/{name}/ 下變動 + hit=0 + for f in "${changed[@]}"; do + if [[ "$f" == "$d"/* ]]; then hit=1; break; fi + done + if [[ $hit -eq 0 && "$d" == .component-builds/* ]]; then + name="${d#.component-builds/}" + for f in "${changed[@]}"; do + if [[ "$f" == "registry/components/$name"/* ]]; then hit=1; break; fi + done + fi + if [[ $hit -eq 1 ]]; then targets+=("$d"); fi + done + fi + else + targets=("${all_dirs[@]}") + fi + + echo "Deploying ${#targets[@]} workers (will be split into 2 tiers):" + for t in "${targets[@]}"; do echo " - $t"; done + + # 分成兩層: + # tier1 = .component-builds/*(零件 Worker,需要 WASM build) + # tier2 = 其他(orchestration Worker,可能有 service binding 相依於 tier1) + emit_json() { + local -n arr=$1 + local out="[" + local first=1 + for t in "${arr[@]}"; do + local name needs_wasm + name="$(basename "$t")" + needs_wasm="false" + if [[ "$t" == .component-builds/* ]]; then + needs_wasm="true" + name="${t#.component-builds/}" + fi + if [[ $first -eq 0 ]]; then out+=","; fi + out+="{\"name\":\"$name\",\"path\":\"$t\",\"needsWasm\":$needs_wasm}" + first=0 + done + out+="]" + echo "$out" + } + + declare -a tier1=() tier2=() + for t in "${targets[@]}"; do + if [[ "$t" == .component-builds/* ]]; then + tier1+=("$t") + else + tier2+=("$t") + fi + done + + tier1_json=$(emit_json tier1) + tier2_json=$(emit_json tier2) + + echo "Tier 1 (${#tier1[@]}):" + for t in "${tier1[@]}"; do echo " - $t"; done + echo "Tier 2 (${#tier2[@]}):" + for t in "${tier2[@]}"; do echo " - $t"; done + + echo "tier1=$tier1_json" >> "$GITHUB_OUTPUT" + echo "tier2=$tier2_json" >> "$GITHUB_OUTPUT" + echo "tier1_count=${#tier1[@]}" >> "$GITHUB_OUTPUT" + echo "tier2_count=${#tier2[@]}" >> "$GITHUB_OUTPUT" + + # ── Job 2a:Tier 1 並行部署(零件 Worker,需要 WASM build) ─────────────── + # tier1 所有 Worker 互不相依,全部平行;tier1 全綠後才啟動 tier2。 + deploy-tier1: + name: Deploy tier1/${{ matrix.worker.name }} + needs: discover + if: needs.discover.outputs.tier1_count != '0' runs-on: ubuntu-latest + strategy: + fail-fast: false + max-parallel: 5 + matrix: + worker: ${{ fromJson(needs.discover.outputs.tier1) }} steps: - uses: actions/checkout@v5 + + - uses: pnpm/action-setup@v4 + with: + version: 10 + - uses: actions/setup-node@v5 with: - node-version: '20' + node-version: '22' + cache: 'pnpm' + cache-dependency-path: ${{ matrix.worker.path }}/pnpm-lock.yaml + + - name: Setup TinyGo + uses: acifani/setup-tinygo@v2 + with: + tinygo-version: '0.40.1' + binaryen-version: '116' + + - name: Rebuild component.wasm from source + working-directory: registry/components/${{ matrix.worker.name }} + run: | + set -euo pipefail + if [[ ! -f main.go ]]; then + echo "no main.go at registry/components/${{ matrix.worker.name }}/ — skipping rebuild" + exit 0 + fi + tinygo build -target=wasi -o "${{ matrix.worker.name }}.wasm" main.go + ls -lh "${{ matrix.worker.name }}.wasm" + + - name: Copy .wasm into Worker build dir + run: | + set -euo pipefail + src="registry/components/${{ matrix.worker.name }}/${{ matrix.worker.name }}.wasm" + dst="${{ matrix.worker.path }}/component.wasm" + if [[ -f "$src" ]]; then + cp "$src" "$dst" + echo "Copied $src → $dst" + else + echo "WARNING: $src not found, using existing $dst" + fi + - name: Install deps - working-directory: credentials - run: npm install --legacy-peer-deps + working-directory: ${{ matrix.worker.path }} + run: | + if [[ -f pnpm-lock.yaml ]]; then + pnpm install --frozen-lockfile + else + echo "no pnpm-lock.yaml at ${{ matrix.worker.path }} — running pnpm install (no lock)" + pnpm install --no-frozen-lockfile + fi + - name: Deploy - working-directory: credentials - run: npx wrangler deploy + working-directory: ${{ matrix.worker.path }} env: CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: pnpm exec wrangler deploy + + # ── Job 2b:Tier 2 並行部署(orchestration Worker,可能有 service binding 相依於 tier1) ─ + # needs: deploy-tier1 → tier1 全綠才開始;首次部署時避免 service binding target 未存在。 + deploy-tier2: + name: Deploy tier2/${{ matrix.worker.name }} + needs: [discover, deploy-tier1] + # tier2 也要跑:即使 tier1 沒東西(tier1_count=0)也要跑 tier2 + if: | + always() && + needs.discover.outputs.tier2_count != '0' && + (needs.deploy-tier1.result == 'success' || needs.deploy-tier1.result == 'skipped') + runs-on: ubuntu-latest + strategy: + fail-fast: false + max-parallel: 5 + matrix: + worker: ${{ fromJson(needs.discover.outputs.tier2) }} + steps: + - uses: actions/checkout@v5 + + - uses: pnpm/action-setup@v4 + with: + version: 10 + + - uses: actions/setup-node@v5 + with: + node-version: '22' + cache: 'pnpm' + cache-dependency-path: ${{ matrix.worker.path }}/pnpm-lock.yaml + + - name: Install deps + working-directory: ${{ matrix.worker.path }} + run: | + if [[ -f pnpm-lock.yaml ]]; then + pnpm install --frozen-lockfile + else + echo "no pnpm-lock.yaml at ${{ matrix.worker.path }} — running pnpm install (no lock)" + pnpm install --no-frozen-lockfile + fi + + - name: Deploy + working-directory: ${{ matrix.worker.path }} + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: pnpm exec wrangler deploy + + # ── Job 3:彙總結果 ──────────────────────────────────────────────────── + summary: + name: Summary + needs: [discover, deploy-tier1, deploy-tier2] + if: always() + runs-on: ubuntu-latest + steps: + - name: Report + run: | + { + echo "## Deploy Summary" + echo "- Tier 1 count: ${{ needs.discover.outputs.tier1_count }}" + echo "- Tier 1 result: ${{ needs.deploy-tier1.result }}" + echo "- Tier 2 count: ${{ needs.discover.outputs.tier2_count }}" + echo "- Tier 2 result: ${{ needs.deploy-tier2.result }}" + echo "" + echo "### Tier 1 (WASM components)" + echo '```json' + echo '${{ needs.discover.outputs.tier1 }}' + echo '```' + echo "" + echo "### Tier 2 (orchestration)" + echo '```json' + echo '${{ needs.discover.outputs.tier2 }}' + echo '```' + } >> "$GITHUB_STEP_SUMMARY"