go-ci-workflow Skill Evaluation Report¶

Evaluation date: 2026-03-12 Evaluation subject: go-ci-workflow Evaluation method: skill-creator framework (3 scenarios × 2 configs = 6 runs)

Reference baseline: issue2md project (https://github.com/johnqtcg/issue2md) real CI workflow (.github/workflows/ci.yml)

go-ci-workflow is a GitHub Actions CI design and refactoring skill for Go repositories. It generates honest, maintainable CI workflows that match local execution based on repo structure, Makefile entry points, and test types. Its three main strengths are: repository shape detection before deciding workflow architecture, avoiding forcing unsuitable CI templates onto repos; strong Make-driven delegation with explicit fallback when stable entry points are missing, so "how you run locally is how CI runs"; and unified conventions for tool version pinning, output contracts, and local-equivalence markers for long-term maintenance and debugging.

1. Evaluation Overview¶

This evaluation reviews the go-ci-workflow skill along two dimensions: actual task performance and token cost-effectiveness. It uses the real Makefile and CI workflow of the issue2md project as a reference baseline, with 3 progressive scenarios and 35 assertions.

2. Test Methodology¶

2.1 Scenario Design¶

2.2 Reference Baseline¶

issue2md real CI workflow characteristics: - 6 independent jobs: ci, docker-build, api-integration, e2e-web, govulncheck, fieldalignment - Core gate delegates via make ci COVER_MIN=80 - E2E only on push/schedule - Tool versions aligned with Makefile - No concurrency (improvement opportunity)

3. Assertion Pass Rate¶

3.1 Scenario 1: Create CI from Scratch (15 items)¶

3.2 Scenario 2: Refactor Poor CI (12 items)¶

3.3 Scenario 3: No-Makefile Go Library (8 items)¶

3.4 Overview¶

3.5 Trend: Skill Advantage Inversely Related to Prompt Information¶

Conclusion: When the prompt contains enough structural information, the baseline approaches the skill. When prompt information is low (e.g. scenario 3), the baseline lacks the skill’s degradation handling, equivalence markers, etc. The skill’s core value is structured knowledge completion, especially best practices not mentioned in the prompt.

4. Comparison with Real CI¶

issue2md has a manually written, high-quality CI workflow. Comparing With-Skill output to real CI:

Findings: - With-Skill output closely matches real CI in job layout and Make delegation - With-Skill improves real CI (adds concurrency and permissions; real CI lacks both) - Without-Skill uses inline Docker build instead of make docker-build, violating local-equivalence - Without-Skill E2E job adds unnecessary server startup logic (curl polling, etc.), increasing complexity

5. Dimension-by-Dimension Comparison¶

5.1 Make-Driven Delegation (Core Delta)¶

The skill’s "Execution Priority" rules ensure consistent Make delegation. The baseline matched in scenario 2 (with explicit prompts) but fell back to inline Docker in scenario 1 (no prompt).

5.2 Output Contract (Skill-Only)¶

With-Skill produces structured reports in each scenario:

Without-Skill has no such structured output.

5.3 Degradation Handling (Scenario 3 Key Delta)¶

Scenario 3 exposes the baseline’s main weakness without structured guidance: - Uses @latest (non-deterministic builds) - Hardcoded Go version - No concurrency - No degradation awareness (does not mark missing Makefile)

5.4 Security and Permissions¶

Both set permissions, but With-Skill uses stricter deny-all default (permissions: {}) + job-level escalation in scenario 1, and explicitly analyzes Fork PR safety in the Output Contract.

6. Token Cost-Effectiveness Analysis¶

6.1 Skill Size¶

6.2 Typical Load Scenarios¶

6.3 Cost-Effectiveness Calculation¶

6.4 Cost-Effectiveness vs Other Skills¶

go-ci-workflow has the best SKILL.md cost-effectiveness (~45 tok/1%) but large reference set (~10,100 tokens), so typical-load cost-effectiveness is worse (~136 tok/1%). Similar to go-makefile-writer.

6.5 Token Segment Cost-Effectiveness¶

High-leverage instructions (~610 tokens SKILL.md → 11.5 assertion delta) are ~41% of SKILL.md and drive all direct deltas.

7. Overall Score¶

7.1 Dimension Scores¶

7.2 Weighted Total Score¶

7.3 Comparison with Other Skills¶

go-ci-workflow scores slightly lower, mainly due to token cost-effectiveness (5.5/10). Reference set ~10,100 tokens is the largest among evaluated skills; typical load ~4,500 tokens also has high cost per 1% (~136 tok/1%).

Score breakdown: - Token cost-effectiveness (5.5/10): References too large. workflow-quality-guide.md (445 lines) and golden-examples.md (385 lines) total ~5,600 tokens but only indirect contribution in eval - Assertion delta (9.0/10): Scenario 2 delta only +8pp (baseline 92%), lowering overall delta

Highlights: - Degradation handling (10.0/10): Scenario 3 +94pp delta is the largest single-scenario delta among evaluated skills, proving Degraded Output Gate value - SKILL.md cost-effectiveness: ~45 tok/1% is best among skills; core rules are compact and efficient

8. Conclusion¶

The go-ci-workflow skill adds clear value in three areas:

1. Degradation handling (+94pp single-scenario delta): The largest single-scenario delta among evaluated skills, proving the value of Degraded Output Gate and Local Parity markers. The baseline has no degradation awareness without a Makefile.

2. Make-driven delegation consistency: Ensures all jobs run via Makefile targets, matching local development. The baseline falls back to inline commands without explicit prompts (e.g. Docker build in scenario 1).

3. Output Contract: Structured reports make CI workflow changes auditable and traceable, including repo shape, execution path classification, missing targets, etc.

Main risk: Reference set ~10,100 tokens is the largest among evaluated skills; typical load ~4,500 tokens. Trimming workflow-quality-guide.md and golden-examples.md could reduce token cost ~24% and improve tokens/1% from ~136 to ~103.

Comparison with real CI validates the skill: With-Skill output not only matches the quality of issue2md’s manually written CI but improves it on concurrency and permissions.