google-search Skill Evaluation Report¶

Evaluation framework: skill-creator Evaluation date: 2026-03-12 Subject: google-search

google-search is a research/search skill that turns "help me search for this" into a verifiable search workflow. It suits fact lookups, error debugging, official docs retrieval, technology comparisons, and public-information gathering that needs source support. Its three main strengths are: classifying the question, defining the evidence chain, and choosing the mode first—elevating search from "finding links" to "finding evidence for conclusions"; outputs include confidence, source tier, budget status, and reusable queries so the search process is reviewable and continuable; and it emphasizes execution completeness and degradation declarations, clearly distinguishing "verified conclusions" from "partial results with insufficient evidence".

1. Evaluation Overview¶

This evaluation assesses the google-search skill along two dimensions: actual task performance and Token cost-effectiveness. It uses 3 search scenarios of increasing complexity (Quick-mode fact lookup, Standard-mode error debugging, Deep-mode framework comparison). Each scenario runs with both with-skill and without-skill configurations, for 3 scenarios × 2 configurations = 6 independent subagent runs, scored against 27 assertions.

Key finding: The core value of the google-search skill is search discipline and report structure, not search content quality. The base model already has strong search and synthesis ability (answer correctness, source coverage, code example quality all good), but lacks metadata for the search process (mode choice, budget control, evidence chain tracking, degradation declaration, confidence labels, reusable queries). The skill fills this "search operation discipline" gap.

2. Test Methodology¶

2.1 Scenario Design¶

2.2 Execution¶

• With-skill runs first read SKILL.md and related references (query-patterns, programmer-search-patterns, source-evaluation, etc.)
• Without-skill runs read no skill; search follows model default behavior
• All runs may use WebSearch and WebFetch tools
• 6 subagents run in parallel (with-skill uses default model, without-skill uses fast model)

2.3 Skill Characteristics¶

google-search is a multi-file skill (1 SKILL.md + 6 reference files) with conditional loading.

Actual load per scenario:

3. Assertion Pass Rate¶

3.1 Summary¶

3.2 Item-by-Item Scoring¶

Eval 1: Go database/sql default pool size (Quick mode)¶

Eval 2: gRPC context deadline exceeded (Standard mode)¶

Eval 3: Go HTTP framework comparison (Deep mode)¶

3.3 Classification of 20 Without-Skill Failed Assertions¶

Note: As with the deep-research evaluation, all 20 failures are search discipline / report format failures, not content quality failures. Without-skill passed on answer correctness, source coverage, and code examples.

3.4 Comparison with deep-research Skill¶

google-search has a larger assertion delta because it requires not only a report template (deep-research’s 7-section) but also search process metadata (mode, budget, evidence chain, degradation level, reusable queries, precise query strategies). The base model does not produce these concepts at all.

4. Dimension-by-Dimension Comparison¶

4.1 Output Contract (8 Fields)¶

Practical value: - Degradation level showed highest value in Eval 3—With-skill honestly declared "Partial" (TechEmpower data from third-party interpretation, no named company production cases), while Without-skill gave conclusions without marking uncertainty - Evidence chain status lets readers track "which evidence is satisfied, which is missing", avoiding treating partial data as complete conclusions - Reusable queries give readers the ability to "continue searching"—5 well-designed Google queries are more lasting value than a single answer

4.2 Search Strategy Discipline¶

4.3 Confidence + Source-Tier Labels¶

Eval 3 With-skill output labeled all 14 key numbers with dual labels:

| Fiber real-world RPS | ~36,000 | May 2024 | Medium | Primary (independent benchmark) |
| Fiber JSON RPS (TechEmpower R23) | ~735,000 | March 2025 | Low | Third-party interpretation of Official |

This distinguishes "Medium confidence from Primary source" from "Low confidence from Third-party interpretation", so readers know TechEmpower data is secondhand and downgraded. Without-skill Eval 3 cited 16 sources and many numbers but no number had credibility or source-tier labels.

4.4 Honest Degradation¶

Eval 3 With-skill output best illustrates this mechanism:

Degradation Level: Partial — Strong benchmark data and ecosystem analysis available. However: TechEmpower Round 23 Go-specific per-framework numbers could not be directly verified from TechEmpower's own site... Large-scale production experience reports... were not found from named companies with disclosed architectures.

This degradation statement clearly informs readers of two specific uncertainties, avoiding treating the comparison as fully confirmed fact. Without-skill Eval 3 also found no named company cases but did not declare this limitation.

4.5 Content Quality Comparison¶

Key conclusion: Consistent with the deep-research skill evaluation—the base model is already strong on content; the skill’s increment is entirely in search discipline and report metadata.

5. Token Cost-Effectiveness Analysis¶

5.1 Skill Size¶

5.2 Token Cost for Quality Gain¶

5.3 Token Segment Cost-Effectiveness¶

5.4 High-Leverage vs Low-Leverage Instructions¶

High leverage (~1,050 tokens → 18 assertion delta): - Output Contract 8-field definition (~300 tok → 6) - Confidence + Source-tier dual-label rules (~200 tok → 3) - Reusable Queries requirement (~100 tok → 3) - Evidence Chain Gate (~300 tok → 3) - Source Assessment requirement (~150 tok → 3)

Medium leverage (~5,150 tokens → 2 direct + indirect): - query-patterns.md (~1,800 tok → 2 + indirect search quality) - programmer-search-patterns.md (~1,500 tok → indirect) - source-evaluation.md (~1,400 tok → indirect) - Other Gates (~450 tok → indirect)

Low leverage (~800 tokens → 0 direct delta): - Worked Examples (~500 tok) - Anti-Examples (~300 tok)

5.5 Comparison with Other Skills’ Cost-Effectiveness¶

google-search has the highest absolute pass-rate gain (+74.1%), but SKILL.md-level unit cost-effectiveness (~42 tok/1%) is similar to yt-dlp-downloader (~43) and tdd-workflow (~45). Typical-load cost-effectiveness (~105 tok/1%) is higher due to more reference files.

6. Boundary Analysis vs Base Model Capabilities¶

6.1 Base Model Capabilities (No Skill Increment)¶

6.2 Base Model Gaps (Skill Fills)¶

Core finding: The base model’s "search results → answer" ability is strong, but "search process auditability" and "conclusion credibility labeling" are zero. The google-search skill’s value lies in the latter two.

7. Overall Score¶

7.1 Dimension Scores¶

7.2 Weighted Total¶

The lower Token cost-effectiveness score (7.0/10) reflects higher typical load (~7,800 tok) from more reference files, even though SKILL.md cost-effectiveness (~42 tok/1%) is comparable to peer skills.

8. Evaluation Artifacts¶