compare-evaluation-protocols
Build a protocol-difference matrix and estimate which differences can materially change measured performance.
Skill by yogsoth-ai
Chalk Labs
Need help deploying compare-evaluation-protocols?
Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.
Install
Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install skill/yogsoth-ai-compare-evaluation-protocols
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.
- Tools
- No tools found
- Secrets
- None asked for
- Install scripts
- None found
Network hosts it talks to 1
codeload.github.com· found in install command
Files it touches
~/.claude/skills/compare-evaluation-protocols
What's inside
- Skill name
compare-evaluation-protocols- Description
- Build a protocol-difference matrix and estimate which differences can materially change measured performance.
- License
- Apache-2.0
- Instructions
- 43 lines in SKILL.md · view on GitHub
Files (1)
- SKILL.md
SKILL.md (first lines)
# compare-evaluation-protocols ## Purpose Build a protocol-difference matrix and estimate which differences can materially change measured performance. ## Input contract ```yaml required: [protocol_records, metric_schema, comparison_target] optional: [paired_results, sensitivity_assumptions] constraints: [differences require explicit protocol fields and a comparable outcome] ``` ## Procedure 1. Extract protocol elements into a normalized comparison schema. 2. Align datasets, populations, metrics, baselines, and evaluation conditions. 3. Mark differences and assess their plausible performance effect. 4. Separate observed effects from unresolved protocol confounding. ## Output contract ```yaml produces: [protocol_difference_matrix, materiality_assessment, confounding_notes, comparability_judgment] delta_fields: [findings, evidence_updates, uncertainties, open_questions] ``` ## Quality gates - The compared metric and target are held constant or explicitly qualified. - Missing protocol fields remain visible. ## Failure and counterexamples Do not attribute score differences to method quality when protocol differences are unmeasured. ## Provenance map - `resolved: knowledge-acquisition-evaluation-protocol-comparison`
From yogsoth-ai/de-anthropocentric-research-engine, Apache-2.0.
More from yogsoth-ai/de-anthropocentric-research-engine
- 0.602score
falsification-first-audit
yogsoth-ai
Convert a sharp scientific claim into the cheapest decisive falsification program.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
design-falsification-test
yogsoth-ai
Specify a concrete observation, computation, intervention, counterexample, or measurement that would count against the sharpened claim, prioritizing cheap…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
statistical-testing
yogsoth-ai
Apply the pre-declared statistical/estimation procedure and report effect size, uncertainty, model checks, and decision-relevant interpretation.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
estimate-effective-evidence-count
yogsoth-ai
Estimate an effective number of independent evidence/reasoning paths after discounting dependence/correlation; report nominal N, dependence structure, and…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
adjust-abstraction-scope
yogsoth-ai
Move a scientific object up/down in abstraction or narrow/broaden selected scope dimensions (population, mechanism, context, outcome, timeframe, system…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
adversarial-deliberation
yogsoth-ai
Run structured attack/defense/adjudication over a claim, candidate, criterion set, or current winner.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analyze-heterogeneity
yogsoth-ai
Identify clinical, methodological, and statistical heterogeneity sources and prioritize subgroup/meta-regression/outlier investigation.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analogical-discovery
yogsoth-ai
Abstract relational structure from source domains, map it to the target, validate depth, and instantiate transferable mechanisms.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analyze-constraints-readiness
yogsoth-ai
Characterize what blocks a research target: readiness dimensions, obstacles, bottlenecks, resource gaps, causal constraints, dependencies, conflicts…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analyze-experiment-results
yogsoth-ai
Interpret completed experimental outputs after host/runtime execution using pre-declared statistical tests, effect/uncertainty estimates, reproducibility…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analyze-future-scenarios
yogsoth-ai
Construct and compare plausible future/competitive/temporal/stress scenarios and test whether the research path remains robust across them.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
analyze-intervention
yogsoth-ai
Apply an intervention/do-style graph modification and propagate predicted directional effects while recording assumptions.
SkillAFreshness: A★ 505Claude Code +9
Signals
- Stars
- 505
- Forks
- 43
- Open issues
- 1
- Last commit (this folder)
- Sep 23, 2026 (1 month ago)
- Last release
- Oct 23, 2026 (7 days ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- Apache-2.0
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- GitHubgithub-graphql
More from yogsoth-ai
- 0.602score
falsification-first-audit
yogsoth-ai
Convert a sharp scientific claim into the cheapest decisive falsification program.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
design-falsification-test
yogsoth-ai
Specify a concrete observation, computation, intervention, counterexample, or measurement that would count against the sharpened claim, prioritizing cheap…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
estimate-effective-evidence-count
yogsoth-ai
Estimate an effective number of independent evidence/reasoning paths after discounting dependence/correlation; report nominal N, dependence structure, and…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
statistical-testing
yogsoth-ai
Apply the pre-declared statistical/estimation procedure and report effect size, uncertainty, model checks, and decision-relevant interpretation.
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
adjust-abstraction-scope
yogsoth-ai
Move a scientific object up/down in abstraction or narrow/broaden selected scope dimensions (population, mechanism, context, outcome, timeframe, system…
SkillAFreshness: A★ 505Claude Code +9 - 0.600score
adversarial-deliberation
yogsoth-ai
Run structured attack/defense/adjudication over a claim, candidate, criterion set, or current winner.
SkillAFreshness: A★ 505Claude Code +9
Done for you
Rather not set it up yourself?
Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.
See how we build AI agents →Read the case studies
Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.
- launches shipped
- 50+
- launches shipped
- media placements
- 500+
- media placements
- founders on every account
- 2
- founders on every account
Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block
Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.
Get compare-evaluation-protocols running in your business
Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.
Cite this page
compare-evaluation-protocols skill. Agent Mart. https://agentmart.dev/skills/yogsoth-ai-compare-evaluation-protocols (last updated 2026-10-24)