Skip to content
Agentmart
Submit a tool

method-candidate-ab-validity-trigger-confound

A/B qualification candidate distilled verbatim by Qwen3.8-27B from the working-set receipts teacher episode; experimental candidate only, not promoted.

Skill by deyi2026

IndexedApache-2.0★ 1
Write a review

Chalk Labs

Need help deploying method-candidate-ab-validity-trigger-confound?

Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.

Book a call

Install

Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install skill/deyi2026-method-candidate-ab-validity-trigger-confound

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

A/B qualification candidate distilled verbatim by Qwen3.8-27B from the working-set receipts teacher episode; experimental candidate only, not promoted.

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.

Tools
No tools found
Secrets
None asked for
Install scripts
None found

Network hosts it talks to 1

  • codeload.github.com · found in install command

Files it touches

  • ~/.claude/skills/method-candidate-ab-validity-trigger-confound

What's inside

Skill name
method-candidate-ab-validity-trigger-confound
Description
A/B qualification candidate distilled verbatim by Qwen3.8-27B from the working-set receipts teacher episode; experimental candidate only, not promoted.
License
Apache-2.0
Instructions
36 lines in SKILL.md · view on GitHub

Files (1)

  • SKILL.md

SKILL.md (first lines)

# Qwen-authored candidate (verbatim)

FRICTION
- 首个 OFF 因环境缺 providers.json 走 unknown provider,属 invalid arm,不能参与 A/B 结论。
- ON 的 fold 到第 15 个大 read_file 才真实触发,前段与 OFF 行为近似,机制未早触发。
- elapsed/cache-hit 的 ON 是第二个 run,同物理模型且无安全 cache clear,存在 warm/order confound。

DISCRIMINATOR
- 有效 arm 判据:同 model/prompt SHA/tools/reasoning settings + 独立 session/data dir + 环境可解析 provider。
- 机制触发判据:必须观测到 fold 实际发生(folded_results 非空/达到阈值),而非仅配置开启。
- 收益判据:任务正确性(4/4、15/15)与收敛轮次/token 分开看;elapsed/cache 在 warm/order 未控时只作 observation。

SHORTEST_PATH
- 先做 arm 有效性预检(provider 可解析、配置 SHA 一致、独立目录),失败即标 invalid 不进入对比。
- 在 ON 侧埋点确认机制真实触发点(第几次 read 触发 fold、folded 数量),未触发则该 run 对机制无效。
- 对 elapsed/cache 类指标,要么做 cache clear/顺序平衡,要么降级为 observation,不用于 promotion 决策。

GENERAL_RULE
- A/B 结论前,先排除 invalid arm(环境/配置不可解析)与未触发机制的 run,再谈收益。
- 机制收益需“触发证据 + 任务正确性 + 收敛/成本”三者齐备;任一缺失则 mixed/不足 promotion。
- 时序相关指标(elapsed、cache-hit)在 warm/order 未受控时不得作为因果或 promotion 依据。

COUNTEREXAMPLE
- 若把首个 unknown-provider 的 OFF 当作有效基线,会高估 ON 的相对收益。
- 若只看 ON 配置开启就断言 fold 生效,会忽略前 14 次 read 未触发的空窗。
- 若把第二个 run 的 elapsed/cache 优势归因于 ON,会混入 warm/order confound。

METHOD CARD
- 名称:working-set receipts A/B 蒸馏
- 适用:验证某机制(如 fold)是否带来真实收益的 A/B
- 步骤:1) arm 预检(provider/SHA/独立目录)2) 埋点确认机制触发 3) 分维度判收益
- 判据:机制 PASS=触发证据;任务 PASS=正确性+可恢复性;收益=收敛/成本且无 confound
- 降级:elapsed/cache 未控 warm/order 时仅 observation
- 反例:invalid arm 入对比、未触发即断言、warm run 归因
- 产出:PASS/mixed/不足 promotion 三态结论,附触发点与 confound 说明

From deyi2026/llm-first-loop, Apache-2.0.

More from deyi2026/llm-first-loop

Signals

Stars
1
Forks
0
Open issues
0
Last commit (this folder)
Sep 8, 2026 (1 month ago)
Last release
Oct 22, 2026 (8 days ago)

From GitHub (fetched Oct 24, 2026).

Details

License
Apache-2.0
First seen
Oct 24, 2026
Last checked
Oct 24, 2026

More from deyi2026

Done for you

Rather not set it up yourself?

Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.

See how we build AI agents →Read the case studies

Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.

launches shipped
50+
launches shipped
media placements
500+
media placements
founders on every account
2
founders on every account

Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block

Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.

Get method-candidate-ab-validity-trigger-confound running in your business

Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.

Goes to the Chalk Labs founders, not the newsletter. Privacy

Need help deploying method-candidate-ab-validity-trigger-confound?

CallGet help
Cite this page

method-candidate-ab-validity-trigger-confound skill. Agent Mart. https://agentmart.dev/skills/deyi2026-method-candidate-ab-validity-trigger-confound (last updated 2026-10-24)