experiment

Files (1)

• SKILL.md

SKILL.md (first lines)

# Create Experiment

Create an experiment protocol to test a hypothesis in HDD.

**Goal:** Validate an enhancement BEFORE merging to main. Only proven improvements ship.

## Overview

An experiment is a **pre-registered protocol** for testing a hypothesis:
- Methods (what you'll do)
- Measures (what you'll measure)
- Data location (where results stored)
- Analysis plan (how to interpret)
- **Branch**: The feature branch with the implementation

**Pre-registration** = commit protocol BEFORE running. Prevents HARKing.

## The Feedback Loop

**Every experiment runs on a feature branch.** Based on results:

| Outcome | Action | Software Impact |
|---------|--------|-----------------|
| **VALIDATED** | Merge branch | Enhancement ships |
| **REFUTED** | Don't merge | We learned why |
| **NEEDS-MORE-DATA** | Extend | Keep branch open |

## Steps

### 1. Create Feature Branch

```bash
git checkout -b exp-{expedition}-{description}
# Example: git checkout -b exp-042-redis-caching
```

### 2. Implement the Enhancement

Build the feature you want to test.

### 3. Create Experiment File

Prefer the CLI, which allocates the id for you. A hypothesis is **optional** —
an experiment can stand on its own, and you should not invent a hypothesis (or
a paper) so the command will run:

```bash
yurtle-kanban experiment create --title "$TITLE"                          # standalone
yurtle-kanban experiment create --title "$TITLE" --hypothesis H130.1      # testing one
```

If you write the file by hand, note that the id is `EXPR-{nnn}` — its **own**
number, never the paper's — and `paper:`/`hypothesis:` are omitted entirely
when there is none:

```markdown
---
id: EXPR-001
type: experiment
status: draft
created: YYYY-MM-DD
---

# EXPR-001: $TITLE

## Purpose

[What hypothesis this tests and why it matters]

## Pre-Registration

**Locked before experiment runs:**

| Element | Value | Git Hash |
|---------|-------|----------|
| Hypothesis | H{paper}.{n} | TBD |
| Primary Outcome | [measure] | TBD |
| Target | [threshold] | TBD |
| Analysis Plan | [how to interpret] | TBD |
| Pre-registered | YYYY-MM-DD | TBD |

## Hypotheses Tested

| ID | Statement | Target | Status |
|----|-----------|--------|--------|
| H{paper}.{n} | [statement] | [target] | PENDING |

## Method

[Detailed description of what you'll do]

### Procedure

1. [Step 1]
2. [Step 2]
3. [Step 3]

### Measures

| Measure | ID | Unit | How Collected |
|---------|----|------|---------------|
| [Name] | M-XXX | [unit] | [method] |

## Run Command

```bash
# Example run command
./run_experiment.sh --experiment EXPR-{nnn}
```

## Results

| Metric | Target | Actual | Status |
|--------|--------|--------|--------|
| [measure] | [target] | TBD | PENDING |

## Data Location

`research/experiments/EXPR-{nnn}/data/`

From Congruentsys/yurtle-kanban, MIT.

More from Congruentsys/yurtle-kanban