product-analytics

Files (6)

• README.md
• SKILL.md
• evals.json
• feature-adoption.md · references/
• funnel-methodology.md · references/
• metrics-catalog.md · references/

SKILL.md (first lines)

## Key principles

1. **Instrument before you need data** - Tracking is a prerequisite, not an afterthought.
   Add instrumentation when a feature ships, not when a stakeholder asks "do we track that?"
   Retrofitting events means losing the baseline period and the ability to compare pre/post.

2. **Define metrics before building features** - Before writing a line of code, agree on
   what success looks like and how it will be measured. A feature without a success metric
   cannot be evaluated and cannot be killed. Write the metric definition into the spec.

3. **Segment everything** - Aggregate numbers hide the truth. Always break down metrics
   by user segment (new vs. returning, plan tier, acquisition channel, geography) before
   drawing conclusions. An overall retention rate that looks healthy can mask a collapsing
   new-user cohort.

4. **Retention is the ultimate metric** - Acquisition and activation are table stakes.
   Retention - whether users come back and get repeated value - is the only signal that
   proves product-market fit. A product with strong retention can fix acquisition; a
   product with broken retention cannot be saved by growth spend.

5. **Correlation requires investigation, not celebration** - Two metrics moving together
   is a hypothesis, not a conclusion. Before attributing causation, check for confounders,
   test the relationship with a controlled experiment, and document the evidence. Acting
   on spurious correlations wastes engineering capacity and can harm users.

---

## Core concepts

### Event taxonomy

Events are the atoms of product analytics. An event represents a discrete user action
(or system action) at a point in time. A well-designed taxonomy makes querying intuitive
and avoids the "event graveyard" where hundreds of poorly named events accumulate with
no documentation.

**Naming convention:** `object_action` in snake_case. The object is the thing being acted
on; the action is what happened.

```
user_signed_up
dashboard_viewed
report_exported
onboarding_step_completed
subscription_upgraded
```

Every event should carry a consistent set of properties:
- `user_id` - anonymous or authenticated identifier
- `session_id` - groups events within a single session
- `timestamp` - ISO 8601, always UTC
- `platform` - web, ios, android, api
- `event_version` - allows schema evolution without breaking queries

Entity-specific properties are added per event:
```
report_exported:
  report_type: "weekly_summary"
  format: "csv"
  row_count: 1450
```

### Funnel analysis

A funnel is an ordered sequence of steps a user must complete to reach a goal. Funnel
analysis reveals where users drop off and quantifies the conversion loss at each step.

**Key measurements:**
- **Step conversion rate** - users who completed step N+1 / users who completed step N
- **Overall conversion rate** - users who completed the final step / users who entered step 1
- **Time-to-convert** - median and 90th percentile time between first and last step
- **Drop-off point** - the step with the steepest conversion decline

Funnels should be analyzed with a defined window (e.g., within 7 days, within a single
session) to avoid counting users who convert months later by coincidence.

**Common funnels by product type:**

| Product type | Acquisition funnel | Activation funnel |
|---|---|---|
| SaaS | Landing page -> Sign up -> Verify email -> First login | Login -> Create first item -> Invite team member |
| E-commerce | Product page -> Add to cart -> Checkout start -> Purchase | First purchase -> Second purchase within 30 days |
| Marketplace | Search -> Listing view -> Contact/Bid -> Transaction | First transaction -> Second transaction |

### Cohort analysis

A cohort is a group of users who share a defining characteristic at a specific point in
time - most commonly the week or month they first signed up. Cohort analysis tracks how
that group's behavior evolves over time.

**Retention cohort table:**

```
         Week 0  Week 1  Week 2  Week 3  Week 4
Jan W1:  100%    42%     31%     27%     25%
Jan W2:  100%    38%     29%     26%     24%
Jan W3:  100%    45%     34%     30%     28%
```

Reading across a row shows how a specific cohort retains over time. Reading down a column
shows whether a given time-since-signup period is improving or degrading across cohorts.
Improvement down a column - newer cohorts retaining better than older ones - is the
strongest early signal that product improvements are working.

**Behavioral cohorts** group users by an action rather than signup date (e.g., users who
completed onboarding vs. those who skipped it). Comparing behavioral cohorts quantifies
the impact of a specific behavior on downstream retention.

### Retention curves

A retention curve plots the percentage of a cohort that remains active over successive
time periods. The shape of the curve matters as much as the final number.

**Curve shapes:**

- **Flat decay to zero** - all users eventually churn; the product has no habit-forming
  loop. Fundamental product problem.
- **Decaying to a stable floor** - some users churn, but a core group stays. The floor
  percentage is the product's "true retention." The goal is to raise the floor.

From mkurman/zorai, MIT.

More from mkurman/zorai