validation

Files (1)

• SKILL.md

SKILL.md (first lines)

## Overview

**This skill is MANDATORY after creating or modifying any Honeydew object.**

Validation ensures:

- The object compiles and executes without errors
- Results are returned (not NULL or empty)
- Values make business sense
- Related objects are consistent with each other

---

## Validation by Object Type

### Metrics

**Step 1: Execute the metric**

Call `get_data_from_fields` with:

- `metrics`: `["<entity>.<metric_name>"]`

If the tool call fails (API error, permission denied, timeout), report the error to the user before proceeding. Do not confuse a tool error with suspicious data.

**Step 2: Sanity checks**

| Check           | What to Look For                      | Action if Failed                                      |
| --------------- | ------------------------------------- | ----------------------------------------------------- |
| Returns data    | Not NULL, not empty                   | Check SQL syntax, entity references                   |
| Magnitude       | Reasonable for business context       | Verify calculation logic                              |
| Sign            | Positive for revenue/counts (usually) | Check for inverted logic                              |
| Related metrics | Parts sum to whole                    | Query related metrics together and verify consistency |
| Ratios          | Between 0-100% (usually)              | Check numerator and denominator metrics independently |

**Step 3: Cross-validation**

If the new metric is a filtered subset of an existing metric, or a ratio of existing metrics, query them together to verify consistency:

- For filtered metrics: query the filtered metric alongside the unfiltered total — the filtered value should be less than or equal to the total.
- For ratio/derived metrics: query the numerator and denominator independently to confirm they return sensible values before checking the ratio.

Call `get_data_from_fields` with both metrics:

- `metrics`: `["<entity>.<filtered_metric>", "<entity>.<total_metric>"]`

**Alert user if:**

- Metric returns $0 or NULL unexpectedly
- Revenue/count is negative
- Ratio exceeds 100% or is negative (unless expected)
- Magnitude seems off by orders of magnitude

---

### Attributes

**Step 1: Execute the attribute**

If the attribute references a related entity (multi-entity attribute), first verify the relation exists using `get_entity` on the source entity.
If the relation is missing, report that before attempting to query the attribute.

Sample rows — call `get_data_from_fields` with:

- `attributes`: `["<entity>.<attribute_name>"]`

For boolean attributes, check distribution — call `get_data_from_fields` with:

- `attributes`: `["<entity>.<boolean_attribute>"]`
- `metrics`: `["<entity>.count"]`

**Step 2: Sanity checks by attribute type**

| Attribute Type      | Valid Range              | Red Flags                           |
| ------------------- | ------------------------ | ----------------------------------- |
| **Age/Duration**    | 0 to reasonable max      | Negative values, >150 years         |
| **Boolean**         | TRUE/FALSE mix           | All TRUE or all FALSE               |
| **Percentage**      | 0-100 (usually)          | Negative, >100 (unless growth rate) |
| **Date**            | Past to near future      | Year 1900, year 2099, all NULLs     |
| **Category/Bucket** | Expected labels          | 95%+ in "Unknown" or NULL           |
| **Rank**            | Starts at 1, consecutive | Starts at 0, gaps, all same value   |
| **Running total**   | Monotonically increasing | Decreases, resets unexpectedly      |
| **Any type**        | Mix of values            | >50% NULLs warrants investigation   |

**Alert user if:**

- All values are NULL or >50% NULLs
- Boolean is 100% one value
- Dates are invalid or in wrong century
- Buckets are mostly "Unknown"

---

### Entities

**Step 1: Verify entity was created**

Use `list_entities` and filter results for the new entity name.

**Step 2: Verify data flows**

Call `get_data_from_fields` with:

- `metrics`: `["<entity>.count"]`

Also call with a list of attributes to verify they are accessible:

- `attributes`: `["<entity>.<attribute1>", "<entity>.<attribute2>", "<entity>.<attribute3>"]`

**Step 3: Sanity checks**

| Check                 | What to Look For                 | Action if Failed             |
| --------------------- | -------------------------------- | ---------------------------- |
| Entity exists         | Shows in `list_entities` results | Check `create_entity` call   |
| Has rows              | Count > 0                        | Verify source table path     |
| Key is unique         | Count = count distinct of key    | Fix key or add composite key |
| Attributes accessible | Can query attributes             | Check dataset definition     |

**Alert user if:**

From honeydew-ai/honeydew-ai-coding-agents-plugins, Apache-2.0.

More from honeydew-ai/honeydew-ai-coding-agents-plugins