feather-data-extraction

Files (1)

• SKILL.md

SKILL.md (first lines)

# Feather: Data Extraction

Builds on **using-feather-browser**. This is the *reading* side of the loop: `observe` is for
acting (refs, overlays, diff); **snapshot** is for reading a page, **extract** for pulling
structured fields. Don't use `observe { includeText: true }` for reading — it caps at 4k chars;
snapshot is the read tool.

## Reading a page → snapshot

For "what's on this page" / "summarize this" / "get the text or links":
```http
POST /v1/sessions/:sessionId/snapshot   {}
```
Returns:
- `markdown` — cleaned page content (nav/header/footer/ads/scripts stripped), capped at 20k chars.
  **This is your primary read** — it's the human-readable content, not raw HTML.
- `text` — plain text (also capped; tune with `limits.textChars`).
- `links` — `[{ text, href }]` (tune with `limits.links`).
- `title`, `url`, `meta.description`.

```http
POST /v1/sessions/:sessionId/snapshot   { "limits": { "textChars": 50000, "links": 500 } }
```
(`markdown`'s 20k cap is fixed and not request-configurable.)

## Pulling structured fields → extract

For named fields off the page:
```http
POST /v1/sessions/:sessionId/extract
{ "recipe": { "fields": {
    "title": { "selector": "h1", "type": "text" },
    "price": { "selector": ".price", "type": "text" },
    "image": { "selector": "img.hero", "type": "attribute", "attribute": "src" }
} } }
```
- `type: "text"` grabs the element's text; `type: "attribute"` grabs the named `attribute`;
  `type: "value"` grabs an **input/textarea/select's current value** — what's typed in the box,
  which text reads can't see. Use it to verify a form before submitting (no screenshot detour).
- `type` is optional: defaults to `"text"`, inferred as `"attribute"` when `attribute` is present.
  A flat `{ "fields": { … } }` body without the `recipe` wrapper is also accepted.
- **Each field selector resolves to its FIRST match.** This is intentional (no strict-mode crash on
  multiple matches) — but it means you can't collect a whole list with one top-level selector.

## Collecting a list / repeating rows

Because each field is first-match, scope to a row. Two approaches:

1. **Scope selectors to a container** so each call targets one row, iterating containers by index via
   the page, or
2. **Snapshot's `markdown`/`links`** — for simple lists (article links, nav items), the `links`
   array or the markdown's bullet lists are often all you need, no recipe required.

For genuinely repeating structured rows, snapshot first to understand the row markup, then extract
per-row with scoped selectors.

## Choosing the right tool

| Goal | Tool |
|------|------|
| "Read / summarize this page" | `snapshot` → `markdown` |
| "Get all the links" | `snapshot` → `links` |
| "Get these specific named fields" | `extract` with a recipe |
| "Get a list of items" | `snapshot` (markdown/links) first; `extract` per-row if structured |
| "Find then click a result" (e.g. a Google SERP) | `observe` — results are self-describing action entries (title + source + URL in `name`); click by ref, no href-extraction detour |

## Gotchas

- **extract is first-match only** — don't expect arrays from one selector; scope per row.
- **markdown is capped at 20k** — for very long pages, target the section you need or use `extract`.
- **Snapshot after navigation/interaction** — stale reads come from reading before the page settled;
  pair with `wait { until: "stable" }` if content loads async.
- **A consent wall in the way?** `dismiss` first (see using-feather-browser), then snapshot — the
  overlay's text pollutes reads.
- **Verify extracted facts semantically** — "next holiday" means after *today*; resolve relative
  dates against the current date before writing them anywhere.

From ROI-DANINO/feather-browser, Apache-2.0.

More from ROI-DANINO/feather-browser