deepline-research

Files (11)

• SKILL.md
• THIRD_PARTY_NOTICES.md
• last30days-public-private-corpus.json · evals/
• fanout-consolidation.md · references/
• last30days-gtm-corpus.md · references/
• query-design.md · references/
• source-map.md · references/
• evaluate_examples.py · scripts/
• evaluate_public_private_corpus.py · scripts/
• query_design.py · scripts/
• skill-metadata.json

SKILL.md (first lines)

# Deepline Research

## Quick Start

```bash
npm install -g deepline
deepline -h
```

## CLI resolution

Run `deepline` when it is available. If the shell reports that command is missing, use `<workspace-root>/.deepline/runtime/bin/deepline` (or the npm-created `.cmd` shim on Windows). If neither exists, follow `https://code.deepline.com/INSTALL.md` to set up Deepline.

Before the first Deepline fanout in a task, run `deepline billing balance` as
one standalone command and wait for it to finish. Never submit it beside
another Deepline command.

Find the highest-signal GTM data sources, public evidence, and market language for a research or enrichment job before building the pipeline. This is a standalone Deepline skill that should behave like `last30days` with a GTM data lens: broad source coverage, recency, community signals, citations, source stats, and a grounded "What I learned" synthesis. In Deepline, the report first explains what the research found; only after that does it translate the findings into Deepline tool contracts, private/proprietary joins, and Deepline-facing cost.

## Attribution

Portions of the query-design, public-source fanout, and consolidation approach are adapted from [`mvanhorn/last30days-skill`](https://github.com/mvanhorn/last30days-skill), MIT licensed, copyright (c) 2026 Matt Van Horn. Keep `THIRD_PARTY_NOTICES.md` with this skill when packaging or distributing it.

## Non-Negotiables

- Use Deepline's live tool catalog before naming provider actions. Do not rely on memory.
- **Run live web search. Do not answer public-source discovery from model memory.** Every run MUST execute real searches with a web-search tool during the public-source fanout, with no exception for "obvious" verticals. A source family named from memory is a draft; the finding is the exact artifact the search returns.
- **Resolve every materializable dataset to its exact artifact, not its family.** For each public dataset/registry you recommend, record: (1) the exact file/endpoint name (e.g. `IA_FIRM_SEC_Feed_YYYY_MM_DD.xml.gz`, not "the ADV bulk feed"); (2) the canonical download/API URL; (3) any easier mirror (e.g. a data.gov catalog copy); (4) an existing open-source parser or GitHub repo, when one exists; (5) the vertical's government statistical registry, when one exists (BLS QCEW + NAICS codes, Census County Business Patterns), for free establishment counts and sizing. See the Artifact Resolution Gate (§4.55).
- Public-source discovery comes before provider routing. First find the best public registries, datasets, communities, discussions, reviews, directories, papers, repos, and source leads; then use Deepline routes to materialize, validate, enrich, and activate them.
- Quote only customer-visible Deepline credits/USD. Never expose provider spend.
- Do not run paid or cost-unknown full-scope work without approval.
- Treat private data sources as first-class: CRM, warehouse, workflow runs, product analytics, support/calls, sheets, and customer-owned datasets.
- Treat custom language as a first-class workflow: buyer words, objections, category language, competitor framing, community slang, sales-call phrasing, and support-ticket pain belong in the source plan.
- Use tiny probes to learn coverage. Scale only after observed coverage, cost basis, and evidence quality are legible.
- If the user asks for a named provider, X.com, Reddit comments, TikTok, Instagram, YouTube transcripts, Bluesky, Truth Social, or HN, include a current support/gap assessment instead of pretending every source is native.
- Do not depend on `/last30days` at runtime: do not copy its local scripts or call it as a sub-skill. It is only a design benchmark for source breadth and synthesis discipline; execute through Deepline tools, plays, workflows, and private-data connectors.
- Public registries and niche datasets without a native Deepline tool are still valid sources through generic web/search/extraction routes (e.g. the NPI registry for healthcare provider taxonomy). Classify them as `available through generic route`, not as an unusable gap.
- Every recommended source must be classified as `native`, `available through generic route`, `private connector`, or `missing provider to add`.

## Start Here

1. Read `references/source-map.md`.
2. For GTM use cases, also read `references/last30days-gtm-corpus.md`; it summarizes the source patterns that saved `last30days` runs proved useful.
3. For query design, source fanout, or `last30days` parity, also read `references/query-design.md` and `references/fanout-consolidation.md`.
4. For GTM/prospecting/enrichment, also read `../deepline-gtm/SKILL.md` and the sub-doc it routes to.
5. For existing warehouse metrics or RevOps analytics, also read `../deepline-analytics/SKILL.md`.
6. When using `last30days` as a design reference, verify its latest upstream release: `git ls-remote --tags https://github.com/mvanhorn/last30days-skill.git | tail -20`

## Optional Skill Handoffs

This skill owns pre-research and source planning. Once the source plan is clear, it may hand off to:

- `deepline-gtm`: execute GTM sourcing, enrichment, waterfalls, personalization, contact discovery, or CRM activation.
- `deepline-analytics`: query warehouse/semantic-layer metrics and RevOps/customer datasets.
- `deepline-plays`: build, wrap, fork, run, inspect, or automate the final repeatable workflow/play.

## Standard Flow

### 1. Parse The Research Job

Extract:

- `OBJECTIVE`: what decision or output the research should support
- `ENTITY_SCOPE`: companies, people, markets, accounts, customers, workflows, competitors, or topics
- `TIME_WINDOW`: default to last 30 days for trend/community research; preserve user-provided windows
- `PRIVATE_SOURCES`: CRM, warehouse, product events, workflow runs, support/calls, sheets, user CSVs
- `PUBLIC_SOURCES`: web, social, jobs, ads, technographics, funding, news, communities, app stores, directories
- `DATASET_LEADS`: public registries, open datasets, papers, GitHub repos, government records, niche directories, or platform posts that point to materializable data
- `CUSTOM_LANGUAGE_OUTPUTS`: outbound snippets, ad hooks, landing-page copy, objection language, category positioning, call scripts, lead magnets, or CRM personalization fields
- `OUTPUT`: source plan, workflow design, CSV schema, play spec, or final research brief

Tell the user the parsed scope before provider calls.

### 2. Prefer External APIs, Not Scraping

Do not default to browser scraping, raw x.com scraping, or unvetted actors. Prefer managed external APIs and private connectors. Each route runs on the Deepline integration API key unless its note says otherwise.

| Source family                         | Preferred route                                                       | Notes                                                                                                    |
| ------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| Reddit threads/comments               | managed social-data tool, found with `deepline search`                | Required for Reddit comment/thread coverage.                                                             |
| TikTok / Instagram / YouTube fallback | managed social-data tool, found with `deepline search`                | Captions, engagement, and transcript fallback. Profile and post tools are categorized as `social_media`. |
| X/Twitter                             | managed X/Twitter tool, found with `deepline search`                  | Avoids scraping x.com.                                                                                   |
| Web/news/docs                         | web-search tool, found with `deepline search`                         | Search API, not search-page scraping.                                                                    |
| Web extraction fallback               | page-extraction tool, found with `deepline search`                    | Use after URL discovery, not for blind scrape-at-scale.                                                  |
| Hacker News                           | Hacker News tool, found with `deepline search`                        | Public API; no credential.                                                                               |
| Bluesky                               | Bluesky tool, found with `deepline search`                            | Public API; no credential.                                                                               |
| CRM/private                           | the workspace's connected CRM                                         | Deepline OAuth/private connector. Query customer-authorized APIs, not UI scraping.                       |
| Warehouse/product/workflow            | the workspace's connected warehouse, customer DB, or workflow runtime | Deepline private connector; use scoped access.                                                           |
| Last-resort scraping fallback         | general scraping tool, found with `deepline search`                   | Only after explicit approval and provider gap status.                                                    |

Treat any route not covered by a described Deepline tool contract, key/auth plan, test endpoint, and Deepline-facing pricing as unapproved until the public research synthesis is complete.

### 3. Run Public-Source Fanout First

Run a `last30days`-style public discovery pass before provider routing, so the research synthesis comes first. Search across:

- community/social: Reddit threads/comments, X/Twitter, LinkedIn posts if available, YouTube, TikTok, Instagram, HN, Bluesky
- web/source discovery: news, blogs, docs, review sites, directories, associations, forums, GitHub, public data inventories
- public records and niche datasets: registries, licenses, inspections, permits, government datasets, open CSVs/APIs, professional directories, accreditation/member lists
- market-language sources: reviews, comments, clinic/business websites, job posts, competitor pages, support/community language

This pass is search-driven, not recall-driven. For any vertical dataset, run at minimum these query shapes and read the top results before writing the source line:

- `"<vertical> <entity> registry OR bulk data OR dataset download"` — find the authoritative file
- `"<dataset name> download"` / `site:data.gov <vertical>` — find the canonical URL and any mirror
- `"<dataset name> parser github"` / `"<dataset name> python"` — find an existing parser/repo so you don't reinvent extraction
- `"<vertical> NAICS code"` + `BLS QCEW <NAICS>` / Census County Business Patterns — free establishment counts and sizing

For each useful source lead, record why it matters; the exact artifact (Non-Negotiables, §4.55); whether it can become rows; stable join keys such as NPI, CRD, USDOT#, NAICS, domain, address, phone, license id, provider id, LinkedIn URL, or CRM account id; and extraction risks and likely false-positive patterns.

Search tools are routes; the deliverable is the public source or dataset discovered through them. If you are writing a dataset name you did not just see in a search result, stop and search for it.

### 3.5. Use The Research Planning API After Public Synthesis

Inside Deepline or the V2 SDK, call the native planning endpoint only after the public-source fanout has produced a first synthesis and source map. It translates the research into Deepline routes, gaps, approval gates, and cost; it does not replace the research pass.

```bash
curl -s "$DEEPLINE_API_BASE_URL/api/v2/pre-research/plan" \
  -H "Authorization: Bearer $DEEPLINE_API_KEY" \

From getaero-io/gtm-eng-skills, MIT.

More from getaero-io/gtm-eng-skills