indeed-brightdata

Files (37)

• plugin.json · .claude-plugin/
• .gitignore
• LICENSE
• Makefile
• README.md
• SKILL.md
• install.sh
• api-reference.md · references/
• keyword-expansions.json · references/
• _lib.sh · scripts/
• indeed_check_pending.sh · scripts/
• indeed_company_by_industry.sh · scripts/
• indeed_company_by_keyword.sh · scripts/
• indeed_company_by_url.sh · scripts/
• indeed_format_results.sh · scripts/
• indeed_jobs_by_company.sh · scripts/
• indeed_jobs_by_keyword.sh · scripts/
• indeed_jobs_by_url.sh · scripts/
• indeed_list_datasets.sh · scripts/
• indeed_poll_and_fetch.sh · scripts/
• indeed_smart_search.sh · scripts/
• package.sh · scripts/
• sample_companies.json · tests/fixtures/
• sample_jobs.json · tests/fixtures/
• setup.bash · tests/helpers/
• test_check_pending.bats · tests/
• test_discovery_flags.bats · tests/
• test_format_results.bats · tests/
• test_install.bats · tests/
• test_lib.bats · tests/
• test_lib_history.bats · tests/
• test_package.bats · tests/
• test_pending.bats · tests/
• test_poll_and_fetch.bats · tests/
• test_scripts_smoke.bats · tests/
• test_skill_frontmatter.bats · tests/
• test_smart_search.bats · tests/

SKILL.md (first lines)

# Indeed Bright Data Skill

Search Indeed for job listings and company info via Bright Data's Web Scraper API. Designed for recruiting workflows on messaging platforms (Telegram, Signal) with smart defaults.

## Prerequisites

- `BRIGHTDATA_API_KEY` environment variable must be set
- `curl` and `jq` must be available

## Workflow Decision Tree

```text
User wants job info?
├── Has a specific Indeed URL?
│   ├── Job URL (/viewjob?) → indeed_jobs_by_url.sh [SYNC — seconds]
│   ├── Company jobs URL (/cmp/*/jobs) → indeed_jobs_by_company.sh [ASYNC — minutes]
│   └── Company page URL (/cmp/*) → indeed_company_by_url.sh [SYNC — seconds]
├── Wants to search by keyword/location?
│   └── indeed_smart_search.sh [ASYNC — 3-8 min]
│       Agent says: "Searching now, this takes a few minutes."
│       If results < 5: auto-expands date range, do NOT ask user
│       Always pipe output through: indeed_format_results.sh --top 5
├── Wants company info?
│   ├── Has Indeed company URL → indeed_company_by_url.sh [SYNC — seconds]
│   ├── Has keyword → indeed_company_by_keyword.sh [ASYNC — minutes]
│   └── Has industry + state → indeed_company_by_industry.sh [ASYNC — minutes]
└── Check pending results? → indeed_check_pending.sh (run on heartbeat)
```

**Always prefer sync (URL-based) scripts when the user provides a URL — they return in seconds.**

## Scripts Reference

| Script | Purpose | Mode |
|--------|---------|------|
| `indeed_smart_search.sh` | **Primary job search** — keyword expansion, parallel queries, dedup, caching | ASYNC |
| `indeed_jobs_by_url.sh` | Collect job details by URL(s) | SYNC |
| `indeed_jobs_by_keyword.sh` | Low-level single-keyword job search (used by smart search internally) | ASYNC |
| `indeed_jobs_by_company.sh` | Discover jobs from company page | ASYNC |
| `indeed_company_by_url.sh` | Collect company info by URL | SYNC |
| `indeed_company_by_keyword.sh` | Discover companies by keyword | ASYNC |
| `indeed_company_by_industry.sh` | Discover companies by industry/state | ASYNC |
| `indeed_format_results.sh` | Format JSON results into summary, full, or CSV | Local |
| `indeed_check_pending.sh` | Check/fetch completed pending searches + auto-cleanup | Local/API |
| `indeed_poll_and_fetch.sh` | Poll async job and fetch results (internal) | API |
| `indeed_list_datasets.sh` | List available Indeed dataset IDs | API |

## Quick Start

User says: "Find me cybersecurity jobs in New York"
```bash
scripts/indeed_smart_search.sh "cybersecurity" US "New York, NY" \
  | scripts/indeed_format_results.sh --type jobs --top 5
```

User says: "Get details on this job: https://www.indeed.com/viewjob?jk=abc123"
```bash
scripts/indeed_jobs_by_url.sh "https://www.indeed.com/viewjob?jk=abc123"
```

## Behavior Rules (MANDATORY)

1. **NEVER return raw JSON to the user.** Always pipe results through `indeed_format_results.sh`.
2. **NEVER ask "want me to try broader keywords?"** if results < 5. The smart search auto-expands automatically. Just tell the user: "Found only N results with recent postings, expanding search..."
3. **NEVER present results older than 30 days** without noting they may be stale.
4. **When a discovery search is running**, immediately acknowledge: "Searching Indeed now — this usually takes 3-5 minutes. I'll come back with results."
5. **If the user asks a follow-up while a search is pending**, run `indeed_check_pending.sh` first before starting a new search.
6. **For Telegram**: keep each message under 3500 characters. Use the `---SPLIT---` markers from `indeed_format_results.sh` to break across messages.
7. **Always show total result count** and offer to show more: "Showing top 5 of 23 results. Want to see more, or filter by salary/location?"
8. **Default to "Last 7 days"** for date filtering. If the user says "find me jobs" without a time preference, the default is already set.

## Smart Search (Primary Entry Point)

```bash
# Basic search (expands keywords, deduplicates, defaults to last 7 days)
scripts/indeed_smart_search.sh "cybersecurity" US "Remote"

# All-time search
scripts/indeed_smart_search.sh "nursing" US "Texas" --all-time

# Skip keyword expansion
scripts/indeed_smart_search.sh "registered nurse" US "Ohio" --no-expand

# Bypass 6-hour cache
scripts/indeed_smart_search.sh "data science" US "New York" --force
```

Output is `{"meta": {...}, "results": [...]}` with metadata including query params, keywords used, and result counts.

## Result Formatting

```bash
# Telegram-friendly summary (default)
scripts/indeed_format_results.sh --type jobs --top 5 results.json

# CSV export
scripts/indeed_format_results.sh --type jobs --format csv results.json

# Companies
scripts/indeed_format_results.sh --type companies --top 5 companies.json

# Pipe from smart search
scripts/indeed_smart_search.sh "nurse" US "Ohio" | scripts/indeed_format_results.sh --top 5
```

## Heartbeat: Checking Pending Results

```bash
scripts/indeed_check_pending.sh
# Output: {"completed":[...],"still_pending":[...],"failed":[...]}
```

Run this periodically. If `~/.config/indeed-brightdata/pending.json` exists and is non-empty, check for completed results. Format completed results with `indeed_format_results.sh` and send to the user.

## Exit Codes

| Code | Meaning | Agent should... |
|------|---------|-----------------|
| 0 | Success — results on stdout | Format and present results |

From foreztgump/indeed-brightdata, MIT.