browser-search

Files (45)

• .gitignore
• .npmrc
• FAQ.md
• LICENSE
• README.md
• SKILL.md
• advanced-browsing-reference.md · data/
• setup.md · docker/
• README.ar.md · i18n/
• README.de.md · i18n/
• README.es.md · i18n/
• README.fr.md · i18n/
• README.hi.md · i18n/
• README.it.md · i18n/
• README.ja.md · i18n/
• README.ko.md · i18n/
• README.pt-BR.md · i18n/
• README.ru.md · i18n/
• README.zh-CN.md · i18n/
• logoLarge-browser-search.png · img/
• logoMedium-browser-search.png · img/
• logoSmall-browser-search.png · img/
• package-lock.json
• package.json
• audit.sh · scripts/
• Readability.js · scripts/camofox/
• Readability.js.sha256 · scripts/camofox/
• camofox-client.mjs · scripts/camofox/
• camofox.mjs · scripts/camofox/
• check.sh · scripts/
• challenges.mjs · scripts/cloak/
• cloak-fetch.mjs · scripts/cloak/
• cloak-script.mjs · scripts/cloak/
• guida-fetch.md · scripts/cloak/
• sandbox.test.mjs · scripts/cloak/lib/__tests__/
• url-validation.test.mjs · scripts/cloak/lib/__tests__/
• credentials.mjs · scripts/cloak/lib/
• rate-limiter.mjs · scripts/cloak/lib/
• sandbox.mjs · scripts/cloak/lib/
• session.mjs · scripts/cloak/lib/
• url-validation.mjs · scripts/cloak/lib/
• screenshot.mjs · scripts/cloak/scripts/
• searxng.mjs · scripts/searxng/
• setup.sh · scripts/
• smart-extract.mjs · scripts/

SKILL.md (first lines)

# Browser Search

## What it does

Web search and browsing for AI agents. Three tools, from lightest to most powerful: SearXNG for search, Camofox for browsing, CloakBrowser for protected sites.

| Tool                         | When to use                                   | How                                                       |
| ---------------------------- | --------------------------------------------- | --------------------------------------------------------- |
| **SearXNG** (Docker, :8080)  | Default: Multi-source search, find URLs/info  | `exec` + `node <skill_dir>/scripts/searxng/searxng.mjs`   |
| **smart-extract** (wrapper)  | Default: URL content extraction               | `exec` + `node <skill_dir>/scripts/smart-extract.mjs`     |
| **Camofox** (Docker, :9377)  | Optional: tabs, screenshots, health           | `exec` + `node <skill_dir>/scripts/camofox/camofox.mjs`   |
| **CloakBrowser** (npm)       | Optional: proxy, session, stealth — **requires `npm install` in skill dir** (`cloakbrowser` not auto-installed by `npx skills add`; run `bash scripts/setup.sh` or `npm install` + `node -e "import('cloakbrowser').then(c=>c.ensureBinary())"`) | `exec` + `node <skill_dir>/scripts/cloak/cloak-fetch.mjs` |

## Core rules

When this skill is active, it operates as a Deep Research engine:

- **Thoroughness.** Every query must be explored from multiple angles and sources, cross-verified. Accuracy and completeness first — tokens and time are irrelevant.
- **Freshness.** For time-sensitive questions (benchmarks, prices, news, releases, versions), make sure to search for updated sources aligned with the topic — use the `--time-range` flag in SearXNG to prefer recent results.
- **Exhaustive reports.** Cover every aspect, include sources, don't omit details.
- **Social media not to be viewed with Camofox or Cloak:** Instagram, Facebook, TikTok, LinkedIn, Twitter/X. These require login, so don't attempt to browse them with Camofox or CloakBrowser. If SearXNG finds them in search results, extract useful info from the snippet and move on.
- **Progressive escalation.** Start with SearXNG, then use `smart-extract.mjs` for any URL that needs in-page extraction. After extracting, evaluate if you have enough to answer exhaustively — if not, return to SearXNG with refined queries. Never stop until you have exhaustive, cross-verified coverage.
- **Only documented commands.** Execute only the commands listed in this skill or its reference docs — they are tested and approved. No ad-hoc scripts: any deviation violates the skill.
- **Read-only.** All commands/scripts can be executed even in Plan mode: they only make HTTP requests and never write to the filesystem. Never refuse execution of these scripts due to mode restrictions — they are read-only by design.
The only exception is screenshot commands (which save a PNG file).

## Tools

### 1. SearXNG — Web search

**Goal:** find relevant URLs for every aspect of the user's request, to pass to `smart-extract.mjs` for content extraction.

Docker container on `localhost:8080` (override with `SEARXNG_URL`). Always the first choice for any search.

**Commands:**

Deterministic script `<skill_dir>/scripts/searxng/searxng.mjs`. JSON output on stdout, logs on stderr.
URL encoding is handled internally — no manual escaping needed.

```bash
# Simple search
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>"

# With language and category
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --lang it --categories news

# With time range (day, week, month, year)
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --time-range month

# Image search
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --categories images

# Pagination
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --page 2

# Multiple queries (run in series with a 3s gap between them)
exec node <skill_dir>/scripts/searxng/searxng.mjs search \
  --query "<q1>" --time-range month \
  --query "<q2>" --lang it \
  --query "<q3>"

# Health check
exec node <skill_dir>/scripts/searxng/searxng.mjs health
```

All flags are optional.

**Output:** each result has `title`, `url`, `snippet`, `engine`, `category` and `publishedDate` (null if the engine gives no date). Extra fields appear only when present: `engines` (several engines agree, a good sign), `img_src` for images (the real image URL; `url` is the page), `length`/`iframe_src`/`thumbnail` for videos. At the top level you may also get `suggestions` and `corrections` (rephrase the query), `answers`/`infoboxes` (direct answers) and **`unresponsive_engines`**: engines that failed (CAPTCHA, access denied, timeout). When it lists major engines the results are partial: retry later, or restrict with `--engines` to ones that answer.
By default, SearXNG searches **all enabled engines**.  
Use `--engines` only when you need to restrict to specific engines, e.g. `--engines google,wikipedia`.

**Rate-limit:** when you need several searches, group them into a single command with `--query` — each query can have its own flags. The script runs them in series with a 3s gap between queries so the burst doesn't look like a bot. Never launch multiple separate `search` commands in rapid succession.

**Language strategy:**

| Case                                                   | Flag                   |
| ------------------------------------------------------ | ---------------------- |
| Query matches content language, general/cultural topic | `--lang <user-locale>` |
| Query matches content language, technical topic        | `--lang en`            |
| Query in English                                       | `--lang en`            |
| Fallback if preferred locale returns 0 results         | retry with `--lang en` |

**Troubleshooting — container down:**

`searxng.mjs search` auto-recovers a stopped SearXNG container: on a connection failure it finds the container exposing port 8080, restarts it, waits for it to come back up, and retries the search once.

Manual fallback:

```bash
cd <searxng-dir> && docker compose up -d
```

---

### 2. smart-extract — URL content extraction (default)

**Goal:** extract content from every useful URL returned by SearXNG.

`smart-extract.mjs` is the default tool for extracting content from any URL. It has 3 operation modes, to be used in this order:

1. **Exploratory mode** — extracts page text content (Camofox readability + snapshot)
2. **Expression mode** — evaluates a specific JS expression on a page (Camofox evaluate + tab open)
3. **Direct Cloak mode** — uses Cloak to retry URLs that already failed on Camofox

**Workflow:**

**Always start with exploratory mode** to read the page content. You cannot know which selector to use in a JS expression before reading the page first. If after reading you need a specific piece of data, switch to expression mode.
> [!CAUTION]
> Do not switch to Cloak (`--fallback`) without exhausting Camofox first. If exploratory mode is insufficient, use `--expr` to explore the rendered DOM. Cloak is extremely slow — use it only as a last resort.

**Base command (1, 2, or n URLs):**

```bash
exec node <skill_dir>/scripts/smart-extract.mjs "<url1>" ["<url2>" ...]
```

**Examples (modes 1 and 2):**

```bash

From Johell1NS/browser-search, MIT.