web-fetch

Files (1)

• SKILL.md

SKILL.md (first lines)

# Web Fetch Skill

Prefer `web` with `action="fetch"` for routine page reads.

Use `lane: "default"` (or leave the lane unset) for web pages; internal hosts
are reachable only when the operator enables internal network access and
allowlists them.

The raw crawl4ai endpoint is deployment-specific backend detail. Never guess or
hard-code an internal hostname; the operator must supply it through the
gateway's private deployment configuration.

## Methods

### Method 1: Get Markdown (Simple)
```
GET /md/{url_encoded}?f=fit
```
- URL must be URL-encoded
- `f=fit` extracts main content, removes headers/footers/nav/ads
- Returns clean markdown

Example:
- To fetch `https://example.com/article`
- Request: `GET /md/https%3A%2F%2Fexample.com%2Farticle?f=fit`

### Method 2: Batch Crawl (Advanced)
```
POST /crawl
Content-Type: application/json

{
  "urls": ["https://example.com"],
  "crawler_config": { "stream": false }
}
```
- Max 100 URLs per request
- Returns results after all processed

## Usage
When asked to read a URL or fetch web content:
1. Prefer `web` with `action="fetch"`
2. Only fall back to a raw crawl4ai endpoint when the operator has explicitly
   supplied one for this deployment
3. Return the fetched content to the user

## Notes
- This service runs on the local network
- Use for reading articles, documentation, etc.
- The fit filter provides clean, readable content

From CGIC-AI/psfn-framework, AGPL-3.0.

More from CGIC-AI/psfn-framework