Skip to content
Agentmart
Submit a tool

crawl-render-audit

Specialist skill assessing whether a website is machine-crawlable — HTTP status, robots.txt directives, sitemap coverage, redirect chains, canonical…

Skill by Adamkoda2306

IndexedMIT
Write a review

Install

Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install skill/adamkoda2306-crawl-render-audit

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Specialist skill assessing whether a website is machine-crawlable — HTTP status, robots.txt directives, sitemap coverage, redirect chains, canonical consistency, duplicate URLs, and source-vs-rendered content risk. Used by audit-orchestrator; independently runnable for testing.

Use cases: testing, security, writing

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.

Tools
No tools found
Secrets
None asked for
Install scripts
None found
Runs as
Python package

Network hosts it talks to 1

  • codeload.github.com · found in install command

Files it touches

  • ~/.claude/skills/crawl-render-audit

What's inside

Skill name
crawl-render-audit
Description
Specialist skill assessing whether a website is machine-crawlable — HTTP status, robots.txt directives, sitemap coverage, redirect chains, canonical consistency, duplicate URLs, and source-vs-rendered content risk. Used by audit-orchestrator; independently runnable for testing.
License
MIT
Instructions
26 lines in SKILL.md · view on GitHub

Before installing

  • Ships executable scripts

Agent Skills spec issues

  • allowed-tools: must be a space-separated string

Files (3)

  • SKILL.md
  • methodology.md · references/
  • crawl_audit.py · scripts/

SKILL.md (first lines)

# crawl-render-audit

## Purpose
Determines whether an automated reader can *reach* a site's important pages
and whether the content a non-rendering fetcher receives materially matches
what a browser would show. Never renders every page by default; if no
rendering engine is available, that is recorded as an `audit_metadata`
limitation, not treated as a site defect.

## Checks
See `references/methodology.md` for the full list (robots.txt compliance,
sitemap discovery/parsing, redirect chains/loops, canonical alignment,
duplicate URL detection, HTTP error sampling).

## Usage
```bash
python scripts/crawl_audit.py https://example.com   # standalone/dev mode
```
Normally invoked in-process by audit-orchestrator with an already-crawled
`CrawlResult`.

## Safety
Read-only fetches only. Always checks `robots.txt` before fetching a URL and
skips disallowed URLs.

From Adamkoda2306/Brand-AI-Readiness-Audit, MIT.

More from Adamkoda2306/Brand-AI-Readiness-Audit

Signals

Stars
0
Forks
0
Open issues
0
Last commit (this folder)
Sep 12, 2026 (1 month ago)

From GitHub (fetched Oct 24, 2026).

Details

License
MIT
Repository
github.com/Adamkoda2306/Brand-AI-Readiness-Audit
First seen
Oct 24, 2026
Last checked
Oct 24, 2026
Source
GitHubgithub-graphql

More from Adamkoda2306

Cite this page

crawl-render-audit skill. Agent Mart. https://agentmart.dev/skills/adamkoda2306-crawl-render-audit (last updated 2026-10-24)