robots-txt
Publish a /robots.txt that allows reputable crawlers and points them at the sitemap index.
Skill by cicero-im
Install
Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install skill/cicero-im-robots-txt
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.
- Tools
- No tools found
- Secrets
- None asked for
- Install scripts
- None found
Network hosts it talks to 1
codeload.github.com· found in install command
Files it touches
~/.claude/skills/robots-txt
What's inside
- Skill name
robots-txt- Description
- Publish a /robots.txt that allows reputable crawlers and points them at the sitemap index.
- License
- MIT
- Instructions
- 15 lines in SKILL.md · view on GitHub
Files (1)
- SKILL.md
SKILL.md (first lines)
# robots.txt Expose `/robots.txt` with allow rules and a `Sitemap:` line referencing the canonical sitemap index URL. ## Requirements - `User-agent: *` and `Allow: /` (or specific deny rules where needed) - One absolute `Sitemap:` URL per sitemap index - Served with `Content-Type: text/plain` ## Implementation Pattern Use `astro-robots-txt` (auto-generates from `site` in `astro.config.mjs`) or commit a static `public/robots.txt` and keep the `Sitemap:` URL in sync with the deployed origin.
From cicero-im/astro-static-cloudflare, MIT.
More from cicero-im/astro-static-cloudflare
- 0.406score
paraglide-i18n
cicero-im
Compile message bundles into per-locale JS modules with Inlang Paraglide so translations are tree-shaken, type-safe, and SSR-safe.
SkillDFreshness: DClaude Code +9 - 0.406score
cloudflare-workers-static
cicero-im
Deploy a fully static Astro build to Cloudflare Workers via the [assets] binding so every page is served from the global edge.
SkillDFreshness: DClaude Code +9 - 0.402score
i18n-routing
cicero-im
Serve every page in every supported locale under a prefixed URL with hreflang alternates so search engines and agents discover translations.
SkillDFreshness: DClaude Code +9 - 0.402score
json-ld-structured-data
cicero-im
Embed schema.org JSON-LD on every page so search engines and AI agents can parse entities, articles, and breadcrumb trails.
SkillDFreshness: DClaude Code +9 - 0.402score
web-manifest
cicero-im
Publish a Web App Manifest at /manifest.webmanifest with localized variants so the site can be installed and rendered as a PWA.
SkillDFreshness: DClaude Code +9 - 0.399score
og-image-generation
cicero-im
Generate a unique Open Graph image for every page so link previews on social platforms and chat apps render correctly.
SkillDFreshness: DClaude Code +9 - 0.399score
link-response-headers
cicero-im
Add RFC 8288 Link headers on the homepage for API and service discovery.
SkillDFreshness: DClaude Code +9 - 0.399score
agent-skills-index
cicero-im
Publish and maintain a non-empty Agent Skills discovery index at /.well-known/agent-skills/index.json.
SkillDFreshness: DClaude Code +9 - 0.399score
rss-feed
cicero-im
Publish an RSS 2.0 feed at /rss.xml that mirrors the blog collection so readers and agents can subscribe.
SkillDFreshness: DClaude Code +9 - 0.399score
sitemap-index
cicero-im
Publish a discoverable XML sitemap index at /sitemap-index.xml so crawlers and agents can enumerate every page.
SkillDFreshness: DClaude Code +9
Signals
- Stars
- 0
- Forks
- 0
- Open issues
- 1
- Last commit (this folder)
- Apr 25, 2026 (6 months ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- MIT
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- GitHubgithub-graphql
More from cicero-im
- 0.406score
paraglide-i18n
cicero-im
Compile message bundles into per-locale JS modules with Inlang Paraglide so translations are tree-shaken, type-safe, and SSR-safe.
SkillDFreshness: DClaude Code +9 - 0.406score
cloudflare-workers-static
cicero-im
Deploy a fully static Astro build to Cloudflare Workers via the [assets] binding so every page is served from the global edge.
SkillDFreshness: DClaude Code +9 - 0.402score
i18n-routing
cicero-im
Serve every page in every supported locale under a prefixed URL with hreflang alternates so search engines and agents discover translations.
SkillDFreshness: DClaude Code +9 - 0.402score
json-ld-structured-data
cicero-im
Embed schema.org JSON-LD on every page so search engines and AI agents can parse entities, articles, and breadcrumb trails.
SkillDFreshness: DClaude Code +9 - 0.402score
web-manifest
cicero-im
Publish a Web App Manifest at /manifest.webmanifest with localized variants so the site can be installed and rendered as a PWA.
SkillDFreshness: DClaude Code +9 - 0.399score
og-image-generation
cicero-im
Generate a unique Open Graph image for every page so link previews on social platforms and chat apps render correctly.
SkillDFreshness: DClaude Code +9
Cite this page
robots-txt skill. Agent Mart. https://agentmart.dev/skills/cicero-im-robots-txt (last updated 2026-10-24)