llm-wiki

Files (5)

• SKILL.md
• __init__.py · scripts/
• core.py · scripts/
• ingest.py · scripts/
• search.py · scripts/

SKILL.md (first lines)

# Karpathy's LLM Wiki

Build and maintain a persistent, compounding knowledge base as interlinked markdown files.
Based on [Andrej Karpathy's LLM Wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).

Unlike traditional RAG (which rediscovers knowledge from scratch per query), the wiki
compiles knowledge once and keeps it current. Cross-references are already there.
Contradictions have already been flagged. Synthesis reflects everything ingested.

**Division of labor:** The human curates sources and directs analysis. The agent
summarizes, cross-references, files, and maintains consistency.

## When This Skill Activates

Use this skill when the user:
- Asks to create, build, or start a wiki or knowledge base
- Asks to ingest, add, or process a source into their wiki
- Asks a question and an existing wiki is present at the configured path
- Asks to lint, audit, or health-check their wiki
- References their wiki, knowledge base, or "notes" in a research context

## Wiki Directory Structure

All paths are computed dynamically by the scripts, based on `src/data/wiki/` under the project root.

```json
{
  "wiki": {
    "root": ["SCHEMA.md", "index.md", "log.md"],
    "raw": {
      "description": "Raw sources (read-only, never modified)",
      "articles": "Web articles / online sources",
      "papers": "Papers / research reports",
      "transcripts": "Transcripts / interviews",
      "assets": "Images / attachment resources"
    },
    "entities": "Entity profiles (people/orgs)",
    "concepts": "Concept/topic analyses",
    "comparisons": "Side-by-side comparisons",
    "queries": "Filed query results"
  }
}
```

## Quick Start

### Inspect the Wiki path and structure

```python
from skills.builtin.llm_wiki.scripts import get_wiki_path, print_structure

print(f"Wiki path: {get_wiki_path()}")
print(f"Directory structure: {print_structure()}")
```

### Initialize the Wiki

```python
from skills.builtin.llm_wiki.scripts import init_wiki

result = init_wiki()
print(result)
```

### Search the Wiki

```python
from skills.builtin.llm_wiki.scripts import search_wiki

results = search_wiki("keyword")
for r in results:
    print(f"{r['file']} — {r['matches']} matches")
```

### Health Check

```python
from skills.builtin.llm_wiki.scripts import lint_wiki

report = lint_wiki()
print(report)
```

### Save a Raw Source

```python
from skills.builtin.llm_wiki.scripts import save_source

result = save_source("source content", category="articles", filename="my-source.md")
print(result)
```

`save_source` is unchanged (text only). For PDF / image / audio / video / docx /
xlsx / markdown payloads use the additive binary path:

```python
from skills.builtin.llm_wiki.scripts import save_binary_source

result = save_binary_source(
    pdf_bytes, "pdf", source_url="https://example.test/paper.pdf", filename="paper.pdf"
)
```

It extracts text with the skill's own extractors (mineru / pypdf / pdftotext for
PDF, `image_to_text` for images, `speech_to_text` for audio,
`video_text_to_text` for video, stdlib ZIP/XML readers for docx/xlsx), runs one
auxiliary-LLM semantic pass (fail-open), writes `raw/<category>/<name>.md` with
the optional `extracted_entities` / `extracted_relations` frontmatter, and
refreshes the explicit graph.

### Graph queries

`search_wiki(keyword)` is still the plain grep (unchanged). Once a wiki has a
graph, prefer the graph queries — they return structure, not matching lines:

```python
from skills.builtin.llm_wiki.scripts import (
    build_wiki_graph,
    path,

From lasertsole/sherry_agent, MIT.

More from lasertsole/sherry_agent