screen-recording-skill

Files (1)

• SKILL.md

SKILL.md (first lines)

# Screen Recording Skill

## What This Skill Does

Screen Recording Skill turns screen recordings into reusable agent skills. Record your screen while doing a task, and this skill extracts the steps, actions, and patterns to create a SKILL.md that any agent can follow.

**Key features:**
- Accept screen recordings (MP4, WebM, GIF)
- Extract visual steps from frames
- Identify UI elements and actions
- Generate structured SKILL.md
- Works with any agent framework

## When To Use This Skill

Use this skill when:
- User wants to create a skill from a screen recording
- User says "record this" or "turn this into a skill"
- User has a video of a workflow they want to automate
- User wants to document a process visually

## How It Works

### Input Methods

1. **Video file** — MP4, WebM, or GIF of screen recording
2. **Frame sequence** — Series of screenshots
3. **Live recording** — Record screen and generate skill in real-time

### Processing Pipeline

```
Recording → Frame Extraction → Element Detection → Step Generation → SKILL.md
```

1. **Frame Extraction** — Extract key frames at regular intervals
2. **Element Detection** — Identify buttons, inputs, menus, text
3. **Action Recognition** — Detect clicks, typing, navigation
4. **Step Generation** — Create numbered steps from actions
5. **SKILL.md Output** — Format as installable skill

## CLI Usage

```bash
# Generate skill from video
talocode-skills from-video recording.mp4 --output my-skill

# Generate skill from frame directory
talocode-skills from-frames ./screenshots/ --output my-skill

# Preview generated skill
talocode-skills preview my-skill/SKILL.md
```

## SDK Usage

### TypeScript

```typescript
import { ScreenRecordingProcessor } from '@talocode/skills';

const processor = new ScreenRecordingProcessor();

// From video file
const skill = await processor.fromVideo('recording.mp4', {
  outputDir: './my-skill',
  frameInterval: 2, // Extract frame every 2 seconds
  detectElements: true,
});

// From frames
const skill = await processor.fromFrames('./screenshots/', {
  outputDir: './my-skill',
});
```

### Python

```python
from talocode_skills import ScreenRecordingProcessor

processor = ScreenRecordingProcessor()

# From video file
skill = processor.from_video('recording.mp4', output_dir='./my-skill')

# From frames
skill = processor.from_frames('./screenshots/', output_dir='./my-skill')
```

## Generated SKILL.md Format

```markdown
# <Skill Name>

## What This Skill Does
<Auto-generated description based on recording>

## When To Use This Skill
<Use cases detected from workflow>

## Steps

### Step 1: <Action>
<Description of what to do>
- Element: <button/input/menu>
- Location: <where to find it>

### Step 2: <Action>
<Description of what to do>

...

## Tips
<Tips extracted from recording patterns>
```

## API Usage

```bash

From talocode/skills, MIT.

More from talocode/skills