roboflow-training-and-evaluation

Files (6)

• SKILL.md
• active-learning.md
• improvement-playbook.md
• model-diagnosis.md
• model-ids.md
• nas-results.md

SKILL.md (first lines)

> **For agents:** This skill is authored in [`roboflow/computer-vision-skills`](https://github.com/roboflow/computer-vision-skills). If your client already has it loaded as a skill (for example from a Roboflow plugin), use that copy. The `skill://roboflow/<name>/SKILL.md` MCP resources serve the same skills for clients that don't have them loaded.

# Training & Evaluation on Roboflow

## Training Flow

```
Upload/Annotate Images
  → Generate Dataset Version (preprocessing + augmentation + train/val/test split)
    → Pick Model Architecture + Size
      → Pick Checkpoint (COCO, Universe model, or previous version)
        → Train
          → Evaluate (auto-runs for paid users)
```

**Version = frozen snapshot.** Changes to the project after version creation do not affect it. Configure preprocessing (resize, contrast, etc.) and augmentation (flip, rotate, mosaic, etc.) during version generation.

## Available Model Architectures

### Object Detection

| Architecture | Sizes | Default Resolution | Notes |
|---|---|---|---|
| **RF-DETR** | Pico, Nano, Small, Base, Medium, Large, XL, 2XL | 384-880 (varies by size) | Best accuracy among named sizes; see RF-DETR NAS |
| Roboflow 3.0 | Fast, Accurate, Medium, Large, XL | 640x640 | YOLOv8-based. Medium+ require paid plan |
| YOLO26 | n/s/m/l/x | 640x640 | Also supports seg + pose |
| YOLOv12 | n/s/m/l/x | 640x640 | OD only |
| YOLOv11 | n/s/m/l/x | 640x640 | Also supports seg + pose |
| YOLOv8 | n/s/m/l/x | 640x640 | Also supports seg + pose |
| YOLO-NAS | Small, Medium | 640x640 | |
| YOLOLite CPU | n/s/m/l/x | 640x640 | Edge-optimized, beta |
| YOLOLite GPU | n/s/m/l/x | 640x640 | Edge-optimized, beta |
| **Roboflow Instant** | single | N/A (no resize) | Few-shot, free, OD only |

### Instance Segmentation

| Architecture | Sizes | Default Resolution |
|---|---|---|
| **RF-DETR Seg** | Nano, Small, Medium, Large, XL, 2XL | 312-768 (varies) | Pico and Base not available for seg |
| Roboflow 3.0 Seg | Fast, Accurate, Medium, Large, XL | 640x640 |
| YOLO-seg | v8/v11/v26 (n/s/m/l/x each) | 640x640 |
| SAM 3 (Segment Anything 3) | Large | 1008x1008 |

### Semantic Segmentation

| Architecture | Sizes | Default Resolution |
|---|---|---|
| DeepLabV3+ | Base | >=512x512 |

### Classification

| Architecture | Sizes | Default Resolution |
|---|---|---|
| ViT | Base | 224x224 |
| ResNet | 18/34/50/101 | 224x224 |
| DINOv3 | Base, Small | 224x224 |

### Keypoint / Pose

| Architecture | Sizes | Default Resolution |
|---|---|---|
| YOLO-pose | v8/v11/v26 (n/s/m/l/x each) | 640x640 |

### Multimodal / VLM

| Architecture | Sizes | Default Resolution |
|---|---|---|
| **Qwen3.5** | 0.8B, 2B | 448x448 |
| Qwen3 VL | 2B | 448x448 |
| SmolVLM | 256M, 2B | 384x384 |
| Florence 2 | Base, Large | 768x768 |
| PaliGemma 2 | 3B | 448x448 |
| Qwen2.5 VL | 7B | 448x448 |

## Model Selection Decision Tree

Follow this flowchart to pick the right model. Start at Step 1.

1. **Task type?** OD / Instance Seg / Keypoint → Step 2. Classification / Semantic Seg / VLM → use specialized block directly.
2. **Target classes in COCO 80?** Yes → Step 3. No → Step 6.
3. **Real-time?** No (images/recorded video) → Step 4. Yes (live video) → Step 5.
4. **Non-real-time, COCO** — OD / Inst Seg → prefer **RF-DETR NAS**; if unavailable, RF-DETR (detection) or RF-DETR Seg (instance segmentation) at Medium, Small for constrained HW, XL for accuracy-first. Keypoint → YOLO26 pose. **Done.**
5. **Real-time, COCO** — Same, and NAS is the strongest option here because it reports measured latency per target hardware. Without it, same families at Nano–Small. **Done.**
6. **Non-COCO, which sub-task?** OD → Step 7. Inst Seg → Step 8. Keypoint → Step 9.
7. **OD, non-COCO** — Check Rapid exclusions (see below). If excluded → Step 13. Otherwise → recommend **Roboflow Rapid** (default) or SAM3 zero-shot as secondary option → Step 10.
8. **Inst Seg, non-COCO** — SAM3 zero-shot (`sam3/sam3_final`, set `class_names`). Rapid does not support segmentation → Step 10.
9. **Keypoint, non-COCO** — Not real-time → YOLO26 pose Medium–XL. Real-time → YOLO26 pose Nano–Small. **Done.**
10. **Real-time?** No → try model (Step 11). Yes → warn about latency, try model (Step 12).
11. **Non-real-time trial** — User confirms works → **Done.** Poor results → Step 13.
12. **Real-time trial** — User confirms works → **Done.** Poor results → Step 13.
13. **Universe Model Search** — search community models on Roboflow Universe. Good match → **Done.** No match → Step 14.
14. **Custom Training** — OD / Inst Seg → prefer **RF-DETR NAS**. If NAS is unavailable, fine-tune RF-DETR (detection) or RF-DETR Seg (instance segmentation), sized by HW constraints. **Done.**

## Model ID Reference

Read [`model-ids.md`](./model-ids.md) before calling a training tool. It contains the exact supported `model_id` values and the COCO class reference. Do not guess model IDs.

## Model Selection Quick Guide

**When you are training a model on the user's data for detection or instance segmentation, start
with NAS.** Neural Architecture Search searches the RF-DETR space against that data and reports a
speed/accuracy frontier, so it is the option most likely to land on the best model for it. Reach
for a single named architecture when NAS is unavailable (see prerequisites below), when the user
asks for a specific one, or when a quick throwaway baseline is all that's wanted.

This summarises the training branches of the decision tree above; it does not override it. The
tree may route a non-COCO request to Roboflow Rapid or SAM3 zero-shot first, neither of which
trains a model — NAS only applies once custom training is the chosen path.

| Goal | Recommended |
|---|---|
| **Object detection — no strong prior** | **RF-DETR NAS** (`rfdetr-nas-parent`) |
| **Instance segmentation — no strong prior** | **RF-DETR NAS Seg** (`rfdetr-nas-seg-parent`) |
| Best accuracy, object detection, NAS unavailable | RF-DETR (Large or XL) |
| Fast inference, object detection, NAS unavailable | RF-DETR Nano or YOLOv11n |
| Best accuracy, instance segmentation, NAS unavailable | RF-DETR Seg |
| Quick proof-of-concept (<1000 images) | Roboflow Instant |
| Classification | ViT or DINOv3 |
| Multimodal / text prompts | Qwen3.5 or SmolVLM |

From roboflow/computer-vision-skills, Apache-2.0.

More from roboflow/computer-vision-skills