yolo-detection-2026

Files (14)

• SKILL.md
• config.yaml
• deploy.bat
• deploy.sh
• requirements.txt
• requirements_cpu.txt
• requirements_cuda.txt
• requirements_intel.txt
• requirements_mps.txt
• requirements_rocm.txt
• detect.py · scripts/
• env_config.py · scripts/
• yolo26n.onnx
• yolo26n_names.json

SKILL.md (first lines)

# YOLO 2026 Object Detection

Real-time object detection using the latest YOLO 2026 models. Detects 80+ COCO object classes including people, vehicles, animals, and everyday objects. Outputs bounding boxes with labels and confidence scores.

## Model Sizes

| Size | Speed | Accuracy | Best For |
|------|-------|----------|----------|
| nano | Fastest | Good | Real-time on CPU, edge devices |
| small | Fast | Better | Balanced speed/accuracy |
| medium | Moderate | High | Accuracy-focused deployments |
| large | Slower | Highest | Maximum detection quality |

## Hardware Acceleration

The skill uses [`env_config.py`](../../lib/env_config.py) to **automatically detect hardware** and convert the model to the fastest format for your platform. Conversion happens once during deployment and is cached.

| Platform | Backend | Optimized Format | Compute Units | Expected Speedup |
|----------|---------|------------------|:-------------:|:----------------:|
| NVIDIA GPU | CUDA | TensorRT `.engine` | GPU | ~3-5x |
| Apple Silicon (M1+) | MPS | CoreML `.mlpackage` | **Neural Engine** (NPU) | ~2x |
| Intel CPU/GPU/NPU | OpenVINO | OpenVINO IR `.xml` | CPU/GPU/NPU | ~2-3x |
| AMD GPU | ROCm | ONNX Runtime | GPU | ~1.5-2x |
| CPU (any) | CPU | ONNX Runtime | CPU | ~1.5x |

> **Apple Silicon Note**: Detection defaults to `cpu_and_ne` (CPU + Neural Engine), keeping the GPU free for LLM/VLM inference. Set `compute_units: all` to include GPU if not running local LLM.

### How It Works

1. `deploy.sh` detects your hardware via `env_config.HardwareEnv.detect()`
2. Installs the matching `requirements_{backend}.txt` (e.g. CUDA → includes `tensorrt`)
3. Pre-converts the default model to the optimal format
4. At runtime, `detect.py` loads the cached optimized model automatically
5. Falls back to PyTorch if optimization fails

Set `use_optimized: false` to disable auto-conversion and use raw PyTorch.

## Auto Start

Set `auto_start: true` in the skill config to start detection automatically when Aegis launches. The skill will begin processing frames from the selected camera immediately.

```yaml
auto_start: true
model_size: nano
fps: 5
```

## Performance Monitoring

The skill emits `perf_stats` events every 50 frames with aggregate timing:

```jsonl
{"event": "perf_stats", "total_frames": 50, "timings_ms": {
  "inference": {"avg": 3.4, "p50": 3.2, "p95": 5.1},
  "postprocess": {"avg": 0.15, "p50": 0.12, "p95": 0.31},
  "total": {"avg": 3.6, "p50": 3.4, "p95": 5.5}
}}
```

## Protocol

Communicates via **JSON lines** over stdin/stdout.

### Aegis → Skill (stdin)
```jsonl
{"event": "frame", "frame_id": 42, "camera_id": "front_door", "timestamp": "...", "frame_path": "/tmp/aegis_detection/frame_front_door.jpg", "width": 1920, "height": 1080}
```

### Skill → Aegis (stdout)
```jsonl
{"event": "ready", "model": "yolo2026n", "device": "mps", "backend": "mps", "format": "coreml", "gpu": "Apple M3", "classes": 80, "fps": 5}
{"event": "detections", "frame_id": 42, "camera_id": "front_door", "timestamp": "...", "objects": [
  {"class": "person", "confidence": 0.92, "bbox": [100, 50, 300, 400]}
]}
{"event": "perf_stats", "total_frames": 50, "timings_ms": {"inference": {"avg": 3.4}}}
{"event": "error", "message": "...", "retriable": true}
```

### Bounding Box Format
`[x_min, y_min, x_max, y_max]` — pixel coordinates (xyxy).

### Stop Command
```jsonl
{"command": "stop"}
```

## Installation

The `deploy.sh` bootstrapper handles everything — Python environment, GPU backend detection, dependency installation, and model optimization. No manual setup required.

```bash
./deploy.sh
```

### Requirements Files

| File | Backend | Key Deps |
|------|---------|----------|
| `requirements_cuda.txt` | NVIDIA | `torch` (cu124), `tensorrt` |
| `requirements_mps.txt` | Apple | `torch`, `coremltools` |
| `requirements_intel.txt` | Intel | `torch`, `openvino` |
| `requirements_rocm.txt` | AMD | `torch` (rocm6.2), `onnxruntime-rocm` |
| `requirements_cpu.txt` | CPU | `torch` (cpu), `onnxruntime` |

From SharpAI/DeepCamera, MIT.

More from SharpAI/DeepCamera