depth-estimation

Files (200)

• branch-management.md · .agents/workflows/
• command-execution.md · .agents/workflows/
• FUNDING.yml · .github/
• bug_report.md · .github/ISSUE_TEMPLATE/
• custom.md · .github/ISSUE_TEMPLATE/
• feature_request.md · .github/ISSUE_TEMPLATE/
• jekyll-gh-pages.yml · .github/workflows/
• pr-target-check.yml · .github/workflows/
• .gitignore
• .travis.yml
• CODE_OF_CONDUCT.md
• CONTRIBUTING.md
• Contributions.md
• LICENSE
• README.md
• SKILL.md
• _config.yml
• .env · docker/
• .gitignore · docker/
• Dockerfile · docker/broker/
• Dockerfile.arm64v8 · docker/broker/
• Dockerfile.x86 · docker/broker/
• README.md · docker/broker/
• brokerConfig.js · docker/broker/
• docker-entrypoint.sh · docker/broker/
• build.sh · docker/
• build_x86.sh · docker/
• docker-compose-arm64v8.yml · docker/
• docker-compose-nano-r32.6.1.yml · docker/
• docker-compose-x86.yml · docker/
• docker-compose.yml · docker/
• env_statement.md · docker/
• Dockerfile · docker/flower/
• Dockerfile.arm64v8 · docker/flower/
• Dockerfile.nano_r32.6.1 · docker/flower/
• Dockerfile.x86 · docker/flower/
• Dockerfile · docker/minio/
• Dockerfile.arm64v8 · docker/minio/
• Dockerfile.x86 · docker/minio/
• config.json · docker/minio/
• .gitignore · docker/od/
• Dockerfile · docker/od/
• Dockerfile.arm64v8 · docker/od/
• Dockerfile.x86 · docker/od/
• labelmap.prototxt · docker/od/assets/
• checkpoint · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• frozen_inference_graph.pb · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• model.ckpt.data-00000-of-00001 · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• model.ckpt.index · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• model.ckpt.meta · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• mscoco_label_map.pbtxt · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• pipeline.config · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/
• saved_model.pb · docker/od/assets/models/ssdlite_mobilenet_v2_coco_2018_05_09/saved_model/
• requirements.txt · docker/od/assets/
• sources.list.aliyun · docker/od/assets/
• build.sh · docker/od/
• build_x86.sh · docker/od/
• od.py · docker/od/
• panda.jpg · docker/od/
• test.py · docker/od/
• …and 140 more

SKILL.md (first lines)

# Depth Estimation (Privacy)

Real-time monocular depth estimation using Depth Anything v2. Transforms camera feeds with colorized depth maps — near objects appear warm, far objects appear cool.

When used for **privacy mode**, the `depth_only` blend mode fully anonymizes the scene while preserving spatial layout and activity, enabling security monitoring without revealing identities.

## Hardware Backends

| Platform | Backend | Runtime | Model |
|----------|---------|---------|-------|
| **macOS** | CoreML | Apple Neural Engine | `apple/coreml-depth-anything-v2-small` (.mlpackage) |
| Linux/Windows | PyTorch | CUDA / CPU | `depth-anything/Depth-Anything-V2-Small` (.pth) |

On macOS, CoreML runs on the Neural Engine, leaving the GPU free for other tasks. The model is auto-downloaded from HuggingFace and stored at `~/.aegis-ai/models/feature-extraction/`.

## What You Get

- **Privacy anonymization** — depth-only mode hides all visual identity
- **Depth overlays** on live camera feeds
- **3D scene understanding** — spatial layout of the scene
- **CoreML acceleration** — Neural Engine on Apple Silicon (3-5x faster than MPS)

## Interface: TransformSkillBase

This skill implements the `TransformSkillBase` interface. Any new privacy skill can be created by subclassing `TransformSkillBase` and implementing two methods:

```python
from transform_base import TransformSkillBase

class MyPrivacySkill(TransformSkillBase):
    def load_model(self, config):
        # Load your model, return {"model": "...", "device": "..."}
        ...

    def transform_frame(self, image, metadata):
        # Transform BGR image, return BGR image
        ...
```

## Protocol

### Aegis → Skill (stdin)
```jsonl
{"event": "frame", "frame_id": "cam1_1710001", "camera_id": "front_door", "frame_path": "/tmp/frame.jpg", "timestamp": "..."}
{"command": "config-update", "config": {"opacity": 0.8, "blend_mode": "overlay"}}
{"command": "stop"}
```

### Skill → Aegis (stdout)
```jsonl
{"event": "ready", "model": "coreml-DepthAnythingV2SmallF16", "device": "neural_engine", "backend": "coreml"}
{"event": "transform", "frame_id": "cam1_1710001", "camera_id": "front_door", "transform_data": "<base64 JPEG>"}
{"event": "perf_stats", "total_frames": 50, "timings_ms": {"transform": {"avg": 12.5, ...}}}
```

## Setup

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

From SharpAI/DeepCamera, MIT.

More from SharpAI/DeepCamera