img2threejs

Files (90)

• FUNDING.yml · .github/
• .gitignore
• CHANGELOG.md
• CONTRIBUTING.md
• LICENSE
• README.md
• ROADMAP.md
• SKILL.md
• openai.yaml · agents/
• crown-chest.gif · assets/
• demo.gif · assets/
• doraemon-house.gif · assets/
• gerber-knife.gif · assets/
• issaca-shotgun.gif · assets/
• logo.svg · assets/
• sony-wf1000xm3.gif · assets/
• warhauler.gif · assets/
• TOKEN_COST.md · docs/
• UPGRADE_PLAN.md · docs/
• artifact_cache.py · forge/_shared/
• color_metrics.py · forge/_shared/
• feature_acceptance_policy.py · forge/_shared/
• image_hash.py · forge/_shared/
• requirements.txt · forge/
• analyze_texture.py · forge/stage1_intake/
• bind_detail_properties.py · forge/stage1_intake/
• build_detail_inventory.py · forge/stage1_intake/
• check_intake_correctness.py · forge/stage1_intake/
• check_reference_admission.py · forge/stage1_intake/
• delight_albedo.py · forge/stage1_intake/
• detect_reference_effects.py · forge/stage1_intake/
• extract_gradient_stops.py · forge/stage1_intake/
• extract_landmarks.py · forge/stage1_intake/
• extract_part_color_recipe.py · forge/stage1_intake/
• extract_pbr_evidence.py · forge/stage1_intake/
• probe_image.py · forge/stage1_intake/
• solve_camera_pose.py · forge/stage1_intake/
• derive_geometry.py · forge/stage2_spec/
• new_pre_spec_assessment.py · forge/stage2_spec/
• new_sculpt_spec.py · forge/stage2_spec/
• validate_sculpt_spec.py · forge/stage2_spec/
• bake_projected_texture.py · forge/stage3_build/
• generate_threejs_factory.py · forge/stage3_build/
• module_cache.py · forge/stage3_build/
• orchestrate_passes.py · forge/stage3_build/
• append_review.py · forge/stage4_review/
• calibrate_eye.py · forge/stage4_review/
• correction_loop.py · forge/stage4_review/
• diagnose_render.py · forge/stage4_review/
• diagnose_render_multi_angle.py · forge/stage4_review/
• divine_eye.py · forge/stage4_review/
• make_comparison_sheet.py · forge/stage4_review/
• objectness.py · forge/stage4_review/
• per_feature.py · forge/stage4_review/
• vlm_gate.py · forge/stage4_review/
• test_analyze_texture.py · forge/tests/
• test_calibrate_eye.py · forge/tests/
• test_color_metrics.py · forge/tests/
• test_color_recipe.py · forge/tests/
• test_correction_loop.py · forge/tests/
• …and 30 more

SKILL.md (first lines)

# img2threejs — Image to procedural Three.js

Rebuild the object visible in a reference image as a **code-only** procedural Three.js model,
gated by a staged sculpting pipeline and an AI-vision self-correction loop. This is
reconstruction-by-code, **not** photogrammetry, mesh extraction, or downloaded art packs.

Agent-agnostic: works under Claude Code, Codex, OpenCode, or WebDesigner hosts. Wherever this doc says "agent
vision" or "agent browser tool", use whatever the host provides — native image reading, a
browser MCP (playwright/chrome-devtools), the project preview, or a user-supplied screenshot.

## WebDesigner Contract

- **Stage**: `build` (optional design-stage prep when a reference image must be assessed first)
- **Reads**: `TaskIntent`, `StackSelection`, design artifacts, reference image path, generated workspace
- **Emits artifacts**: `object-sculpt-spec`, `threejs-factory`, `img2threejs-review-log`, optional `comparison-sheet`
- **Upstream**: [img2threejs v1.3](https://github.com/hoainho/img2threejs/releases/tag/v1.3) (`2c78038`)
- **Activation**: `StackSelection.integrations` contains `img2threejs`, or `TaskIntent.constraints.requiresImageToThreeJS` is true, or the user explicitly asks for image-to-Three.js reconstruction

### WebDesigner Rules

- Work only inside the generated application workspace; never install Three.js into the WebDesigner control plane.
- Require `nextjs` or `react-vite`, React, and TypeScript. If the workspace is incompatible, record the reason and stop or offer a framework-native fallback.
- Run forge scripts from this skill root (`.antigravity/skills/img2threejs`) with Python 3.10+ stdlib only — no pip installs into the control plane.
- Emit the factory TypeScript into the generated workspace (for example `src/three/createObjectModel.ts` or an equivalent app-local path).
- Install `three` (and `@types/three` when needed) only in the generated workspace package manager.
- Keep the control-plane skill assets (forge scripts, grimoire rubrics) separate from deployable app code.
- Prefer a canvas/WebGL viewer surface that mounts the returned `THREE.Group`; expose `root.userData.sculptRuntime` for animation, sockets, and colliders.
- Record pass reviews, comparison sheets, and factory paths in the `ArtifactManifest`.

### WebDesigner Process

1. Confirm `integrations` contains `img2threejs` or that the user explicitly requested image-to-Three.js.
2. Inspect the generated workspace, package manager, TypeScript setup, and reference image path.
3. Follow the staged sculpting loop below (probe → assess → detail inventory → spec → validate → pass-gated codegen → render → comparison review).
4. Install Three.js only if missing from the generated workspace.
5. Mount the generated factory in a viewer component that fits the selected frontend runtime.
6. Emit `object-sculpt-spec`, `threejs-factory`, and `img2threejs-review-log` artifacts; include comparison-sheet paths when available.

## When To Use

The user attaches/points to an object image and wants a procedural Three.js model, a
reconstruction/animation/destruction plan, a sculpt spec, or code. Also for material studies,
action-ready props, game objects, botanical/mechanical parts, and stylized reconstructions.
In WebDesigner, activate when product UI needs a procedural 3D prop rebuilt from a reference image.

## Core Promise

Sculpt from a photo, in order — never one-shot a mesh:
1. **Validate** the image is a suitable 3D target (`grimoire/intake/validation_rubric.md`).
2. **Assess** object class + complexity, then write a `qualityContract` before any code.
3. **Spec** it: component hierarchy, materials, lighting, pivots, sockets, action anchors.
4. **Build pass-by-pass** from blockout → structure → form → material → lighting → interaction → optimization.
5. **Verify** each pass with a screenshot compared against the reference; fail a pass if an identity-defining feature is wrong even when the global score looks fine.

State explicitly when output is approximate/stylized/low-poly. A single image cannot reveal
hidden sides or guarantee exact geometry — say so instead of faking confidence.

## Required Inputs

- one image path / screenshot / URL / attached image (if missing or unreadable, ask)
- intended use: prop, game object, hero render, playable/destructible object, animation rig
  (default: real-time browser prop with interactive performance)

## The Loop (scripts do enforcement; agent vision does judgment)

Run scripts from the skill root (`forge/...`). Pure Python 3.10+ stdlib, no pip installs.
Full flags: `grimoire/scripts.md`. Never let a script *score* visuals — that is the agent's job.

1. Probe local images: `forge/stage1_intake/probe_image.py <image>` (metadata only, not a visual check).
2. **Pre-Spec Assessment Gate** — classify + score complexity + write the quality contract:
   `forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --complexity <simple|moderate|complex|ultra-complex> --out assessment.json`. Rules: `grimoire/intake/quality_contract.md`.
   Set `objectClass.primaryDomain` (`object` | `character` | `hybrid`) and fill the seeded
   `detailInventory` (its `targetMinDetails` scales with complexity).
2b. **Detail inventory** (do not skip for detailed subjects) — scan zones and enumerate every
   identity-defining small detail (gloss, bevel, fasteners, linework, contours, stains):
   `forge/stage1_intake/build_detail_inventory.py <image> --mode grid-3x3 --out-dir <dir> --out di.json`.
   Each detail MUST map to a `component.localFeatures` or `material.localOverrides` entry — never
   prose only. Taxonomy + 3D-term recipes: `grimoire/intake/detail_inventory.md`.
2c. **Character/hybrid subjects** — capture head-unit proportions + facial/body landmarks:
   `forge/stage1_intake/extract_landmarks.py <image> --out anatomy.json --overlay overlay.png`, then
   fill `preSpecAssessment.anatomy`. Route: `grimoire/character/reconstruction.md`. For maximum
   likeness use the projection-first path (`grimoire/character/likeness_maximization.md`): solve the camera
   (`stage1_intake/solve_camera_pose.py`), de-light the photo (`stage1_intake/delight_albedo.py`), and project it onto
   the fitted mesh (`stage3_build/bake_projected_texture.py`). A single image cannot guarantee 100% likeness —
   report per-region confidence and request more views for a real person.
3. Author the spec from the assessment:
   `forge/stage2_spec/new_sculpt_spec.py "Name" --image <img> --assessment assessment.json --out object-sculpt-spec.json`.
   Replace generic starter `featureReviewTargets` with the object's real identity-defining
   systems (≤5 critical, ≤3 important per pass); for characters add `anatomy-proportion`,
   `face-landmark-placement`, `pose-silhouette`, `outfit-and-palette`. Use 3D-graphics terms only
   (`grimoire/glossary/3d_vocabulary.md`), never "nice/smooth/shiny". Classify every component's
   `topologyClass`/`topologyRationale` per `grimoire/intake/surface_topology.md` before picking a
   `primitive` — this is what prevents a continuous organic form from being picked as a box.
4. When material fidelity matters and a source image exists, analyze each material's **finish** then
   extract reference PBR evidence, both per crop (crop the correct region — verify the crop is on the
   part you think it is):
   - `forge/stage1_intake/analyze_texture.py <crop> --spec spec.json --material-id <id> --in-place`
     classifies the finish (`gem-metal | gemstone | painted-metal | worn-composite | brushed-steel |
     plastic`), extracts the gradient palette, and writes doc-grounded MeshPhysicalMaterial scalars
     (metalness/roughness/clearcoat/transmission/ior/anisotropy/envMapIntensity) onto the material.
     Recipes + Three.js texture/PBR rules (colorSpace, CanvasTexture/DataTexture, height→normal) live
     in `grimoire/build/threejs_texture_reference.md`. Rule of thumb: **solid albedo for flat paint,
     real reference crop for patterned finishes** (doppler/quartz/hydro-dip/camo).
   - `forge/stage1_intake/extract_pbr_evidence.py <crop> --out-dir <dir> --material-id <id> --target-threshold 0.7`.
   Confidence < 0.7 is a stop/refine-input signal, not a pass. It is inference, not inverse rendering.
5. Validate, then strict-validate before generating code:
   `forge/stage2_spec/validate_sculpt_spec.py object-sculpt-spec.json` then `--strict-quality`.
   Strict blocks shallow specs (a complex object with one root, no repetition systems, no
   local overrides, no micro groups is NOT implementation-ready even if JSON validates).
6. **Locked build passes** — only touch the currently unlocked pass:
   `forge/stage3_build/orchestrate_passes.py status object-sculpt-spec.json`
   `forge/stage3_build/orchestrate_passes.py check object-sculpt-spec.json --pass-id <pass>`
   `forge/stage3_build/generate_threejs_factory.py object-sculpt-spec.json --out src/createObjectModel.ts`
   (generator is pass-gated: a future `--pass-id` fails until prior passes are reviewed `continue`).
7. Render the current pass in a browser/preview, capture a screenshot at a review viewpoint.
8. Package one side-by-side sheet, then inspect it with agent vision:
   `forge/stage4_review/make_comparison_sheet.py --reference <img> --render <shot> --out cmp.png --json`.
9. Record the review (overall + per-layer + per-feature scores + decision):
   `forge/stage4_review/append_review.py object-sculpt-spec.json --pass-id <pass> --fidelity <0-1> --action <continue|refine-spec|refine-code|request-input|stop> --summary "..." --render-screenshot <shot> --comparison-image cmp.png --ai-vision-score <0-1> --layer-scores-json '{...}' --feature-reviews-json <f.json> --in-place`.

From uset82/webdesigner, MIT.

More from uset82/webdesigner