canvas-gestures-and-hit-testing

Files (63)

• SKILL.md
• shared.json · assets/
• template-placeholders.json · assets/
• intent-to-move.test.ts · examples/arena-aim/
• intent-to-move.ts · examples/arena-aim/
• hit-targets.test.ts · examples/inbox-planner/
• intent-to-action.test.ts · examples/inbox-planner/
• intent-to-action.ts · examples/inbox-planner/
• layout-planner.ts · examples/inbox-planner/
• planner-rules.ts · examples/inbox-planner/
• hit-targets.test.ts · examples/progress-grid/
• intent-to-action.test.ts · examples/progress-grid/
• intent-to-action.ts · examples/progress-grid/
• progress-grid-rules.ts · examples/progress-grid/
• use-progress-grid-input.ts · examples/progress-grid/
• intent-to-move.test.ts · examples/sliding-puzzle/
• intent-to-move.ts · examples/sliding-puzzle/
• sliding-puzzle-rules.ts · examples/sliding-puzzle/
• gesture-handler-versions.md · references/
• input-pipeline.md · references/
• intent-to-action.md · references/
• stick-input.md · references/
• testing-gestures.md · references/
• check-canvas-input.mjs · scripts/
• check-lib.mjs · scripts/
• input-scope.mjs · scripts/lib/
• source-scan.mjs · scripts/lib/
• selftest.mjs · scripts/
• canvas-input.json · templates/contracts/
• canvas-input.schema.json · templates/contracts/
• canvas-layout.test.ts · templates/src/canvas/
• canvas-layout.ts · templates/src/canvas/
• canvas-types.ts · templates/src/canvas/
• canvas-gesture-probe.tsx · templates/src/canvas/gestures/
• canvas-intent.ts · templates/src/canvas/gestures/
• classify-swipe.test.ts · templates/src/canvas/gestures/
• classify-swipe.ts · templates/src/canvas/gestures/
• pan-intent.test.ts · templates/src/canvas/gestures/
• pan-intent.ts · templates/src/canvas/gestures/
• stick-command.test.ts · templates/src/canvas/gestures/
• stick-command.ts · templates/src/canvas/gestures/
• stick-gesture-probe.tsx · templates/src/canvas/gestures/
• touch-targets.test.ts · templates/src/canvas/gestures/
• touch-targets.ts · templates/src/canvas/gestures/
• use-canvas-gestures.test.tsx · templates/src/canvas/gestures/
• use-canvas-gestures.ts · templates/src/canvas/gestures/
• sample.test.ts · templates/src/canvas/timeline/
• sample.ts · templates/src/canvas/timeline/
• track.ts · templates/src/canvas/timeline/
• package.json · tests/bases/app/
• quality-gates.json · tests/bases/app/
• _layout.tsx · tests/bases/app/src/app/
• surface-canvas.tsx · tests/bases/app/src/canvas/
• canvas-model.ts · tests/bases/app/src/features/tasks/canvas/
• layout-progress-grid.ts · tests/bases/app/src/features/tasks/canvas/
• progress-grid-surface.ts · tests/bases/app/src/features/tasks/canvas/
• to-view.ts · tests/bases/app/src/features/tasks/canvas/
• tasks-screen.tsx · tests/bases/app/src/features/tasks/
• use-tasks-model.ts · tests/bases/app/src/features/tasks/
• tsconfig.json · tests/bases/app/
• …and 3 more

SKILL.md (first lines)

# Canvas gestures and hit-testing

Every touch on a hand-drawn canvas (a progress grid, a planner, a chart, a game board) goes through one
gesture module, `useCanvasGestures`, which hit-tests against the same `CanvasLayout` the renderer draws
with and emits at most one typed `CanvasIntent` per gesture; the surface's pure `intentToAction(state,
intent)` alone decides whether it becomes a domain action. This skill ships that input kit
(`src/canvas/gestures/`), the shape of each surface's intent function and hit-targets test, and a
checker that proves both.

## Rules that must hold

1. **One gesture module builds every canvas gesture:** `src/canvas/gestures/use-canvas-gestures.ts`
   (`Gesture.*` builders on Gesture Handler 2.x, hooks on 3.x). No builder or gesture hook in the canvas
   kit, a surface folder, a file that renders a canvas or feeds one (a hook named in a canvas
   `GestureDetector` or `SurfaceCanvas input`), or the feature folder of a canvas that takes touch. A
   self-contained control there (a sheet's own `GestureDetector`) is ordinary UI; a gesture hook that is
   ordinary UI is listed in `contracts/canvas-input.json` `gestureExempt`. Why: a Gesture Handler major then rewrites one file, and no second gesture can
   double-fire on a canvas.
2. **One gesture yields at most one intent,** emitted in `onEnd` (`onStart` for long press), never in
   `onUpdate`/`onChange`; every pan has an `onFinalize` that resets the pointer (or the stick command).
   Why: a drag that emitted per update would apply many actions, and a cancelled drag (a call, the
   system back gesture) must not leave its preview behind.
3. **Hit-testing uses the surface's shared layout with slop:** `hitTest(layout.get(), point,
   HIT_SLOP)` on the derived layout the picture draws with (`useSurfacePlayer().layout`). Why: draw and
   touch cannot drift at any canvas size, and taps on the outer edge still land.
4. **Meaning and legality live only in the surface's pure intent function:** `intentToAction(state,
   intent)` in the surface folder beside `to-view.ts` (a game surface: `intent-to-move.ts`,
   `intentToMove`). It builds a candidate and returns it only when the domain's one source of truth
   accepts it (`listActions(state)` lists it, or `validate(state, action)` passes), and imports no
   React, React Native, Expo, Skia, store, service, API or UI value. Why: tests, undo, hints, bots and
   replays share one definition of allowed, and a finished or locked state gives `null`.
5. **The intent function handles every intent kind explicitly** (`tap`, `long-press`, `swipe`,
   `drag-end`, `aim`), returning `null` for the kinds the surface ignores. Why: an exhaustive switch
   turns a new kind into a compile error instead of a silent no-op.
6. **A tap-then-tap selection is screen UI state, never domain state or an action.** The model hook
   keeps it and passes it as `highlight.selected`; the gesture layer always sends `selected: null` and
   the model hook fills it in; a tap inside a select region only selects; a miss or an applied action
   clears it. Why: the store, undo, persistence, analytics and automated players never see a selection.
7. **The drag lift is applied once, in the gesture layer** (`surface.dragLiftPt`, default 0), to the
   hover, the drawn preview and the drop target; the intent function and `draw()` add none, and a
   surface with a lift keeps that much canvas below its last row (`unreachableByDrag` proves it). Why:
   the preview cell is then always the drop cell, and no edge row becomes unreachable.
8. **Touches are physical.** Gesture code never reads the writing direction; a surface that mirrors in
   right-to-left locales does it in its layout (regions mirrored as rectangles, never the touch point),
   and swipe directions stay physical. Why: `event.x/y` do not mirror under RTL (verified on iOS), so
   flipping again sends touches to the wrong cell.
9. **Per-frame finger state stays on the UI thread:** the `pointer` shared value feeds the picture
   (`frame.fx.pointer`); only hover changes cross to JS, behind `isSameTarget`. Why: a React render
   per finger move cannot fit a 120 Hz frame (8.3 ms).
10. **Every tappable region is at least `MIN_TOUCH_PT` on the smallest supported canvas,** proven by
    a hit-targets test with `smallestTargetPt`; `MIN_TOUCH_PT` follows `quality-gates.json`
    `a11y.minTouchPt` (44 pt; 48 for Material-sized Android designs). Fix the layout, never the test,
    never `hitSlop`. Why: accessibility-as-code's target minimum applies to canvas cells too.
11. **No React Native touch on a canvas, and the app's outermost view is `GestureHandlerRootView`.** No
    Pressable, Touchable or button inside a canvas `GestureDetector` or around a `<Canvas>`/`<SurfaceCanvas>`,
    and no `onPress*`/`onLongPress`/`onTouch*`/`onResponder*` on the canvas or on the element that directly
    wraps it; a whole-card press around a display-only canvas is an ordinary control, listed in
    `gestureExempt`. Why: React Native's touch system hit-tests on its own (no shared layout, no slop, no
    single intent) and competes with Gesture Handler; without the root view no gesture fires on a device.
12. **Everything a gesture callback calls is a `'worklet'` module** (`classify-swipe.ts`,
    `stick-command.ts`, `pan-intent.ts`, the kit's `canvas-layout.ts`) importing values only from other
    `'worklet'` modules; JS is reached with `scheduleOnRN` (never `runOnJS`), shared values with
    `.get()`/`.set()` (never `.value`). Why: a plain function throws on the UI thread on a device while
    every Jest test passes, and React Compiler needs the accessors.
13. **Gesture Handler follows the Expo SDK's pin** (`npx expo install`; ~2.32.0 on SDK 57): the builder
    API on 2.x, the hook API on 3.x, never mixed, the gesture module migrated whole. Why: the 3.x
    composition hooks accept only 3.x gestures.
14. **Continuous values are quantised before they enter state:** an aim vector becomes integers
    (thousandths of a unit vector, `Math.sqrt` only) inside the intent function. Why: saves stay
    compact and replays exact on every device.
15. **Continuous input for a simulation is an integer stick command** (games): 0 idle, 1 to 16 by the
    largest dot product with a committed table, a 12 pt dead zone, written only on change, 0 on lift.
    Why: a fixed-step simulation replays bit-exactly only from integer `(tick, command)` logs.
16. **Pans stay clear of the system back gestures:** the stack screen of a canvas that reads
    horizontal swipes or drags sets `fullScreenGestureEnabled: false` (iOS 26 turns full-screen back on
    by default) and the canvas keeps off the side edges (iOS edge swipe, Android gesture navigation).
    Why: the system claims those swipes first, so the move goes back instead.
17. **A hold-only action has a non-hold alternative** (a mode toggle on the screen, or an
    accessibility action on the canvas). Why: VoiceOver and TalkBack users cannot aim a hold at a cell.

## Workflow

1. **Read [references/input-pipeline.md](references/input-pipeline.md)** and pick the surface's pan
   mode (`none`, `swipe`, `drag`, `aim`; a simulation uses the stick,
   [references/stick-input.md](references/stick-input.md)). Input needs the canvas kit's
   `src/canvas/canvas-layout.ts` and `canvas-types.ts`: when they are missing, set the kit up with
   `skia-canvas-rendering` first (this skill's `templates/src/canvas/` holds identical synced copies).
2. **Copy the input kit.** Copy `templates/src/canvas/gestures/` to `src/canvas/gestures/` verbatim,
   and `templates/contracts/canvas-input.json` with its schema to `contracts/` (replace `__FEATURE__`
   and `__SURFACE__`, list `buttonComponents`). Make `GestureHandlerRootView` the app's outermost view
   (input-pipeline.md, "The root view"). Run `node ${CLAUDE_SKILL_DIR}/scripts/check-canvas-input.mjs .`:
   it names every missing piece.
3. **Write the intent function test-first.** Read
   [references/intent-to-action.md](references/intent-to-action.md) and shape it like
   [examples/progress-grid/](examples/progress-grid/) (tap to act),
   [examples/inbox-planner/](examples/inbox-planner/) (drag with a lift, tap-then-tap),
   [examples/sliding-puzzle/](examples/sliding-puzzle/) (swipe) or
   [examples/arena-aim/](examples/arena-aim/) (aim). Keep the finished-state, ignored-kind and
   legality-property tests. A game surface keeps `intent-to-move.ts` in its canvas folder (game rules
   cannot import the canvas kit); games may import `src/canvas/**` by default
   (architecture-and-boundaries' `gameFacing` defaults), so no contract entry is needed.
4. **Prove touch targets.** Write the surface's `hit-targets.test.ts` like the examples: the largest
   view laid out on the smallest supported canvas, `smallestTargetPt(layout) >= MIN_TOUCH_PT`, and with
   a lift `unreachableByDrag(layout, edgeRowTargets(...), DRAG_LIFT_PT)` empty on portrait and wide
   canvases. A region below the minimum changes the layout, not the test.
5. **Wire the screen.** The feature's model hook calls `useCanvasGestures(player.layout, { panMode,
   dragLiftPt: surface.dragLiftPt ?? 0, onIntent, onHover, onMiss })`, keeps the selection, dispatches
   non-null actions and hands `input` and `highlight` to `<SurfaceCanvas>`
   ([examples/progress-grid/use-progress-grid-input.ts](examples/progress-grid/use-progress-grid-input.ts)).
   Set the stack screen's gesture options (rule 16) in the route's layout.
6. **Run the tests:** `npx jest src/canvas/gestures src/features/<feature> --ci --selectProjects unit`
   (paths first: `--selectProjects` takes several values and would swallow them). Gesture tests follow
   [references/testing-gestures.md](references/testing-gestures.md).
7. **Lint, format and type-check:** `npx eslint --fix` and `npx prettier --write` on the new files, then
   `npm run typecheck`.
8. **Run the check** `node ${CLAUDE_SKILL_DIR}/scripts/check-canvas-input.mjs .` (or `--surface <id>`
   to limit the surface side), fix every `FAIL` line (each names the file, the rule and the fix) and
   rerun until it prints `RESULT: PASS`.
9. **Touch it for real** on a named iOS simulator and a named Android emulator (testing-gestures.md,

From zakariaf/ReactNative-Skills, MIT.

More from zakariaf/ReactNative-Skills