audio-and-haptics-feedback

Files (82)

• SKILL.md
• shared.json · assets/
• template-placeholders.json · assets/
• bundled-sound-loading.ts · examples/
• listening-checklist.md · examples/
• root-layout-wiring.tsx · examples/
• audio-engine.md · references/
• feedback-moments.md · references/
• haptics.md · references/
• sound-design.md · references/
• check-audio-haptics.mjs · scripts/
• check-lib.mjs · scripts/
• check-sound-banks.mjs · scripts/
• app-modules.mjs · scripts/lib/
• synth.mjs · scripts/lib/
• ts-literal.mjs · scripts/lib/
• selftest.mjs · scripts/
• react-native-audio-api.ts · templates/__mocks__/
• audio-feedback.json · templates/contracts/
• audio-feedback.schema.json · templates/contracts/
• audio-plugin.test.ts · templates/src/config/
• audio-plugin.ts · templates/src/config/
• sfc32.test.ts · templates/src/core/rng/
• sfc32.ts · templates/src/core/rng/
• __FEATURE__-feedback.test.ts · templates/src/features/__FEATURE__/
• __FEATURE__-feedback.ts · templates/src/features/__FEATURE__/
• __FEATURE__-sounds.test.ts · templates/src/features/__FEATURE__/sounds/
• __FEATURE__-sounds.ts · templates/src/features/__FEATURE__/sounds/
• use-is-app-active.test.ts · templates/src/hooks/
• use-is-app-active.ts · templates/src/hooks/
• admit-voice.test.ts · templates/src/services/audio/
• admit-voice.ts · templates/src/services/audio/
• audio-api-audio-adapter.android.test.ts · templates/src/services/audio/
• audio-api-audio-adapter.test.ts · templates/src/services/audio/
• audio-api-audio-adapter.ts · templates/src/services/audio/
• audio-port.ts · templates/src/services/audio/
• compose-sound-bank.test.ts · templates/src/services/audio/
• compose-sound-bank.ts · templates/src/services/audio/
• fake-audio.test.ts · templates/src/services/audio/
• fake-audio.ts · templates/src/services/audio/
• recipe-problems.test.ts · templates/src/services/audio/synth/
• recipe-problems.ts · templates/src/services/audio/synth/
• synthesize-recipe.test.ts · templates/src/services/audio/synth/
• synthesize-recipe.ts · templates/src/services/audio/synth/
• ui-sounds.ts · templates/src/services/audio/synth/
• use-audio-lifecycle.test.ts · templates/src/services/audio/
• use-audio-lifecycle.ts · templates/src/services/audio/
• play-cue.test.ts · templates/src/services/feedback/
• play-cue.ts · templates/src/services/feedback/
• ui-feedback.test.ts · templates/src/services/feedback/
• ui-feedback.ts · templates/src/services/feedback/
• expo-haptics-adapter.android.test.ts · templates/src/services/haptics/
• expo-haptics-adapter.test.ts · templates/src/services/haptics/
• expo-haptics-adapter.ts · templates/src/services/haptics/
• fake-haptics.test.ts · templates/src/services/haptics/
• fake-haptics.ts · templates/src/services/haptics/
• haptics-port.ts · templates/src/services/haptics/
• should-pulse.test.ts · templates/src/services/haptics/
• should-pulse.ts · templates/src/services/haptics/
• recording-feedback.test.ts · templates/src/services/test-hooks/
• …and 22 more

SKILL.md (first lines)

# Audio and haptics feedback

Every sound an Expo app plays as feedback is synthesised from a literal recipe and played through one
ambient `AudioContext` behind `AudioPort`; every pulse is a cue from a fixed table played through
`HapticsPort` on iOS and Android, gated by the user's preference and throttled. This skill builds the
ports, the cue tables, the banks, and the two checkers that prove them.

## Rules that must hold

1. **One engine each, behind ports.** Only `src/services/audio/audio-api-audio-adapter.ts` imports
   `react-native-audio-api` and only `src/services/haptics/expo-haptics-adapter.ts` imports
   `expo-haptics`; nothing imports `react-native-sound` or `react-native-haptic-feedback`, the feedback
   layer imports no media player, and `Vibration` never appears in the feedback layer or a view
   (elsewhere only with `// vibration-exempt: <reason>`). Tests use `createFakeAudio()`/`createFakeHaptics()`.
   Why: one file changes on a bump, two engines fight over the session, and tests run without native code.
2. **Pin react-native-audio-api exactly and override all five plugin defaults** (`iosBackgroundMode:
   false`, `androidPermissions: []`, `androidForegroundService: false`, `disableFFmpeg: true`,
   `disableStaticExternalLibs: true`) through `AUDIO_API_PLUGIN` in `app.config.ts`; a new version goes
   through "Before any bump" and is recorded in `contracts/audio-feedback.json`. Why: the library is
   pre-1.0, and the defaults add background audio, Android foreground-service permissions and
   build-time downloads.
3. **One AudioContext, created in the adapter after `setAudioSessionOptions({ iosCategory: 'ambient' })`,
   and no audio focus on Android** (`observeAudioInterruptions(true)` only under `Platform.OS === 'ios'`).
   Why: ambient respects the silent switch and mixes with the user's music; on Android that call requests
   `AUDIOFOCUS_GAIN`, which fades out and mutes the user's music.
4. **Every sound is a literal `SoundRecipe` that passes `recipeProblems`** (finite, peak 0.05-0.9, first
   sample < 0.05, last < 0.01, later layers fade in over at least 1 ms, 10-1500 ms, music up to 30 s);
   the app's synth renders byte-identically to the checker's port. No streaming and no URL audio in the
   feedback layer; every recorded audio file in the repo is listed with its licence and source in the
   contract (`bundledSounds` for feedback, `otherAudio` for a notification sound or a feature's media).
   Why: no clicks, no clipping, no licences to chase, previews that are exactly what the device plays.
5. **Banks and ids:** kebab-case nouns for what happened; `ui.` is reserved for `UI_SOUNDS`; each feature
   or game bank lives in `<owner>/sounds/<owner>-sounds.ts` as `<OWNER>_SOUNDS`, is tested, and is
   composed once in the root layout with `composeSoundBank` (which throws on a clash). Every cue names an
   id of its owner's bank and every sfx id is cued. Why: a typo is silence in production, a clash
   silently replaces another sound.
6. **Keep the voice policy:** three category gains (`sfx` and `ui` follow sound effects, `music`
   follows music), `volume²`, 50 ms ramps, at most 8 overlapping voices, the same sound at most once per
   30 ms (or its `minIntervalMs`, at most 1000), voices as intervals on the audio clock, released by
   `cancelPending`. Why: even loudness, no clicks, no pile-ups, no sound lost after a fast action.
7. **Music is off by default, never `activelyReclaimSession`, and every music control is hidden when
   `hasMusic(bank)` is false.** Why: the library cannot tell whether the user's own audio plays, so the
   app never plays over it unasked.
8. **Feedback fires from events on JS:** a feature's cue table played by its model (`playCue`), a canvas
   track cue scheduled on the audio clock, a UI moment (`playUiFeedback`), or drained sim events; never
   from a view's own handler, a sim, a `'worklet'` module or per frame, and each event in one place
   only; views (`src/ui`, a feature's screen or components) never call `audio.play`, `playCue` or
   `playUiFeedback`. Why: sync with the animation, and the UI thread never touches a port.
9. **Mount `useAudioLifecycle` once, in the root layout;** the context runs only while the app is active
   and nothing full-screen plays; iOS interruptions suspend it and resume it only if the lifecycle
   still wants it running (with a direction gate, mount it and load the bank behind the gate); `await audio.dispose()`
   before every JS reload, or await an injected `beforeReload` hook that the caller wires to
   `() => services.audio.dispose()`, or say why with `// audio-dispose-exempt: <reason>`. Why: no sound
   in the background or over a video, and the native engine closes before the bundle reloads.
10. **Haptics follow the cue table on both platforms** (iOS feedback generators; Android
    `performAndroidHapticsAsync` constants, an API 30 table and a legacy one), only while the user's
    preference is on, at least 40 ms apart, each call ending in `.catch(() => undefined)` inside the
    adapter; `isSupported` is false on iPad. Views use only `selection` (switches, pickers). Why: bursts
    feel like a buzz, a newer Android constant rejects on older phones, and buttons should not buzz.
11. **The app-wide moments go through `UI_FEEDBACK`** (by default `toggle`, `success`, `error`; the
    table is the app's), each moment it keeps played from a model; buttons are silent by default, and an
    app that adds a `tap` moment renders `<PressFeedbackProvider playsSound onPress={...}>` that plays
    it (the canonical press surface then sets `android_disableSound` itself; no synced file is edited).
    Why: one forgotten handler is a silent switch, and Android would play its own click on top of the tap.
12. **Add no microphone purpose string up front;** only if an App Store upload reports ITMS-90683, set
    `iosMicrophonePermission` (localised, with a comment naming ITMS-90683). Why: feedback never records,
    and an unused purpose string invites review questions.
13. **A green gate proves a sound is clean, not right.** Render the WAV previews, list the listening and
    the feel on a real iPhone and Android phone for a human, and never block on the answer; test builds
    prove requests with the recording decorators. Why: an agent cannot hear and simulators have no
    haptic engine.

## Workflow

1. **Pick the job.** A new feedback layer: steps 2-6. Sounds for a feature or game: steps 7-9. Tuning a
   sound: steps 8-9. Bumping the library: step 10. Always finish with step 11.
2. **Read** [references/audio-engine.md](references/audio-engine.md) (library, plugin, port, adapter,
   session and focus, lifecycle, music, wiring, testing) and [references/haptics.md](references/haptics.md)
   (cue table on both platforms, throttle, devices).
3. **Copy the templates** at the paths their first line names (tests included): `templates/src/services/audio/`
   (with `synth/`), `templates/src/services/haptics/`, `templates/src/services/feedback/`,
   `templates/src/services/test-hooks/recording-feedback.ts` (test builds only),
   `templates/src/config/audio-plugin.ts`, `templates/__mocks__/react-native-audio-api.ts`,
   `templates/contracts/audio-feedback.json` (+ schema). `src/hooks/use-is-app-active.ts`,
   `src/ui/press-feedback-context.tsx` and `src/core/rng/sfc32.ts` belong to other skills; copy them
   only if the app lacks them. A haptics-only app copies `haptics/` alone; the checker then notes that
   the audio rules do not apply.
4. **Install:** `npx expo install expo-haptics` and `npm install --save-exact react-native-audio-api@0.13.6`
   (dependency-management records both in its policy). Add `AUDIO_API_PLUGIN` to `app.config.ts`
   plugins; check that `.gitignore` ignores `reports/`.
5. **Wire the composition root** (the lines are in audio-engine.md, "Wiring"): `audio` and `haptics` in
   `Services`, `createDeviceServices` and `createTestServices`; the haptics preference read at call
   time (settings-and-preferences owns it). In the root layout, `services.audio.load(composeSoundBank(...))`
   in the startup initializer and `useAudioLifecycle(...)` once; imitate
   [examples/root-layout-wiring.tsx](examples/root-layout-wiring.tsx). Make every reload path dispose first.
6. **Wire the moments** ([references/feedback-moments.md](references/feedback-moments.md)):
   `playUiFeedback(services, 'toggle')` in switch handlers (the canonical one: the settings model's
   `onToggled`), `'success'`/`'error'` where a requested action ends. Tap sounds only if the app wants
   them (the opt-in there).
7. **Plan the sounds.** Read [references/sound-design.md](references/sound-design.md). Write the event →
   sound → haptic table: most events get nothing, three to six sounds per feature, the most repeated the
   quietest, loud and long only for rare moments.
8. **Write the bank and the cue table test-first.** Copy `templates/src/features/__FEATURE__/` (bank,
   cue table, their tests), replace `__FEATURE__`, `__FEATURE_PASCAL__`, `__FEATURE_SOUNDS__` and
   `__FEATURE_FEEDBACK__`, write one literal recipe per sound, play each row from the model with
   `playCue` (or put it on the canvas track), add the bank to `composeSoundBank`, run
   `npx prettier --write src/features/<feature>` on the copied files, and run
   `npx jest src/features/<feature> --ci --selectProjects unit` until green (paths before the flag).
9. **Render and hand over the listening:** `node ${CLAUDE_SKILL_DIR}/scripts/check-sound-banks.mjs . --wav-dir reports/sfx`,
   fix every `FAIL`, then list the listening as in [examples/listening-checklist.md](examples/listening-checklist.md).
   Carry on; when a human answers, tune one number at a time and re-render.
10. **Bumping react-native-audio-api:** follow "Before any bump" in audio-engine.md (release notes,
    plugin defaults, mock, focus code, adapter tests, a smoke test on a named iOS simulator and Android
    emulator, a listening check), then set `audioApiVersion` in `contracts/audio-feedback.json` with the pin.
11. **Run the checks** until both print `RESULT: PASS`: `node ${CLAUDE_SKILL_DIR}/scripts/check-sound-banks.mjs .`
    then `node ${CLAUDE_SKILL_DIR}/scripts/check-audio-haptics.mjs .`. Each `FAIL` names the file, the
    rule and the fix. Then `npx jest src/services src/features/<feature> --ci --selectProjects unit android`.

## Definition of done

From zakariaf/ReactNative-Skills, MIT.

More from zakariaf/ReactNative-Skills