debugging-surveys

Files (3)

• SKILL.md
• diagnostic-queries.md · references/
• reading-responses.md · references/

SKILL.md (first lines)

# Debugging surveys

PostHog Surveys is a no-code in-app form builder. A customer creates a survey in the
PostHog UI; it must then be evaluated and rendered by whichever SDK their app runs.
**Most "survey not showing" tickets are eligibility problems, not rendering bugs** — the
SDK correctly decided the user is not eligible, and the job is to find _which_ gate
failed and _why_.

## Access requirements

Use PostHog MCP tools or the survey API to inspect survey configuration and responses.
A checkout of the PostHog repository is not required for these diagnostic steps.

## Cross-SDK feature parity (check this FIRST)

A large class of tickets is "customer expects a feature their platform doesn't support."
Confirm the survey's `lib` / the customer's platform before anything else, then consult
this table. Verified against the SDK source — re-verify if it's been months, the gaps
get filled over time.

| Feature                          | Web (posthog-js)                | iOS                                       | Android                            | Flutter                            | React Native                      |
| -------------------------------- | ------------------------------- | ----------------------------------------- | ---------------------------------- | ---------------------------------- | --------------------------------- |
| Rendering                        | DOM + shadow root               | Native SwiftUI (`SurveysWindow`)          | **No built-in UI** — delegate only | Dart widgets (`SurveyBottomSheet`) | RN components (`SurveyModal`)     |
| Event-based triggers             | yes (since 1.137.0, 2024-06-05) | yes                                       | yes                                | yes (native side)                  | yes                               |
| URL / screen targeting           | yes                             | decoded but **NOT evaluated** (`// TODO`) | decoded but **NOT evaluated**      | **NOT evaluated** (native gap)     | **explicitly excluded** in filter |
| Feature-flag / cohort targeting  | yes                             | yes                                       | yes                                | yes (native side)                  | yes                               |
| `seenSurveyWaitPeriodInDays`     | yes                             | yes                                       | yes                                | yes (native side)                  | yes (≥ 4.81.5)                    |
| `surveyPopupDelaySeconds`        | yes                             | **not implemented**                       | **not implemented**                | **not implemented**                | **not implemented** (TODO)        |
| `enable_partial_responses`       | yes (≥ 1.240.0)                 | **no**                                    | **no**                             | **no**                             | **no**                            |
| `skipSubmitButton` (auto-submit) | yes (≥ 1.244.0)                 | **no**                                    | **no**                             | **no**                             | **no**                            |

The last two rows are per the editor's own help text ("Doesn't work with the mobile SDKs for
now" / "Not available for the mobile SDKs at the moment") rather than a per-SDK source audit.

Consequences worth memorizing:

- **`surveyPopupDelaySeconds` is web-only.** If a mobile ticket blames the delay, it's a red herring.
- **Partial responses and auto-submit are web-only too.** On mobile a survey always stores one response at the end, and a rating tap never self-submits. Don't carry a web diagnosis onto a mobile ticket.
- **URL targeting is effectively web-only.** Mobile SDKs decode the field but never enforce it; React Native filters those surveys out entirely. A mobile survey with a URL condition behaves as "no URL condition" (mobile/flutter) or "never shows" (RN).
- **Android ships no survey UI.** The app (or the Flutter plugin) must provide a `PostHogSurveysDelegate`. "Survey never renders on Android" is often a missing delegate, not a PostHog bug.
- **Flutter is hybrid:** triggering/eligibility runs in the native iOS/Android layer; rendering is Dart (`SurveyService.showSurvey` → `showModalBottomSheet`). It does _not_ "just call native" for UI. So a Flutter rendering bug lives in Dart; a Flutter eligibility bug lives in native.
- **React Native enforces the wait period from 4.81.5.** Older versions skip the check, so on those versions don't blame the wait period.

For a deeper version-by-version capability audit, see the `survey-sdk-audit` skill if available.

## How a survey actually gets shown (the web eligibility pipeline)

The web SDK is the most complex and the most common in tickets. Mental model from
`packages/browser/src/extensions/surveys.tsx` (`checkSurveyEligibility`) — checks run in
order, first failure wins:

1. `isSurveyRunning` — has `start_date`, no `end_date`.
2. survey `type` is in-app (Popover / Widget / API).
3. `linked_flag_key` enabled (if set).
4. `targeting_flag_key` enabled (if set) — customer-defined property targeting.
5. `_internalFlagCheckSatisfied` — the auto-generated internal targeting flag.
6. `hasWaitPeriodPassed` — `seenSurveyWaitPeriodInDays` vs `localStorage.lastSeenSurveyDate`.
7. `getSurveySeen` — per-survey seen flag.

Then in `getActiveMatchingSurveys`: URL/device/selector match, event/action trigger fired, and flag re-check.

Two non-obvious facts that drive real tickets:

- **The server returns ALL non-archived surveys** (`SurveyViewSet`, `products/surveys/backend/api/survey.py`). It does **not** pre-filter by the internal targeting flag. All eligibility is client-side. So you cannot conclude "the backend excluded them" — the SDK did.
- **The wait period has TWO independent implementations.** `canActivateRepeatedly` (true when `schedule: 'always'`) short-circuits `_internalFlagCheckSatisfied` (step 5) — so `always` bypasses the internal flag, including its `$last_seen_survey_date` rule. But `hasWaitPeriodPassed` (step 6) reads `localStorage.lastSeenSurveyDate` directly and is **NOT** bypassed by `canActivateRepeatedly`. So a `schedule: 'always'` survey with `seenSurveyWaitPeriodInDays: 30` still enforces the 30-day wait via the localStorage path. `lastSeenSurveyDate` is updated whenever _any_ survey is shown, regardless of completion. The same short-circuit also drops the internal flag's `$survey_responded/<id> is_not_set` rule, so **`schedule: 'always'` lets one person respond repeatedly** — check `uniq(distinct_id)` against `count()` on `survey sent` before reading a response count as a respondent count.

## How a response actually gets stored

Showing a survey and storing a response are separate pipelines. "Wrong data" tickets are about the
second one, and its UI copy is genuinely misleading — reason from the code, not from the label.

### "Response collection" maps to `enable_partial_responses`

The radio in `frontend/src/scenes/surveys/SurveyResponsesCollection.tsx`. "Any question: when at
least one question is answered…" is `true`; "Complete survey: the response is stored when all
questions are answered" is `false`.

- `true` -> `sendSurveyEvent` fires on **every** `onNextButtonClick`, so one `survey sent` per
  question answered, all sharing a `$survey_submission_id` with `$survey_completed` running
  `false ... true`.
- `false` -> one `survey sent` event, at the end of the path.

The setting controls capture, not response visibility. The results include every `survey sent`
and every `survey dismissed` or `survey abandoned` marked `$survey_partially_completed = true`.
`buildMergedSubmissionsSubquery` in `frontend/src/scenes/surveys/utils.ts` merges the latest
answer per question for each submission ID; events without an ID stay separate. **Raw event
counts can exceed response counts.** A completed event takes precedence over earlier partial
closures. Otherwise the outcome chip reads "Dismissed" when the latest response event is a
dismissal, and "Abandoned" for other unfinished submissions. "Abandoned" does not prove a
page-unload event or inactivity timeout occurred; a later completion updates the same row.

**The defaults disagree by creation path:** `NEW_SURVEY` sets `true`
(`frontend/src/scenes/surveys/constants.tsx`) but the Django model default is `False`
(`products/surveys/backend/models.py`). Read it, don't infer it from the creation date.

### "All questions answered" does not mean what it says

`isSurveyCompleted` is `getNextSurveyStep(...) === End` — **the respondent reached the end of their
own branching path.** Nothing checks that every question holds a value, and two things then blank
out cells on a complete response: questions **skipped by branching have no key at all**
(`onNextButtonClick` prunes to `visitedIndices` before capture, so absent rather than `""` or
`null`), and **`optional: true` questions clicked past are stored as `null`**
(`submitDisabled` is `isNull(rating) && !question.optional`).

So on a branching survey with optional tail questions, "only the first question was answered" is
the **expected shape of a complete response**. Blank ≠ unanswered. Check `branching` and `optional`
on every question before believing a bug.

### Branching is in the API — read it, don't ask for it

It's `questions[].branching`, returned verbatim by both the management API and the SDK payload, in
four shapes: `next_question`, `end` (the confirmation message), `specific_question` + `index`, and
`response_based` + `responseValues`. Two traps: for **rating** questions `responseValues` is keyed
by _bucket_ (`negative`/`neutral`/`positive`, or NPS `detractors`/`passives`/`promoters`), not by
value; and a bucket **missing** from `responseValues` silently falls through to
`currentQuestionIndex + 1`. So `{negative: 3, neutral: 3}` on a 5-scale is a complete, working
config that reads like an omission. Bucket boundaries per scale are in
[references/reading-responses.md](references/reading-responses.md) — don't guess them, scale 2 is
inverted.

From PostHog/skills, MIT.