pre-pr-validation

Files (1)

• SKILL.md

SKILL.md (first lines)

# Pre-PR Validation

Every pull request has one automatic candidate workflow. PRs
run the complete six-shard Python 3.12 coverage suite, Cypress, focused
Python 3.11/3.14 compatibility checks, lint, docs drift, guardrails, security
regressions, secret scanning, and confidentiality scanning. Cross-subsystem
coverage must pass before queue admission. Full shards include smoke tests;
the browser job runs the Python 3.12 CLI install check before compatibility
tests, without duplicate subsystem jobs. Trusted test selection shares the
secret-scanning runner so it adds no separate scheduling dependency. Only
narrowly recognized prose edits skip runtime suites; those retain smoke and every documentation,
repository, and security gate. See `CONTRIBUTING.md` for the trusted-base selector,
which uses the base's merge-candidate policy for both events during rollout.
The queue verifies successful, less-than-24-hour PR evidence for the identical
combined Git tree using trusted-base `ci_queue_evidence.py`, then reruns fresh
secret, confidentiality and source/dependency scans. Changed combined trees rerun
all tests, lint, guardrails and hostile-input checks; image checks rerun when the
trusted base policy identifies changed image inputs. Missing, failed or stale
evidence restores the full queue gate, which must pass before merging. Older
PRs can adopt the policy without a forced branch refresh.
`pr-precheck` gives a five-minute early signal without replacing full admission.
The queue execution target is ten minutes; hosted-runner waits can add delay.
A daily audit covers all supported Python versions instead of starting a full
audit after every merge. Lint, docs drift, and guardrails remain candidate gates
and can be dispatched manually, without duplicate main-push runs competing with
the next queue entry. The daily full suite includes guardrails; post-merge
security audits remain automatic. Independent validation jobs use available
GitHub runner
capacity without job-level concurrency locks or matrix `max-parallel` caps.
Preserve per-PR supersession on the parent and distinct workflow group prefixes
for reusable children. Merge candidates use their own SHA-specific groups.
The aggregate gate uses `!cancelled()` to report failed dependencies while
allowing obsolete runs to terminate; `always()` can keep their final job queued
ahead of the replacement candidate.
`test_ci_concurrency` rejects shared validation locks and matrix caps;
`test_ci_workflows` guards cancellation and required results. See the contributor
concurrency guide for organization runner limits and rollout behavior. Refresh
older PR branches after a scheduling change lands; rerunning an old commit
retains its original workflow configuration.
`Security regression / security-regression` requires
every PR component and image-security validation, or verified identical-tree
evidence plus fresh security scans for queue candidates. Changed combined trees
require fresh test results and any affected image checks.
The required workflows have no top-level path filters. Image scope is classified
inside always-reporting jobs: image, packaging, workflow, or security-policy
changes run the deep checks; unrelated source changes take the fast path. Main,
scheduled, and manual image audits always run deeply. Verify actual required
contexts in branch protection before claiming merge enforcement.

All of them are reproducible locally. Run them in cost order so the cheap ones
catch the common mistakes before you spend minutes on the full suite.

For how the test suite itself behaves — markers, hermetic fixtures, parallel
runs, live-infra expectations — use `skills/atomic/testing-conventions/SKILL.md`.
This skill is the gate map.

## Use The Repo Virtualenv

`npa/.venv/bin/python` (Python 3.12). Never bare `python`. The `make` targets
select that virtualenv automatically and use its absolute path before changing
into `npa/`. Override `PYTHON` only for a different environment:

```bash
make test PYTHON=/workspace/npa/.venv/bin/python
```

If that venv is shared across checkouts (worktree, agent sandbox) rather than
installed fresh in this one, `make test`/`test-smoke`/`test-guardrails` fail
fast via `make check-env` before running anything, rather than silently
testing a different checkout's code — see `testing-conventions` for why.

## The Ladder

Start with `make precheck` for dependency fingerprints, lint, formatting, and
focused CI contracts. It reads the working tree and fails before expensive tests.
Before pushing committed work, run `git fetch origin main` and
`make merge-precheck` to catch combined-tree conflicts and stale CI fingerprints
without altering your checkout or index. This second command checks committed
HEAD only; neither command replaces the full Linux or security gates.

```bash
# 1. Lint — seconds. This matches CI and `make lint` across all of npa/.
npa/.venv/bin/python -m ruff check npa

# 2. Onboarding smoke — ~20s.
make test-smoke PYTHON=/workspace/npa/.venv/bin/python

# 3. Guardrails — ~50s, ~2000 static contract assertions. Highest signal per second.
npa/.venv/bin/python -m pytest npa/tests/guardrails -q

# 4. The tests for what you touched — seconds to a minute.
npa/.venv/bin/python -m pytest npa/tests/cli/test_<tool>_cli.py -q

# 5. Full unit suite — minutes. This is the PR gate.
make test PYTHON=/workspace/npa/.venv/bin/python

# 6. Docs drift — ~1-2 min. Only when CLI commands or options changed.
bash scripts/build_docs.sh --check

# 7. Confidentiality, diff-scoped — seconds.
npa/.venv/bin/python -m npa.guardrails.confidentiality \
  --repo-root . --diff-range origin/main..HEAD --built-in-nebius-infra
```

Steps 1 through 3 catch most failures and cost about a minute together. Run them
after every meaningful edit; save 5 and 6 for before you push.

## Which Gates Apply To Your Change

| You changed | Also run |
|---|---|
| A CLI command or any of its options | `bash scripts/build_docs.sh --check`, plus the argv guardrails — a renamed option breaks every `toolRef` that passes it |
| A `toolRef` or the catalog | `test_tool_catalog_argv`, `test_module_toolref_argv`, `test_shown_workflow_catalog`, and the catalog doc sync test |
| An `npa.workflow` spec | `npa workbench workflow validate-spec <path>`, plus live-matrix registration per the testing conventions |
| A workbench tool's surface | `test_three_tier_contract`, and the tool's CLI and workbench tests |
| A Dockerfile or image tag | `npa/.venv/bin/python npa/docker/workbench/check_tag_consistency.py`, plus `npa/tests/docker/` |
| The complete-byte scanner, its dependencies, or publication wiring | `npa/tests/docker/test_image_byte*.py`, plus the actual native integration sequence in `.github/workflows/image-security-scan.yml`; hermetic protocol tests do not replace that native gate |
| A `SKILL.md` or `skills/index.yaml` | `test_skills_index` and `test_develop_skills` |
| Security gate code, scanner pins, or its workflow | Follow `docs/security/merge-security-gate.md`: run the actual scanner regression workload and full base/candidate comparison, plus `test_security_source` and `test_security_gate` |

From nebius/nebius-physical-ai, Apache-2.0.

More from nebius/nebius-physical-ai