optimize-for-gpu

Files (16)

• SKILL.md
• code_transformation_patterns.md · references/
• cucim.md · references/
• cudf.md · references/
• cugraph.md · references/
• cuml.md · references/
• cupy.md · references/
• cuspatial.md · references/
• cuvs.md · references/
• cuxfilter.md · references/
• decision_framework.md · references/
• installation.md · references/
• kvikio.md · references/
• numba.md · references/
• raft.md · references/
• warp.md · references/

SKILL.md (first lines)

# GPU Optimization for Python with NVIDIA

Treat GPU acceleration as an evidence-driven optimization, not an automatic rewrite. Preserve the
user's numerical and algorithmic contract, measure with representative data, and keep the GPU
version only when synchronized end-to-end benchmarks show a useful improvement.

Reviewed against RAPIDS 26.08, CuPy 14.2, Numba-CUDA 0.30.4, and Warp 1.17.
The references contain illustrative GPU examples: source/API review is not execution on CUDA
hardware. Validate them on the user's target GPU before reporting correctness or speedup.
Do not use a moving `latest` documentation page to infer compatibility with a pinned release.

## When This Skill Applies

- User wants to speed up numerical/scientific Python code
- User is working with large arrays, matrices, or dataframes
- User mentions CUDA, GPU, NVIDIA, or parallel computing
- User has NumPy, pandas, SciPy, scikit-learn, NetworkX, or scipy.sparse.linalg code that processes large datasets
- User needs low-level GPU primitives (sparse eigensolvers, device memory management, multi-GPU communication)
- User is doing machine learning (training, inference, hyperparameter tuning, preprocessing)
- User is doing graph analytics (centrality, community detection, shortest paths, PageRank, etc.)
- User is doing vector search, nearest neighbor search, similarity search, or building a RAG pipeline
- User has Faiss, Annoy, ScaNN, or sklearn NearestNeighbors code that could be GPU-accelerated
- User wants GPU-accelerated interactive dashboards, cross-filtering, or exploratory data analysis on large datasets
- User is doing geospatial analysis (point-in-polygon, spatial joins, trajectory analysis, distance calculations) with GeoPandas or shapely
- User is doing image processing, computer vision, or medical imaging (filtering, segmentation, morphology, feature detection) with scikit-image or OpenCV
- User is working with whole-slide images (WSI), digital pathology, microscopy, or remote sensing imagery
- User is loading large binary data files into GPU memory (numpy.fromfile → cupy, or Python open() → GPU array)
- User needs to read files from S3, HTTP, or WebHDFS directly into GPU memory
- User mentions GPUDirect Storage (GDS) or wants to bypass CPU-memory staging for file IO
- User is doing physics simulation (particles, cloth, fluids, rigid bodies) or differentiable simulation
- User needs mesh operations (ray casting, closest-point queries, signed distance fields) or geometry processing on GPU
- User is doing robotics (kinematics, dynamics, control) with transforms and quaternions
- User has Python simulation loops that could be JIT-compiled to GPU kernels
- User mentions NVIDIA Warp or wants differentiable GPU simulation integrated with PyTorch/JAX
- User is doing simulations, signal processing, financial modeling, bioinformatics, physics, or any compute-intensive work
- User wants to optimize existing code and GPU acceleration is the right answer

## Choose the Smallest Suitable Layer

Prefer a maintained library implementation over a custom kernel:

| Existing workload | Preferred path | Use for |
| --- | --- | --- |
| NumPy / SciPy | **CuPy** | arrays, sparse matrices, linear algebra, FFTs, signal processing |
| pandas | **cudf.pandas**, then **cuDF** | accelerator mode first; native API for more control |
| scikit-learn | **cuml.accel**, then **cuML** | accelerator mode first; native estimators as needed |
| NetworkX | **nx-cugraph**, then **cuGraph** | backend dispatch first; native graph API at scale |
| scikit-image | **cuCIM** | GPU image processing and whole-slide imaging |
| Faiss / Annoy / k-NN | **cuVS** | exact and approximate vector search |
| Raw or remote file I/O | **KvikIO** | GPU buffers and GPUDirect Storage |
| Custom array kernels | **Numba-CUDA-MLIR** for new work; **Numba-CUDA** for existing code | explicit SIMT kernels and shared memory |
| Spatial or differentiable kernels | **Warp** | geometry, simulation kernels, robotics, autodiff |
| High-level physics simulation | **Newton** | maintained engine that succeeds the removed `warp.sim` module |
| Low-level RAPIDS primitives | **RAFT** (`pylibraft`) | sparse eigensolvers, resources, multi-GPU building blocks |

Do not move code out of PyTorch, JAX, TensorFlow, or another GPU-native framework merely to use
one of these libraries. First remove CPU round trips and use the framework's compiler, profiler,
mixed-precision, and batching facilities.

Treat these as legacy-only:

| Project | Status | Guidance |
| --- | --- | --- |
| **cuxfilter** | Final release 26.06 | Maintain existing dashboards only. For new work, combine cuDF with HoloViews/hvPlot/Datashader and serve with Panel, Dash, Streamlit, or Bokeh. |
| **cuSpatial** | Archived at 25.04 | Use only in an isolated legacy environment. For new work, keep geometry in GeoPandas/Shapely and accelerate compatible tabular stages with cuDF. |

Full per-library guidance, including when each is the *wrong* choice and how to combine
them, is in [references/decision_framework.md](references/decision_framework.md).
Install commands and CUDA version selection are in
[references/installation.md](references/installation.md). Before/after conversions for
every library are in
[references/code_transformation_patterns.md](references/code_transformation_patterns.md).

## Optimization Workflow

### 1. Define the contract and baseline

- Capture a representative input, expected output, and acceptable numerical tolerance.
- Measure the current end-to-end path, including input, transfers, compute, and output.
- Profile before changing code. Use CPU profilers for CPU code and identify whether the real limit
  is compute, memory bandwidth, allocation, transfer, synchronization, or storage.
- Record hardware, package versions, dtypes, shapes, batch size, and warm-up policy with results.

### 2. Check suitability before porting

GPU execution is promising when the hot path exposes substantial independent work, runs often
enough to amortize initialization and transfer, and has a working set that fits available device
memory with room for temporaries. Keep a CPU path when the workload is small, mostly sequential,
dominated by unsupported operations, or requires frequent host-device round trips.

Do not use fixed row-count thresholds as proof. Benchmark the user's actual shapes and hardware.
For out-of-core data, estimate peak working memory and choose chunking, Dask, or a streaming design
before allocating.

### 3. Try the least disruptive implementation

1. If the code already uses a GPU-native framework, optimize within that framework.
2. Try accelerator or backend modes (`cudf.pandas`, `cuml.accel`, `nx-cugraph`).
3. Move to a native GPU API only where accelerator coverage or performance is insufficient.
4. Write a custom kernel only when profiling shows an operation without a suitable library
   implementation.

Read the relevant library reference before writing code; compatible names can still differ in
defaults, dtypes, output types, and supported arguments.

### 4. Keep a coherent GPU data path

- Transfer inputs once and keep intermediates device-resident.
- Reuse allocations and prefer `out=` or in-place forms when semantics allow.
- Batch small operations; fuse elementwise work when it removes intermediate arrays.
- Use pinned host memory and non-default streams only after profiling shows transfer overlap matters.
- Choose `float32`, mixed precision, or reduced-precision storage only when the contract permits it.

### 5. Validate semantics before speed

- Compare CPU and GPU outputs on small deterministic fixtures and representative data.
- Use explicit tolerances for floating-point results and test edge cases, NaNs, ordering, and dtypes.
- For approximate nearest-neighbor indexes, report recall@k against exact search; do not compare an
  exact CPU algorithm with an approximate GPU algorithm as if they were equivalent.

From K-Dense-AI/scientific-agent-skills, MIT.

More from K-Dense-AI/scientific-agent-skills