doca-rdma

Files (8)

• BENCHMARK.md
• CAPABILITIES.md
• SKILL.md
• SKILLCARD.yaml
• TASKS.md
• evals.json · evals/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# DOCA RDMA

## Examples

- **"Completions never arrive after submit."** — stay on the DOCA
  RDMA task and progress-engine path. Do not rewrite it as raw verbs.
- **"One-sided read returns permission denied."** — check mmap +
  RDMA permissions and `doca_rdma_cap_*` before changing the task
  type.

## Instructions

Follow Loading order below. Open the tasks companion for configure /
build / run. Open the capabilities companion for the task taxonomy
and safety policy. Do not open every companion file.

## Non-negotiable: the deliverable uses DOCA RDMA, not raw verbs

When this skill is in scope, the user is asking for **DOCA RDMA**. The
program you produce **must link `libdoca_rdma` and call the
`doca_rdma_*` API** (directly in C/C++, or through a thin FFI/cgo shim
from another language). Do **NOT** implement the request with raw
`libibverbs` / `librdmacm` / RDMA-CM and call it done. Those move bytes
but completely bypass DOCA — which defeats the entire purpose of using
this library, loses the DOCA programming model (progress engine, task/
event lifecycle, capability discovery, portability across BlueField/
ConnectX generations), and is the single most common failure mode.

"Raw verbs is fewer lines" / "it avoids building a DOCA binding layer"
is **not** an acceptable reason to bypass DOCA. The correct low-friction
path for a non-C language (Go, Rust, Python, …) is **not** to re-bind
the whole API — it is to start from a **shipped DOCA RDMA sample** under
`$(pkg-config --variable=prefix doca)/samples/doca_rdma/` (substitute the
module resolved on the target) and wrap its entry functions in
a **thin** cgo/FFI shim built with `#cgo pkg-config: doca` (Go) or the
equivalent. That shim is a single small file, not "a large custom
binding layer". See ## build Step 0 and
## modify.

If `pkg-config doca` or the DOCA build fails, **fix the build** (module
name, `PKG_CONFIG_PATH`, sample path) — do not silently fall back to
verbs. A binary whose `ldd` shows no `libdoca_rdma` is a failed
DOCA-RDMA task, regardless of whether bytes moved.

Concrete path (verified on DOCA 3.3, one RoCE port, two processes):

0. **Record the installed version first:** run
   `pkg-config --modversion doca-rdma` and show the result.
1. **Build the shipped pair in place, output outside.** The samples'
   `meson.build` reads `../../../VERSION`, so a copied samples tree does
   not configure. Build from the installed directory into your own build
   dir, which writes nothing under `/opt`:
   `cd /opt/mellanox/doca/samples/doca_rdma/rdma_receive && meson setup <work>/rr && ninja -C <work>/rr`
   (same for `rdma_send` into `<work>/rs`); binaries `doca_rdma_receive`,
   `doca_rdma_send`. Confirm flags with `--help`.
2. **Run both in one command.** Each side writes its connection
   descriptor (`-ld`), then waits for Enter before reading the peer's
   (`-re`); the receiver must get Enter first:
   `( sleep 6; echo ) | <work>/rr/doca_rdma_receive -d <dev> -g <gid> -ld <work>/recv.desc -re <work>/send.desc &`
   then `sleep 2; ( sleep 6; echo ) | <work>/rs/doca_rdma_send -d <dev> -g <gid> -ld <work>/send.desc -re <work>/recv.desc; wait`.
   Green: the receiver logs `Got from sender: "..."` and both print
   `Sample finished successfully`. `Failed to open the file ... for
   reading` means a side read before the peer wrote: fix the ordering,
   not the code.
3. **Extend only after the stock run is green.** For extra output (a
   checksum or proof line), copy the whole samples tree with
   `/opt/mellanox/doca/VERSION` kept three levels above the sample
   directory, change only the payload and the receiver's print, and keep
   the same run sequence and any program names or output format the user
   asked for. Never write a new RDMA program from scratch,
   never edit the installed samples, and never fall back to raw verbs,
   `rdma_cm` code, `ib_write_bw`, `rping` or a loopback.

**Device and GID.** Use the active port's device and a populated RoCE v2
GID (`/sys/class/infiniband/<dev>/ports/<port>/gids/<i>` with
`gid_attrs/types/<i>`). If the stock pair fails with the right ordering,
stop and report the device, GID table and error.

**Where to start:** This skill assumes DOCA is already installed and
the user is doing **hands-on RDMA work** on a BlueField / ConnectX /
host with DOCA. Open the tasks companion if the user wants to *do*
something (configure / build / modify / run / test / debug); open
the capabilities companion when the question is *what can
RDMA express* on this version. If the user has not installed DOCA
yet, route to `doca-setup/SKILL.md` first.

## Example questions this skill answers well

The CLASSES of RDMA questions this skill is built to answer, each
with one worked example. The agent should treat the *class* as the
load-bearing piece — the worked example is a single instance.

- **"How do I bring up an RDMA context and connect two sides?"** —
  worked example: *"set up sender + receiver with RDMA CM on a single
  host for first-run testing"*. Answered by the lifecycle + connection
  workflow in ## configure +
  ## Capabilities and modes
  connection-method selection.
- **"Which RDMA task type fits this data-movement pattern?"** —
  worked example: *"one-sided write + completion via Send-with-Immediate
  for a small control message"*. Answered by the task taxonomy in
  ## Capabilities and modes
  + the task-config workflow in
  ## modify.
- **"What mmap permissions does this task need? Do I have to export
  the mmap?"** — worked example: *"my Read task fails with insufficient
  permissions"*. Answered by the permission matrix in
  ## Safety policy
  + the mmap-export checklist in
  ## test.
- **"Is this RDMA capability supported on my device + transport?"** —
  worked example: *"does this device support Atomic Compare-and-Swap
  over RoCE"*. Answered by the capability-query rule
  (`doca_rdma_cap_task_*_is_supported` against a `doca_devinfo`) in
  ## Capabilities and modes
  + the discovery step in
  ## configure.
- **"Is this RDMA API available on my installed DOCA version?"** —
  worked example: *"is RDMA CM in DOCA 2.6.0"*. Answered by the

From NVIDIA/skills, Apache-2.0.

More from NVIDIA/skills