get-available-resources

Files (9)

• SKILL.md
• resource_semantics.md · references/
• snapshot_schema.md · references/
• sources.md · references/
• _common.py · scripts/
• accelerator_diagnostics.py · scripts/
• detect_resources.py · scripts/
• plan_workload.py · scripts/
• snapshot_tools.py · scripts/

SKILL.md (first lines)

# Get Available Resources

Build a conservative picture of resources available to the **current process**.
Keep host inventory, process affinity, cgroup/container limits, scheduler
allocation, and accelerator runtime usability separate.

## Safety contract

Follow these rules:

- Run detection when the user requests it or a specific workload needs resource
  planning. Do not persist a fingerprint for every scientific task.
- Use stdout by default. Persist only when the user chooses an explicit generic
  local filename.
- Do not run stress tests, benchmarks, large allocations, write probes, device
  resets, driver installation, or clock/power changes.
- Do not dump the environment. Read only the named Slurm and accelerator
  variables implemented by the detector.
- Do not report hostnames, absolute paths, cgroup paths, job IDs, device UUIDs,
  PCI addresses, or raw visibility-variable values.
- Treat a missing observation as unknown. Never convert unknown to unlimited.
- Never infer that a visible host CPU, memory pool, or GPU is usable inside a
  scheduler allocation or container.

The bundled detector uses only fixed executable/argument tuples, no shell,
short timeouts, bounded stdout/stderr, and partial-failure warnings.

## Quick start

Run from this skill directory.

### Ephemeral stdout snapshot

```bash
python scripts/detect_resources.py
```

The command emits only JSON to stdout. Redirect it only when ordinary shell
permissions are acceptable.

### Explicit private file

```bash
python scripts/detect_resources.py --output resource-snapshot.json
```

Explicit output is restricted to one `.json` filename in the current
directory, uses private permissions, rejects symlinks and path traversal, and
refuses overwrite unless `--force` is supplied.

### Optional psutil enhancement

The standard-library detector works without installation. For broader
cross-platform physical-core, affinity, available-memory, swap, and disk
coverage:

```bash
uv pip install "psutil==7.2.2"
```

The import is lazy. Failure to import psutil becomes a warning, not a fatal
error.

### Skip management-tool probes

```bash
python scripts/detect_resources.py --skip-accelerators
```

Use this when accelerator discovery latency is undesirable. The detector still
summarizes the presence and state of allowlisted visibility variables without
returning their values.

## Required interpretation

### CPU

Read these as different facts:

- `cpu.host.logical`: system-visible scheduling units.
- `cpu.host.physical`: physical topology, or null; never inferred from logical
  count.
- `cpu.process.affinity_logical`: current affinity-set size when supported.
- `cpu.cgroup_v2.cpuset_logical`: effective cgroup cpuset size.
- `cpu.cgroup_v2.quota_cores`: finite `cpu.max` capacity, possibly fractional.
- `scheduler.allocation.cpu_per_process`: bounded Slurm per-task
  interpretation when scope is clear.
- `cpu.effective.capacity_cores`: minimum positive observed constraint.
- `cpu.effective.worker_ceiling`: conservative floor for CPU process workers.

A quota of 1.5 is CPU-time capacity, not 1.5 physical cores. Affinity and
cpusets constrain placement; quota constrains bandwidth.

### Memory

Keep these separate:

- host total/available memory;
- current cgroup usage, hard `memory.max`, and remaining hierarchical capacity;
- `memory.high`, which is a pressure/throttle boundary rather than a hard cap;
- scheduler memory allocation and its scope; and
- conservative effective hard limit and available estimate.

On Apple silicon, `memory.model` is `unified_cpu_gpu`. Do not add integrated GPU
memory to RAM or describe it as separate VRAM.

### Accelerators

Each device is a backend **candidate**:

- NVIDIA GPU → CUDA candidate;
- AMD GPU → ROCm candidate;
- Apple integrated GPU → Metal candidate.

Management-query visibility does not establish:

1. scheduler/container permission;
2. device-node access;
3. driver/runtime compatibility;

From Zejie0920/mather-workflow, MIT.

More from Zejie0920/mather-workflow