pvc-debug

Files (1)

• SKILL.md

SKILL.md (first lines)

# PersistentVolumeClaim Failure Diagnosis

When a PVC is stuck in `Pending`, a pod cannot mount its volume, or storage provisioning fails, follow this flow to identify the root cause.

**Scope:** This skill is for **diagnosis only**. Once you identify the root cause, report it to the user and stop. Do NOT attempt to modify PVCs, PVs, or StorageClasses — that should be left to the user or cluster administrator.

## Diagnostic Flow

### 1. Check PVC status

```bash
kubectl get pvc <pvc-name> -n <ns>
```

Note the **STATUS**, **VOLUME**, **CAPACITY**, and **STORAGECLASS** columns.

- `Bound` — PVC is healthy, issue may be elsewhere (pod mount, permissions)
- `Pending` — PVC is waiting for a PV to be provisioned or bound
- `Lost` — The PV the PVC was bound to has been deleted

### 2. Describe the PVC

```bash
kubectl describe pvc <pvc-name> -n <ns>
```

Focus on:
- **Events** — provisioner errors, binding failures, or quota exceeded messages
- **Access Modes** — requested access mode (ReadWriteOnce, ReadWriteMany, ReadOnlyMany)
- **StorageClass** — which StorageClass is requested
- **Volume Name** — if bound, which PV it is bound to

### 3. Check the StorageClass

```bash
kubectl get storageclass
```

Verify the StorageClass referenced by the PVC exists and note:
- **PROVISIONER** — the volume provisioner responsible for creating PVs
- **RECLAIMPOLICY** — Delete or Retain
- **VOLUMEBINDINGMODE** — Immediate or WaitForFirstConsumer

If the StorageClass uses `WaitForFirstConsumer`, the PV will not be provisioned until a pod using the PVC is scheduled to a node.

### 4. Check available PVs (for static provisioning)

If the cluster uses pre-created PVs instead of dynamic provisioning:

```bash
kubectl get pv
```

For a PVC to bind to a PV, the PV must:
- Be in `Available` status
- Match the PVC's requested `storageClassName`
- Meet the PVC's capacity request (PV capacity >= PVC request)
- Support the PVC's access mode
- Match any `selector` labels on the PVC

Check a specific PV:

```bash
kubectl describe pv <pv-name>
```

### 5. Match patterns and conclude

---

#### `storageclass.storage.k8s.io "<name>" not found` — StorageClass does not exist

The PVC references a StorageClass that does not exist in the cluster.

```bash
kubectl get storageclass
```

Advise the user to either create the StorageClass or update the PVC to use an existing one. If no StorageClass is specified in the PVC, it uses the cluster's default StorageClass — check if one is set:

```bash
kubectl get storageclass -o jsonpath='{range .items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=="true")]}{.metadata.name}{"\n"}{end}'
```

---

#### `waiting for first consumer to be created` — WaitForFirstConsumer binding mode

The StorageClass has `volumeBindingMode: WaitForFirstConsumer`. The PV will not be provisioned until a pod that uses this PVC is scheduled.

This is normal behavior. Check if a pod using the PVC exists and is being scheduled. If the pod is `Pending`, use the `pod-pending-debug` skill.

---

#### `no persistent volumes available for this claim` — No matching PV

For static provisioning: no available PV matches the PVC's requirements.

Check the mismatch:

```bash
kubectl get pvc <pvc-name> -n <ns> -o jsonpath='storageClass={.spec.storageClassName} capacity={.spec.resources.requests.storage} accessModes={.spec.accessModes[*]}'
```

```bash
kubectl get pv -o custom-columns='NAME:.metadata.name,CAPACITY:.spec.capacity.storage,ACCESS:.spec.accessModes[*],CLASS:.spec.storageClassName,STATUS:.status.phase'
```

Common mismatches:
- **Capacity** — PV is smaller than PVC request
- **Access mode** — PV does not support the requested access mode (e.g., PV is ReadWriteOnce but PVC requests ReadWriteMany)
- **StorageClass** — PV and PVC reference different StorageClasses

---

#### Provisioner error in events — Dynamic provisioning failure

The provisioner failed to create a volume. Common provisioner errors:

From scitix/siclaw, Apache-2.0.

More from scitix/siclaw