kubernetes

Files (1)

• SKILL.md

SKILL.md (first lines)

# Kubernetes Troubleshooting & Operations

> Reusable operational knowledge for diagnosing and remediating Kubernetes cluster and workload failures.

tags: [kubernetes, k8s, pods, deployments, crashloopbackoff, imagepullbackoff, oomkilled]

## Safety Rules
* Always inspect pods, events, and logs BEFORE attempting any pod restart or deployment mutation.
* Never delete namespaces, PersistentVolumes, or secrets without explicit human confirmation.
* Verify rollout status after updating or restarting any workload.
* Check node resource pressure (CPU/Memory/DiskPressure) if multiple pods are pending or evicted.

## Diagnostic Runbook

### Step 1: Workload Observation
1. List pods in namespace: `kubectl get pods -n <namespace> -o wide`
2. Inspect pod conditions: `kubectl describe pod <pod-name> -n <namespace>`
3. Check cluster events: `kubectl get events -n <namespace> --sort-by=.metadata.creationTimestamp`
4. Retrieve container logs: `kubectl logs <pod-name> -n <namespace> --tail=100`

### Step 2: Common Failure Signatures & Diagnosis
* **ImagePullBackOff / ErrImagePull**:
  - Check image repository and tag spelling.
  - Check node internet access or `imagePullSecrets` configuration.
* **CrashLoopBackOff**:
  - Read container exit code and termination message via `describe pod`.
  - Read previous container logs: `kubectl logs <pod-name> -n <namespace> --previous`.
  - Common reasons: missing environment variables, DB connection timeout, bad start command.
* **OOMKilled (Exit Code 137)**:
  - Pod exceeded its memory limit.
  - Review `resources.limits.memory` and consider increasing limit or fixing memory leak.
* **Pending**:
  - Insufficient CPU/Memory on nodes or node selector / toleration mismatch.

### Step 3: Remediation & Verification
* Propose remediation to engineer.
* After approval, apply fix or rollout restart: `kubectl rollout restart deployment/<name> -n <namespace>`.
* Verify rollout completion: `kubectl rollout status deployment/<name> -n <namespace>`.

From NotHarshhaa/argus-agent, MIT.

More from NotHarshhaa/argus-agent