async-jobs

Files (1)

• SKILL.md

SKILL.md (first lines)

## Detection Signals

- `queue`, `worker`, `job`, or `consumer` directories or files in the project
- Dependencies: `laravel/horizon`, `sidekiq`, `celery`, `bullmq`, `bee-queue`, `faktory`, `resque`, `delayed_job`
- Cloud queue services: `aws-sdk` with SQS usage, `@google-cloud/pubsub`, Azure Service Bus, RabbitMQ, Kafka
- `QUEUE_*` / `REDIS_QUEUE_*` / `SQS_*` env vars
- Cron/scheduler configs: `app/Console/Kernel.php`, `celerybeat`, `node-cron`, `whenever` gem

---

## When to Use a Job vs. Synchronous Processing

Use a background job when:
- The operation takes longer than the acceptable HTTP response time (~200ms threshold for user-facing actions)
- The operation involves external services that can be slow or unavailable (email, SMS, payment processor, webhooks)
- The operation can be retried independently if it fails (idempotent by nature or can be made so)
- Fan-out: one event triggers multiple independent downstream actions

Keep it synchronous when:
- The user needs the result to proceed (e.g., payment status before showing confirmation)
- The operation is fast and its failure should block the response (e.g., saving the primary record)
- Ordering guarantee is critical and queues add non-determinism

---

## Idempotency — Most Critical Rule

**Every job must be safe to run more than once with the same payload.**

Queues guarantee at-least-once delivery — a job will be retried on failure and may execute multiple times. A non-idempotent job causes duplicate records, double charges, or double notifications.

```python
# ❌ Non-idempotent — creates a duplicate on retry
def send_welcome_email(user_id: str):
    user = User.find(user_id)
    Email.send(to=user.email, template="welcome")

# ✅ Idempotent — checks state before acting
def send_welcome_email(user_id: str):
    user = User.find(user_id)
    if user.welcome_email_sent_at is not None:
        return  # already sent — safe to skip
    Email.send(to=user.email, template="welcome")
    user.update(welcome_email_sent_at=now())
```

**Idempotency patterns:**
- **State check**: read current state before acting; bail out if the action was already completed
- **Unique constraint**: use a database unique key on `(job_type, entity_id)` to prevent duplicate execution
- **Idempotency key**: pass a UUID with the job payload; store it as processed after first execution

---

## Payload Validation

Treat job payloads with the same rigor as HTTP input — validate before processing:

```ts
// ✅ Validate payload shape before using it
async handle(payload: unknown) {
  const { orderId, userId } = validateOrderPayload(payload); // throws on invalid
  const order = await Order.findOrFail(orderId);
  // ...
}
```

**Rules:**
- Never assume payload fields are present and correctly typed — they may come from an old version of the job or be corrupted
- Validate required fields and types at the start of the handler, before any side effects
- If validation fails: fail permanently (do not retry) — retrying a malformed payload wastes queue capacity

---

## Retry Strategy

| Failure type | Action |
|---|---|
| Transient (network timeout, 503) | Retry with exponential backoff |
| Business rule failure (record not found yet) | Retry with delay |
| Validation failure (malformed payload) | Fail permanently — do not retry |
| Unexpected exception | Retry up to max attempts, then DLQ |

**Exponential backoff formula**: `base_delay * 2^attempt` with jitter — prevents thundering herd when many jobs fail simultaneously.

**Minimum retry baseline** — unless the project defines otherwise: **3 attempts with exponential backoff starting at 2 s** (`2s → 4s → 8s`), then DLQ.

Configure per-job retry limits (not global defaults) — a payment job and an analytics job have different tolerance for retries. A payment job warrants fewer attempts and faster DLQ escalation than an analytics event.

---

## Dead Letter Queue (DLQ)

Every queue must have a DLQ configured. Jobs that exhaust retries move there instead of being silently dropped.

**After moving to DLQ:**
1. Alert the on-call team (do not let DLQ fill silently)
2. Inspect failed jobs to identify root cause
3. Fix the root cause, then replay or discard

**Never delete DLQ messages without inspecting them** — they contain the data of failed operations that may need manual resolution.

---

## Job Structure

```ts
// TypeScript / BullMQ example — same principles apply in any framework
class ProcessOrderJob {
  static queue = 'orders';
  static attempts = 3;
  static backoff = { type: 'exponential', delay: 2000 };

  async handle(payload: ProcessOrderPayload): Promise<void> {
    // 1. Validate payload
    const { orderId } = validateProcessOrderPayload(payload);

    // 2. Idempotency check
    const order = await Order.findOrFail(orderId);
    if (order.status !== OrderStatus.PENDING) return;

From Dev-Toolbelt/dev-team-agents, MIT.

More from Dev-Toolbelt/dev-team-agents