azure-messaging

Files (1)

• SKILL.md

SKILL.md (first lines)

# Azure Messaging

Choose the right Azure messaging service and delivery semantics for reliable, decoupled, event-driven systems.
Handle retries, failures, and exactly-once processing.

## Use This Skill When

- The user needs asynchronous processing design (commands, events, streams)
- The user asks about retries, dead-lettering, idempotency, or ordering
- The user needs to decouple systems via events or messaging
- The user needs high-throughput streaming or reactive patterns

## Context: Messaging Maturity

**Immature**: Synchronous API calls, no retries, tightly coupled  
**Developing**: Simple queues created, basic retry logic  
**Managed**: Service Bus/Event Hub with DLQ, idempotency, monitoring → **Target**  
**Optimized**: Saga pattern for distributed transactions, stream processing, real-time analytics

## Required Inputs

- **Message pattern**: Command (request-response), Event (pub-sub), Stream (high-volume telemetry)
- **Throughput**: 100 msg/sec? 1M msg/sec?
- **Latency**: <1sec? Batch processing acceptable?
- **Delivery guarantee**: At most once? At least once? Exactly once?
- **Ordering requirement**: FIFO per partition? Or unordered?
- **Consumer pattern**: Single consumer? Multiple competing consumers? Fan-out?
- **Scale**: Number of producers, consumers, message size?

## Decision Tree

```
What is the main use case?
├─ Simple queue (deferred processing) → Storage Queues (cheapest, simple)
├─ Enterprise queue (retries, DLQ, partitioning) → Service Bus Queue
├─ Publish-Subscribe (fan-out) → Service Bus Topic or Event Grid
├─ High-volume streaming (1000+ msg/sec) → Event Hubs
└─ Reactive events (webhooks, triggers) → Event Grid

What's the throughput target?
├─ <1000 msg/sec → Storage Queues or Service Bus
├─ 1000-10,000 msg/sec → Service Bus with partitions
├─ > 10,000 msg/sec → Event Hubs
└─ Needs low latency (<100ms) → Event Hubs or Service Bus Premium

Do you need ordering or exactly-once semantics?
├─ Yes, FIFO per partition → Event Hubs or Service Bus with sessions
├─ Yes, exactly-once → Service Bus with deduplication ID
└─ No, at-least-once acceptable → Any service + idempotency in handler

How many consumers need this message?
├─ One → Queue (Service Bus or Storage)
├─ Multiple (fan-out) → Service Bus Topic or Event Grid
└─ Streams (continuous analytics) → Event Hubs
```

## Workflow

### Phase 1: Service Selection

| Service | Use Case | Throughput | Ordering | Features |
|---------|----------|-----------|----------|----------|
| **Storage Queues** | Simple, cheap | <2K msg/sec | No | TTL, visibility timeout |
| **Service Bus Queue** | Enterprise, retries | Up to 2K msg/sec | FIFO (sessions) | DLQ, dedup, TTL, auth |
| **Service Bus Topic** | Pub-Sub, fan-out | Up to 2K msg/sec | No | Subscriptions, filters |
| **Event Hubs** | Streaming, telemetry | 1M+ msg/sec | Per partition | Consumer groups, retention |
| **Event Grid** | Reactive, webhooks | Millions/sec | No | Built-in integrations |

### Phase 2: Create Messaging Infrastructure

1. **Service Bus Queue** (for command processing):
   ```bash
   az servicebus queue create \
     --namespace-name "sb-${app}-${env}" \
     --resource-group $RG \
     --name "order-processing" \
     --max-delivery-count 10 \
     --duplicate-detection-history-time-window "PT5M" \
     --dead-letter-on-filter-evaluation-error \
     --dead-letter-on-message-expiration
   
   # Max retries: 10, then DLQ
   # Dedup window: 5 minutes (for idempotency)
   # Auto DLQ on TTL expiration
   ```

2. **Service Bus Topic** (for event distribution):
   ```bash
   az servicebus topic create \
     --namespace-name "sb-${app}-${env}" \
     --resource-group $RG \
     --name "order-events" \
     --enable-subscriptions true
   
   # Create subscription (consumer group)
   az servicebus topic subscription create \
     --namespace-name "sb-${app}-${env}" \
     --resource-group $RG \
     --topic-name "order-events" \
     --name "email-service" \
     --max-delivery-count 3 \
     --dead-letter-on-filter-evaluation-error
   
   # Different consumers listen on same topic via subscriptions
   ```

3. **Event Hubs** (for streaming):
   ```bash
   az eventhubs eventhub create \
     --namespace-name "eh-${app}-${env}" \
     --resource-group $RG \
     --name "telemetry" \
     --partition-count 4 \
     --message-retention 1 \
     --enable-capture true \
     --capture-interval-in-seconds 300 \
     --capture-size-limit-in-bytes 314572800
   
   # 4 partitions for 4 concurrent consumers

From manu14357/zskills, MIT.

More from manu14357/zskills