databricks-bundles

Files (3)

• SDP_guidance.md
• SKILL.md
• alerts_guidance.md

SKILL.md (first lines)

# DABs Writer

## Overview
Create DABs for multi-environment deployment (dev/staging/prod).

## Reference Files

- **[SDP_guidance.md](SDP_guidance.md)** - Spark Declarative Pipeline configurations
- **[alerts_guidance.md](alerts_guidance.md)** - SQL Alert schemas (critical - API differs)

## Bundle Structure

```
project/
├── databricks.yml           # Main config + targets
├── resources/*.yml          # Resource definitions
└── src/                     # Code/dashboard files
```

### Main Configuration (databricks.yml)

```yaml
bundle:
  name: project-name

include:
  - resources/*.yml

variables:
  catalog:
    default: "default_catalog"
  schema:
    default: "default_schema"
  warehouse_id:
    lookup:
      warehouse: "Shared SQL Warehouse"

targets:
  dev:
    default: true
    mode: development
    workspace:
      profile: dev-profile
    variables:
      catalog: "dev_catalog"
      schema: "dev_schema"

  prod:
    mode: production
    workspace:
      profile: prod-profile
    variables:
      catalog: "prod_catalog"
      schema: "prod_schema"
```

### Dashboard Resources

**Support for dataset_catalog and dataset_schema parameters added in Databricks CLI 0.281.0  (January 2026)**

```yaml
resources:
  dashboards:
    dashboard_name:
      display_name: "[${bundle.target}] Dashboard Title"
      file_path: ../src/dashboards/dashboard.lvdash.json  # Relative to resources/
      warehouse_id: ${var.warehouse_id}
      dataset_catalog: ${var.catalog} # Default catalog used by all datasets in the dashboard if not otherwise specified in the query
      dataset_schema: ${var.schema} # Default schema used by all datasets in the dashboard if not otherwise specified in the query
      permissions:
        - level: CAN_RUN
          group_name: "users"
```

**Permission levels**: `CAN_READ`, `CAN_RUN`, `CAN_EDIT`, `CAN_MANAGE`

### Pipelines

**See [SDP_guidance.md](SDP_guidance.md)** for pipeline configuration

### SQL Alerts

**See [alerts_guidance.md](alerts_guidance.md)** - Alert schema differs significantly from other resources

### Jobs Resources

```yaml
resources:
  jobs:
    job_name:
      name: "[${bundle.target}] Job Name"
      tasks:
        - task_key: "main_task"
          notebook_task:
            notebook_path: ../src/notebooks/main.py  # Relative to resources/
          new_cluster:
            spark_version: "13.3.x-scala2.12"
            node_type_id: "i3.xlarge"
            num_workers: 2
      schedule:
        quartz_cron_expression: "0 0 9 * * ?"
        timezone_id: "America/Los_Angeles"
      permissions:
        - level: CAN_VIEW
          group_name: "users"
```

**Permission levels**: `CAN_VIEW`, `CAN_MANAGE_RUN`, `CAN_MANAGE`

⚠️ **Cannot modify "admins" group permissions** on jobs - verify custom groups exist before use

### Path Resolution

⚠️ **Critical**: Paths depend on file location:

| File Location | Path Format | Example |
|--------------|-------------|---------|
| `resources/*.yml` | `../src/...` | `../src/dashboards/file.json` |
| `databricks.yml` targets | `./src/...` | `./src/dashboards/file.json` |

From ThomazRossito/data-agents, MIT.

More from ThomazRossito/data-agents