reading-portolan

Files (5)

• SKILL.md
• map-inline.html · reference/
• map-multi.js · reference/
• map-style.html · reference/
• style-default.json · reference/

SKILL.md (first lines)

<!-- drift: depends-on: portolan-spec -->

# Reading Portolan Data

A Portolan catalog is a STAC catalog served as static files from object storage. Vector data is GeoParquet paired with PMTiles. Raster data is Cloud Optimized GeoTIFF (COG). Tabular data is plain Parquet with no geometry column. There is no API server. You read the JSON, then query the files in place with DuckDB or rasterio.

The [Portolan specification](https://github.com/portolan-sdi/portolan-spec) is ground truth. Read `specs/portolan/core.md` and `specs/portolan/formats.md` at tag v0.2.0 when this skill and a catalog disagree. Rule ids such as `PORTO-CORE-061` point into `specs/portolan/requirements.yaml`.

## Step 1: Detect the environment

Check what is installed before you suggest a tool.

```bash
duckdb --version
python -c "import duckdb; print(duckdb.__version__)"
python -c "import geopandas; print(geopandas.__version__)"
python -c "import rasterio; print(rasterio.__version__)"
python -c "import rioxarray; print('rioxarray ok')"
gpio --version
ogr2ogr --version
```

| Installed | Vectors and tables | Rasters |
|---|---|---|
| DuckDB | `read_parquet` with the `spatial` and `httpfs` extensions. Best choice. | Not applicable |
| GeoPandas only | `gpd.read_parquet()` | Not applicable |
| rioxarray | Not applicable | `rioxarray.open_rasterio()`. Best choice. |
| rasterio only | Not applicable | `rasterio.open()` |
| Nothing | Suggest DuckDB | Suggest rioxarray |

If nothing is installed, explain the options and offer to guide an install. Respect a no. Every tool above reads remote files by HTTP range request, so nobody needs to download a whole file first.

Install a current DuckDB with the `spatial` extension.

## Step 2: Read the metadata

### Catalog Layout

```
catalog-root/
├── catalog.json                 # Root STAC Catalog
├── AGENTS.md                    # Agent guide, linked as rel: agents
├── README.md                    # Human guide, linked as rel: describedby
└── {collection_id}/
    ├── collection.json          # Extent, providers, license, assets, links
    ├── AGENTS.md
    ├── README.md
    ├── {data}.parquet           # data asset
    ├── {data}.pmtiles           # rel: pmtiles link
    ├── thumbnail.png            # thumbnail asset
    ├── styles/default.json      # style asset with the default role
    └── {item_id}/{item_id}.json # one directory per item, when items exist
```

Each catalog and each collection includes an `AGENTS.md` and a `README.md` (PORTO-CORE-061, PORTO-CORE-062). A `versions.json` beside them is a CLI artifact, not part of the specification. Ignore it when you read data. There is no `llms.txt` in the specification.

### Crawl the catalog

Start at `catalog.json`. Follow every link with `rel: child`. A child with `"type": "Catalog"` is a sub-catalog, so recurse into it. A child with `"type": "Collection"` is a dataset. Build the full inventory before you answer a question.

```bash
BASE=https://example.com/catalog
curl -s $BASE/catalog.json | jq -r '.links[] | select(.rel=="child") | .href'
```

### Read AGENTS.md first

Every collection links its `AGENTS.md` with `rel: agents` and its `README.md` with `rel: describedby`. Read `AGENTS.md` before you write a query. It lists the join keys, the coordinate system, the useful aggregations, and the data quality traps. The README documents the schema and the provenance.

```bash
COLL=$BASE/collection-id
curl -s $COLL/collection.json \
  | jq -r '.links[] | select(.rel=="agents" or .rel=="describedby") | .href'
curl -s $COLL/AGENTS.md
```

### Read collection.json

```bash
curl -s $COLL/collection.json | jq '{
  id, title, license, updated,
  spec: [.stac_extensions[] | select(test("portolan-sdi.org/portolan"))],
  bbox: .extent.spatial.bbox,
  time: .extent.temporal.interval,
  providers: [.providers[] | {name, roles}],
  via: [.links[] | select(.rel=="via" or .rel=="canonical") | .href],
  pmtiles: [.links[] | select(.rel=="pmtiles") | {href, layers: .["pmtiles:layers"]}],
  assets: (.assets | to_entries | map({key, href: .value.href, type: .value.type, roles: .value.roles})),
  glob: .["partition:glob"],
  columns: [.["table:columns"][]? | {name, type}]
}'
```

Each field reports the following:

- `stac_extensions` carries `https://schemas.portolan-sdi.org/portolan/v0.2.0/schema.json`. That URI is the only signal of the specification version (PORTO-CORE-006).
- `license` is an SPDX identifier, or `other` with a `rel: license` link (PORTO-CORE-058, PORTO-CORE-059). Report it before you redistribute anything.
- `providers` lists at least one `producer` and exactly one `host`, last (PORTO-CORE-047). When the producer and the host are the same organization, the catalog is official. When they differ, the catalog is a mirror.
- A mirror has a `via` link (`text/html`) to the original source (PORTO-CORE-053). It has a `canonical` link when the source publishes its own STAC (PORTO-CORE-054). It sets top-level `updated` to the last sync time (PORTO-CORE-057). Compare `updated` with the source when freshness matters.
- `table:columns` documents the columns with names, types, and descriptions. Read it before `DESCRIBE`.
- `partition:glob` is present only on a partitioned collection. See Step 4.

### Find assets by role

An asset key means nothing. Filter on `roles` (PORTO-CORE-027). Each asset has a `type` and at least one role (PORTO-CORE-025).

| Role | What it is | Media type |
|---|---|---|
| `data` | The primary GeoParquet, Parquet, or COG | `application/vnd.apache.parquet` or `image/tiff; application=geotiff; profile=cloud-optimized` |
| `collection-mirror` | `items.parquet`, a stac-geoparquet copy of the items | `application/vnd.apache.parquet` |
| `style` | A MapLibre GL style. One also carries `default`. | `application/vnd.mapbox.style+json` |
| `visual` | The PMTiles file, when it is also a distribution asset | `application/vnd.pmtiles` |
| `thumbnail` | Preview image | `image/png`, `image/jpeg`, or `image/webp` |
| `source` | The upstream original a mirror converted from | Any |

```bash
curl -s $COLL/collection.json \
  | jq -r '.assets | to_entries[] | select(.value.roles | index("data")) | .value.href'
```

From portolan-sdi/portolan-skills, Apache-2.0.

More from portolan-sdi/portolan-skills