Data#

This directory holds intermediate files used to generate the .pmtiles map tile files served by the www app. Output tiles are written to www/static/tiles/. We don't really need suuuuuuper up-to-date maps for this, since we trim a lot of data; the tiles are mainly used as a way to orient the user as they create their own map.

All tile generation is done via the CLI:

deno task data <command> [options]
deno task data --help

Dependencies#

The following tools must be installed and on PATH:

tilemaker — PBF → PMTiles conversion (build, build:world)
osmium — PBF extraction (extract command)

The following resources are necessary to generate pmtiles:

Download a planet .osm.pbf for the world-level pmtiles (put it in ./osm/planet-latest.osm.pbf). For me, using a torrentfile has been the most stable way.
Download regional .osm.pbf files for regions if you don't want to self-extract (can download with download:osm)
Coastlines WGS84 projection from https://osmdata.openstreetmap.de/data/coastlines.html (put it in ./cli/shared/tilemaker/coastline) (todo for me: create a download:coastlines cli cmd)

CLI Commands#

Command	Description
`list [--search <term>]`	Browse available region slugs
`download:osm <region> [--force]`	Download `.osm.pbf` from Geofabrik
`download:poly [region] [--all] [--force]`	Download `.poly` boundary file(s)
`extract <region> --from <source>`	Carve a sub-region from a larger PBF using osmium
`build <region>`	Convert `.osm.pbf` → `.pmtiles` using tilemaker
`trim:world`	Strip planet PBF to only data needed for world tiles (saves memory)
`build:world`	Build world-scale basemap tiles
`clean --region <slug> \| --all`	Remove intermediate files

World Tiles#

The world basemap is built from OSM data using a tilemaker pipeline, sourced from a full planet PBF. Place planet-latest.osm.pbf in data/osm/ and run:

deno task data trim:world                         # recommended first step — see below
deno task data build:world [--maxzoom <5|7|9>]   # default maxzoom: 7

Output is written to www/static/tiles/world_z<maxzoom>.pmtiles. The config and Lua script used are separate from the regional ones:

data/cli/shared/tilemaker/config.world.json — layer definitions for the world overview
data/cli/shared/tilemaker/process.world.lua — feature processing logic

Note: this takes a super long time to run.

Trimming the planet file (recommended)#

The full planet PBF (~75 GB) contains a huge amount of data that is irrelevant at world zoom levels — buildings, addresses, shop/amenity POIs, minor roads, etc. Running trim:world uses osmium tags-filter to produce a much smaller planet-trimmed.osm.pbf containing only the tags that process.world.lua actually reads:

Place nodes (countries, states, cities, towns, villages)
Natural peaks and volcanoes
Administrative boundaries
Major roads (motorway → secondary) and ferry routes
Main-line railways
Rivers, waterways, water polygons
Landcover and landuse polygons
Parks and nature reserves

deno task data trim:world            # creates data/osm/planet-trimmed.osm.pbf
deno task data trim:world --overwrite  # re-run and replace an existing trimmed file

When planet-trimmed.osm.pbf is present, build:world will automatically use it instead of the full planet file. This significantly reduces tilemaker's peak memory usage and overall build time.

Regional Tiles#

Regional tiles follow a three-stage pipeline:

Geofabrik (download) → osmium (extract; optional) → tilemaker (build) → .pmtiles

1. Download#

Regional .osm.pbf data and .poly boundary files are sourced from Geofabrik. Region slugs match Geofabrik's hierarchy (e.g. asia/japan/kansai, europe/monaco).

.poly files for all 499 Geofabrik regions are already committed to data/poly/ and don't need to be refreshed under normal circumstances. OSM .pbf files are gitignored due to size and must be downloaded as needed.

deno task data download:osm europe/monaco
deno task data download:poly europe/monaco   # already present; only needed to refresh

2. Extract (optional)#

If you already have a large regional PBF, you can carve out a sub-region using osmium rather than downloading from Geofabrik again:

deno task data extract asia/taiwan --from asia

This requires both the source PBF and the target .poly file to be present.

3. Build#

Convert .osm.pbf to .pmtiles using tilemaker:

deno task data build europe/monaco

Output is written to www/static/tiles/<region-name>.pmtiles. The tilemaker config and Lua processing script are:

data/cli/shared/tilemaker/config.json — layer definitions (OpenMapTiles schema, zoom 0–14)
data/cli/shared/tilemaker/process.lua — feature processing logic

Common Workflows#

Small country — direct download:

deno task data download:osm europe/monaco
deno task data build europe/monaco

Country carved from a larger extract:

deno task data download:osm asia # large, but reusable across multiple countries
deno task data extract asia/taiwan --from asia
deno task data build asia/taiwan

Directory Structure#

data/
  osm/          Downloaded/extracted .osm.pbf files (gitignored)
  poly/         Boundary .poly files for all 499 Geofabrik regions (committed)
  cli/          CLI source code

External Resources#

Geofabrik — regional OSM extracts; provides both .osm.pbf and .poly files
OpenStreetMap — planet PBF source for world basemap tiles

Configure Feed