TerraSentinel

TerraSentinel watches the planet's slow disasters — wildfires, vanishing sea ice, deforestation and glacier change — using only free public data and free compute. GitHub Actions runs the jobs, Hugging Face Hub stores the lake and the model registry, Turso serves the gold tables, and a Cloudflare Pages dashboard reads them at the edge.

What it is

Four public sources feed the pipeline: NASA FIRMS active-fire detections, Copernicus Sentinel-1 and Sentinel-2 imagery processed in Google Earth Engine, NOAA OISST v2.1 sea-surface temperature plus the NSIDC Sea Ice Index, and ENTSO-E day-ahead prices and load. Collectors land raw data into a versioned bronze lake on Hugging Face. dbt on DuckDB builds staging, intermediate and seven gold marts. An IsolationForest scores the latest features, and batch scoring writes predictions back into the serving database.

The transform layer

Bronze parquet is read straight from the Hugging Face Hub. Staging deduplicates fire detections and unifies MODIS and VIIRS brightness, builds region series and a resolution-5 grid for Sentinel, and computes sea-ice and SST anomalies against the NSIDC 1981-2010 climatology. Intermediate models build a dense region-by-day spine, H3 cells for the map, a circular plus-or-minus-15-day fire baseline, and monthly Sentinel change. Gold ships one mart per source arm, so an outage in one source can only remove its own table.

Anomaly definitions

Every mart states its baseline, because the anomaly is the whole product: daily fire counts against a median over the same plus-or-minus-15 days scaled by MAD, monthly NDVI against the same month a year earlier, sea-ice extent against the published NSIDC normal, and SAR backscatter year-over-year. Fires use median and MAD rather than mean and standard deviation, because a large fire sits inside its own baseline window and would inflate the yardstick it is measured against.

The ML layer

An IsolationForest is trained on 27 strictly causal features, meaning every rolling window ends before the day being scored. The score is a percentile against the training distribution, so it reads the same at serving time as in training. MLflow logs the exact dataset commit behind each run, and the model is published to a Hugging Face model repo.

The dashboard

Eight pages run on Cloudflare's edge: Overview, Analysis, Stories, Catalog, Explorer, SQL and Ops. The charts are hand-rolled SVG rather than a charting library, so the first paint contains the chart and a JavaScript failure cannot blank a panel. The map uses real H3 cells, and severity uses a luminance ramp rather than red-green. Every API route is read-only and precomputed; no inference runs on the request path.

The Databricks hybrid path

Alongside the free-tier pipeline, the same lake is mirrored as a Databricks Asset Bundle: six Workflows jobs on Quartz crons, declarative Spark and Delta Lake pipelines, Unity Catalog DDL and MLflow registration. It ships paused by design, so handing over from GitHub Actions to Databricks is a one-line, side-effect-free flip. The Unity Catalog DDL and MERGE statements are generated from a single source of truth and checked in CI, so the schema cannot drift.

Proving it works

An unsupervised model can always claim to work, so the pipeline ships a falsifiable check. A synthetic lake plants one obvious event per anomaly type, the full dbt build runs against it, and CI asserts each event is flagged while the overall flag rate stays under 2%, which rejects a degenerate baseline that flags everything.