Andrew Hu

Software

Systems, libraries, and instruments

Pithecene

Quarry

Web scraping in TypeScript, with results you can inspect later.

The script stays in control of the browser or HTTP requests. Quarry runs it, saves its output to disk or S3, and keeps a record of the run alongside the data.

Uses Lode to store its results.

Hand-tinted wood engraving of stonecutters working blocks below a sandstone cliff
Quarry’s banner adapts Auguste Lepère’s Fontainebleau Forest: Sandstone Quarry (1890, Cleveland Museum of Art, CC0).

Harbor notices

A saved run collecting notices from several harbor pages.

Inspect run on pithecene.sh →

  1. 41/26 Anchorage boundary moved east Outer Anchorage

  2. 42/26 Buoy 7 relocated Approach Channel

    Starboard-hand buoy 7 now stands 60 metres north of its charted position.

  3. 43/26 Light characteristic changed North Breakwater

Pithecene

Lode

A Go library for versioned datasets on disk or S3.

Writing data and publishing it are separate steps. Lode groups files into snapshots, with a manifest that records which files belong together. An application can finish writing a batch before making that snapshot available to readers.

Quarry uses Lode to store its results.

Engraved geological cross-section of folded strata cut diagonally by a mineral vein
Lode’s banner shows folded, faulted strata cut by a mineral-bearing vein (Pithecene).

Vessel positions

One snapshot brings together position reports from two days.

Inspect snapshot on pithecene.sh →

VesselReported (UTC)PositionKnots
V-20114 Oct · 06:1251.95° N, 1.41° E11.8
V-34714 Oct · 06:1351.90° N, 1.66° E9.4
V-20115 Oct · 06:1052.11° N, 2.05° E12.1
V-51215 Oct · 06:1451.89° N, 1.53° E0.2