geoprimsField-grade geospatial math

Distance by vertical sextant angle

Distance off an object of charted height from its vertical sextant angle: above the sea horizon for an object beyond it (Bowditch Table 15), or from its waterline for one within the horizon (Table 16).

Planning and education aid. Not for primary navigation. Full disclaimer

Experimental: not yet fully verified. How results are checked

1.8NM

The object is 1.8 NM off.

Distance off
Distance to the sea horizon
Provenance
Computed by
navigation.piloting.distance-by-vertical-angle 1.0.0, core 0.1.0
Model
Sea horizon (Table 15): D = √((tan a / 0.0002419)² + (H − h) / 0.7349) − tan a / 0.0002419, the constants for terrestrial refraction; waterline (Table 16): the plane right triangle D = H / tan a, in NM of 6,076.1 ft; horizon 1.17 √h NM
Accuracy
Table 15's refraction constants hold for standard conditions; the result matches the tables to their 0.1 NM
Notes
1 shown with the answer
Cites
National Geospatial-Intelligence Agency, The American Practical Navigator (Bowditch), NGA Pub. No. 9, Volume II

Something look off?

Your values

Showing an example. Change anything.
More options 1
Run many at once from a CSV

Loading…

How we got thisFormula, worked example, sources, and proof

Model: Sea horizon (Table 15): D = √((tan a / 0.0002419)² + (H − h) / 0.7349) − tan a / 0.0002419, the constants for terrestrial refraction; waterline (Table 16): the plane right triangle D = H / tan a, in NM of 6,076.1 ft; horizon 1.17 √h NM

Show your work

  1. Sea horizon

    1.17h

    h = 20 ft = 5.2 NM

  2. Distance

    D=((tan⁡a/0.0002419)2+(H−h)/0.7349)−tan⁡a/0.0002419

    H 120 ft, h 20 ft, a 0.500° = 1.8 NM

The same steps an agent gets from the MCP server with explain: true.

Accuracy: Table 15's refraction constants hold for standard conditions; the result matches the tables to their 0.1 NM

When to use this: Use this to get a distance off a lighthouse, peak, or other object of known height with a sextant: measure the angle from the sea horizon up to the top, correct it for index error and dip, and enter the heights. For an object inside the horizon, measure from its waterline instead.

Limitations: Use the height of the light's center, not the tower's top, and remember charted heights are above mean high water, so the tide changes the real height a little. Refraction differing from standard skews the result. The sea-horizon form needs the object beyond the horizon and the waterline form needs it inside; the tool warns when the answer says otherwise.

Worked example: An object 100 ft above the eye at 0°30'. Source: Bowditch 2024 Vol. II Table 15: 1.8 NM. It is golden test vector v001, and every build checks the tool still gives its answer within its tolerance.

You enter

Corrected angle
0°30'
Height of eye
20 ft
Object height
120 ft

You get

Distance
1.8 NM
Distance off
1.84 NM
Distance to the sea horizon
5.23 NM

Review: Not yet independently reviewed by a geodesist.

Last verified: 2026-09-19, when a maintainer last confirmed this tool's sources at the issuer. See the sources ledger.

Status: version 1.0.0, core 0.1.0. See this tool in the verification report.

Checked against: 21 golden test vectors (download the test vectors, each with its source and tolerance). See how results are checked and every source.

Sources

Experimental means this tool has not yet met the stable bar: at least 20 golden vectors, differential tests, and an independent worked example.