EDM atmospheric and prism corrections
Corrects a measured distance for the air it traveled through, using entered ppm, custom instrument constants, or a named Trimble Access formula, and lists the atmospheric and prism corrections separately.
Experimental: not yet fully verified. How results are checked
The corrected distance is 999.982 m: 12 ppm for the air and -0.03 m for the prism.
- Atmospheric correction
- Atmospheric correction
- Prism correction
Provenance
- Computed by
- survey.reduction.edm-correction 1.1.0, core 0.1.0
- Model
- Custom IAG 1999: N_G = 287.6155 + 4.88660 ÷ λ² + 0.06800 ÷ λ⁴; N_L = (273.15 ÷ 1013.25) · N_G · p ÷ T − 11.27 · e ÷ T; ppm = N_ref − N_L. Trimble Access presets: ppm = J − N·p ÷ (273.16 + t), using the guide's J and N for the selected instrument family. Corrected = D + ppm · 10⁻⁶ · D + prism constant
- Accuracy
- The custom IAG 1999 formula is within 0.25 ppm of the full Ciddor and Hill procedure for ordinary survey conditions. The named presets reproduce Trimble Access's published pressure-and-temperature equation and constants. Instruments that apply ppm internally need no second correction
- Notes
- 1 shown with the answer
- Cites
- Landgate (Western Australian Land Information Authority) with ICSM, The Survey Instrumentation Calibration Technical User Manual; Trimble Inc., Trimble Access Calculations Reference Guide
Something look off?
How we got thisFormula, worked example, sources, and proof
Model: Custom IAG 1999: N_G = 287.6155 + 4.88660 ÷ λ² + 0.06800 ÷ λ⁴; N_L = (273.15 ÷ 1013.25) · N_G · p ÷ T − 11.27 · e ÷ T; ppm = N_ref − N_L. Trimble Access presets: ppm = J − N·p ÷ (273.16 + t), using the guide's J and N for the selected instrument family. Corrected = D + ppm · 10⁻⁶ · D + prism constant
Show your work
Atmospheric correction
ppm × 10⁻⁶ × measured distance12 × 10⁻⁶ × 1,000= 0.012 mCorrected distance
1,000 + 0.012 + -0.03= 999.982 m
The same steps an agent gets from the MCP server with explain: true.
Accuracy: The custom IAG 1999 formula is within 0.25 ppm of the full Ciddor and Hill procedure for ordinary survey conditions. The named presets reproduce Trimble Access's published pressure-and-temperature equation and constants. Instruments that apply ppm internally need no second correction
When to use this: Use this when you need to correct an EDM distance for the air and the prism, such as when the ppm was not set in the instrument. Enter the measured distance, the prism constant, and either the ppm or temperature and pressure. For the latter, choose an exact Trimble Access instrument-family preset or give the instrument's wavelength, reference refractivity, and optional humidity. It returns the corrected distance with each correction listed.
Limitations: If the instrument already applied ppm, do not apply it again. Use a named preset only for that exact instrument family; current Trimble VX/S/M instruments supply their own constants and are not preset here. The Trimble Access formula does not use humidity. Temperature and pressure are taken at one spot, while the air along the line may differ. The tool does not reduce slope to horizontal or apply elevation or grid scale factors; those come next.
Worked example: 1,000 m with +12 ppm and a -30 mm prism. Source: add-survey-suite ppm scenario: 1,000.000 + 0.012 − 0.030 = 999.982 m, each correction listed. It is golden test vector v001, and every build checks the tool still gives its answer within its tolerance.
You enter
- Measured distance
- 1000 m
- Atmospheric correction
- 12
- Prism constant
- -30 mm
You get
- Corrected distance
- 999.982 m
- Atmospheric correction
- 12
- Atmospheric correction
- 0.012 m
- Prism correction
- -0.03 m
Review: Not yet independently reviewed by a licensed surveyor.
Last verified: 2026-09-22, when a maintainer last confirmed this tool's sources at the issuer. See the sources ledger.
Status: version 1.1.0, core 0.1.0. See this tool in the verification report.
Changes
- 2026-10-13, added: EDM atmospheric and prism correction (version 1.1.0, experimental) now includes named Trimble Access formulas for the 5600, 3300/3600, and TTS300/500 instrument families. Each preset uses the guide's published J and N constants, and mixed direct-ppm, custom-IAG, and preset inputs are refused instead of silently ignored. The vector set grows from 9 to 25 without changing existing vectors. Changelog
Checked against: 25 golden test vectors (download the test vectors, each with its source and tolerance). See how results are checked and every source.
Constants this tool assumes
Values you do not enter, and where each comes from.
- Group refractivity constant term (IAG 1999)
287.6155— The Survey Instrumentation Calibration Technical User Manual- Group refractivity coefficient on 1/wavelength squared
4.88660 um2— The Survey Instrumentation Calibration Technical User Manual- Group refractivity coefficient on 1/wavelength to the fourth
0.06800 um4— The Survey Instrumentation Calibration Technical User Manual- Reference pressure for the group refractivity
1013.25 hPa— The Survey Instrumentation Calibration Technical User Manual- Water vapor refractivity coefficient
11.27 K/hPa— The Survey Instrumentation Calibration Technical User Manual- Magnus coefficient a (saturation vapor pressure)
17.625— Alduchov and Eskridge (1996) Magnus form- Magnus coefficient b
243.04 degC— Alduchov and Eskridge (1996) Magnus form- Magnus coefficient c
6.1094 hPa— Alduchov and Eskridge (1996) Magnus form- Trimble Access 5600 J constant
274.41— Trimble Access Calculations Reference Guide- Trimble Access 5600 N constant
79.39 K/hPa— Trimble Access Calculations Reference Guide- Trimble Access 3300/3600 J constant
278.77— Trimble Access Calculations Reference Guide- Trimble Access 3300/3600 N constant
80.653 K/hPa— Trimble Access Calculations Reference Guide- Trimble Access TTS300/500 J constant
270.0— Trimble Access Calculations Reference Guide- Trimble Access TTS300/500 N constant
79.167 K/hPa— Trimble Access Calculations Reference Guide
Sources
- The Survey Instrumentation Calibration Technical User Manual, Landgate (Western Australian Land Information Authority) with ICSM, Online edition, updated 24 November 2025. Section 4.2.3, equations 4.6 and 4.7 (the IAG 1999 Resolution 3 closed formula for group refractivity) and equation 4.1 (the first velocity correction).
- Trimble Access Calculations Reference Guide, Trimble Inc., Revision D, April 2024. Pages 9–10, PPM correction equation and J/N constants.
Terms
- EDM: electronic distance measurement
- Measuring a distance by timing light or infrared bounced off a prism or a surface, as a total station does. The air slows the light a little, so the reading is corrected for temperature and pressure, in parts per million. Source: The Survey Instrumentation Calibration Technical User Manual
Experimental means this tool has not yet met the stable bar: at least 20 golden vectors, differential tests, and an independent worked example.
EDM — electronic distance measurement
Measuring a distance by timing light or infrared bounced off a prism or a surface, as a total station does. The air slows the light a little, so the reading is corrected for temperature and pressure, in parts per million.
Source: The Survey Instrumentation Calibration Technical User Manual