Navigation Mathematics

Ocean Navigation & Celestial Mathematics

The mathematics of celestial and ocean navigation bridges spherical trigonometry, astronomy, and practical seamanship. From computing the altitude of a star to predicting tides, every calculation rests on elegant geometric foundations developed over centuries of blue-water voyaging.

1. Spherical Trigonometry

Ordinary plane trigonometry governs flat surfaces. Celestial navigation takes place on the surface of a sphere — specifically on the celestial sphere, an imaginary sphere of infinite radius centered on the observer. Spherical trigonometry is the geometry of triangles drawn on that surface, where the sides are arcs of great circles and the angles are dihedral angles between the planes of those great circles.

The Spherical Law of Cosines

For a spherical triangle with sides a, b, c (measured as angles, i.e., arcs on the unit sphere) and opposite angles A, B, C, the law of cosines for sides states:

Law of Cosines for Sides:

  • cos(a) = cos(b) cos(c) + sin(b) sin(c) cos(A)
  • cos(b) = cos(a) cos(c) + sin(a) sin(c) cos(B)
  • cos(c) = cos(a) cos(b) + sin(a) sin(b) cos(C)

Law of Cosines for Angles:

  • cos(A) = -cos(B) cos(C) + sin(B) sin(C) cos(a)

Notice the negative sign in the angle formula — a critical difference from the planar case. When the triangle degenerates to a flat (infinitely small) triangle, these formulas reduce to the familiar plane law of cosines.

The Spherical Law of Sines

Analogously to the plane law of sines, the spherical version relates sides and their opposite angles:

sin(a) / sin(A) = sin(b) / sin(B) = sin(c) / sin(C)

Equivalently: sin(A) / sin(a) = sin(B) / sin(b) = sin(C) / sin(c)

The law of sines resolves ambiguity less cleanly on the sphere than in the plane because an angle or side can be either acute or obtuse, and both may satisfy the sine equation. The four-parts formula resolves some of these cases.

The Four-Parts Formula

The four-parts formula (also called the cotangent formula) relates four consecutive parts of a spherical triangle — two sides and two angles that form a connected chain. Labeling the inner angle A between sides b and c, and the outer angle B opposite side a:

cot(a) sin(b) = cot(A) sin(C) + cos(b) cos(C)

In words: (cotangent of inner side)(sine of adjacent side) = (cotangent of inner angle)(sine of opposite angle) + (cosine of adjacent side)(cosine of opposite angle)

This formula is particularly useful when you know two sides and the included angle and want an adjacent angle, avoiding the quadrant ambiguity of the law of sines.

Napier's Rules for Right Spherical Triangles

When one angle of a spherical triangle is 90 degrees (a right spherical triangle), Napier's Rules provide a systematic way to find any part from any two known parts. Arrange the five parts (excluding the right angle C = 90): sides a, b, and complements of A, B, and hypotenuse c. Label them consecutively around Napier's Circle.

Napier's Two Rules:

  • Rule of Sines: The sine of any middle part equals the product of the sines of the two adjacent parts.
  • Rule of Cosines: The sine of any middle part equals the product of the cosines of the two opposite parts.

Example: with C = 90, sin(a) = sin(A) sin(c) (rule of sines) and sin(a) = cos(complement-A) cos(complement-c) = cos(co-A) cos(co-c)

Napier's Rules reduce the six possible formulas for a right spherical triangle to two mnemonic rules, dramatically simplifying computation for navigational problems where one angle or side is known to be 90 degrees.

Why This Matters for Navigation

The navigation triangle PZX (pole, zenith, celestial body) is a spherical triangle. Its three sides are: the co-latitude (90 minus latitude), the polar distance (90 minus declination), and the zenith distance (90 minus altitude). Spherical trigonometry formulas applied to this triangle produce every fundamental formula in celestial navigation.

2. The Celestial Sphere: Coordinates and Hour Angles

The celestial sphere is a conceptual construct: an imaginary sphere of arbitrarily large radius centered on the observer (or on Earth's center, for an earth-centered system). Celestial bodies are projected onto this sphere. Navigation uses two coordinate systems on the celestial sphere: the equatorial system (declination and hour angle) and the horizon system (altitude and azimuth).

Equatorial Coordinates

Declination (d or Dec)

The celestial equivalent of latitude. Measured north (+) or south (-) from the celestial equator, 0 to 90 degrees. The sun's declination ranges from 23.5 degrees N at the summer solstice to 23.5 degrees S at the winter solstice, crossing zero at the equinoxes.

Right Ascension (RA)

Measured eastward from the vernal equinox (First Point of Aries) along the celestial equator, in hours, minutes, seconds (0 to 24 hours). Each hour corresponds to 15 degrees of arc. Navigation tables typically use SHA (Sidereal Hour Angle) rather than RA: SHA = 360 degrees minus (RA in degrees).

Hour Angle System

Hour angles are measured westward, unlike right ascension which is measured eastward. This is because the celestial sphere appears to rotate westward (due to Earth rotating eastward), so hour angles increase as time passes.

Hour Angle Definitions:

  • GHA (Greenwich Hour Angle): Angular distance measured westward from the Greenwich meridian to the meridian of the celestial body. Tabulated in the Nautical Almanac for each hour. Range: 0 to 360 degrees (or 0 to 360 degrees W).
  • LHA (Local Hour Angle): Angular distance measured westward from the observer's meridian to the body's meridian. LHA = GHA + (west longitude is subtracted, east longitude is added). When LHA = 0, the body is on the meridian (upper transit). When LHA = 180, the body is on the meridian below the pole (lower transit).
  • SHA (Sidereal Hour Angle): For stars: the angular distance westward from the vernal equinox (Aries) to the star. GHA star = GHA Aries + SHA star (reduce modulo 360).

LHA Computation Example

Given: DR position 35 degrees 24.0 N, 142 degrees 18.0 W

From Nautical Almanac at observation time:

  • GHA Sun (tabulated hour) = 312 deg 14.7
  • Increment (for minutes/seconds) = + 5 deg 22.5
  • GHA Sun = 317 deg 37.2
  • Longitude (W, so subtract) = -142 deg 18.0
  • LHA Sun = 175 deg 19.2

LHA is between 90 and 270 degrees, confirming the sun is west of meridian and below the equinoctial plane in hour angle terms.

Azimuth Angle Z and True Azimuth Zn

Azimuth angle Z is the angle at the zenith of the navigation triangle, measured from north (in the northern hemisphere) or south (in the southern hemisphere). True azimuth Zn is measured clockwise from true north, 0 to 360 degrees.

Conversion Rules (Zn from Z):

  • Northern Hemisphere, LHA greater than 180 (body east of meridian):
  • Zn = Z
  • Northern Hemisphere, LHA less than 180 (body west of meridian):
  • Zn = 360 degrees minus Z
  • Southern Hemisphere, LHA greater than 180:
  • Zn = 180 degrees minus Z
  • Southern Hemisphere, LHA less than 180:
  • Zn = 180 degrees + Z

3. Altitude and Azimuth Computation: The Navigation Triangle

The navigation triangle PZX connects three points on the celestial sphere: P (the elevated pole, north or south depending on hemisphere), Z (the observer's zenith), and X (the celestial body). The sides and angles of this triangle encode the observer's position and the body's position in a single geometric relationship.

The Three Sides of Triangle PZX

Side PZ

Co-latitude = 90 degrees minus observer latitude L. Connects the pole to the zenith.

Side PX

Polar distance = 90 degrees minus declination d. Connects the pole to the body.

Side ZX

Zenith distance = 90 degrees minus altitude Hc. Connects the zenith to the body.

The Hc Formula (Computed Altitude)

Applying the spherical law of cosines to the navigation triangle with side ZX (zenith distance) opposite angle P (which equals LHA):

Primary Formula:

sin(Hc) = sin(L) sin(d) + cos(L) cos(d) cos(LHA)

Azimuth from the same triangle:

cos(Z) = [sin(d) minus sin(L) sin(Hc)] / [cos(L) cos(Hc)]

Where L = observer latitude, d = body declination (same name as L is positive, contrary name is negative), LHA = local hour angle. Hc is the computed altitude for comparison with the observed altitude Ho.

Same Name vs. Contrary Name

When latitude and declination are both north or both south (same name), declination enters the formula as positive. When one is north and the other south (contrary name), declination is negative. This sign convention is essential for navigators crossing the equator.

Worked Example: Computing Hc and Z

Given: L = 40 deg N, d = 18 deg N, LHA = 45 deg

  • sin(L) = sin(40 deg) = 0.6428
  • sin(d) = sin(18 deg) = 0.3090
  • cos(L) = cos(40 deg) = 0.7660
  • cos(d) = cos(18 deg) = 0.9511
  • cos(LHA) = cos(45 deg) = 0.7071
  • sin(Hc) = (0.6428)(0.3090) + (0.7660)(0.9511)(0.7071)
  • sin(Hc) = 0.1986 + 0.5156 = 0.7142
  • Hc = arcsin(0.7142) = 45 deg 34.0
  • cos(Z) = [0.3090 minus (0.6428)(0.7142)] / [(0.7660)(0.6999)]
  • cos(Z) = [0.3090 minus 0.4590] / 0.5362
  • cos(Z) = -0.1500 / 0.5362 = -0.2797
  • Z = arccos(-0.2797) = 106.2 deg
  • LHA less than 180 (body east), N hemisphere: Zn = 360 minus 106.2 = 253.8 deg T

Sextant Altitude Corrections to Get Ho

The sextant reading (Hs) must be corrected to obtain the true observed altitude Ho. The principal corrections are:

  • Index Correction (IC): Corrects for misalignment of the index mirror. Positive if index error is "on the arc," negative if "off the arc." IC = minus IE (index error).
  • Dip (D): Corrects for observer height above sea level. Always negative. D (in minutes) = -0.97 sqrt(height in feet) or -1.76 sqrt(height in meters). Eye height 16 ft gives dip of about -3.9 minutes.
  • Refraction (R): Corrects for bending of light through the atmosphere. Always negative (lifts the body). R (minutes) = 0.97 / tan(Hs) for altitudes above 15 degrees. Tabulated in almanac for all altitudes.
  • Semi-diameter (SD): For sun and moon: lower limb add SD, upper limb subtract SD. Sun SD is approximately 16 minutes.
  • Parallax in Altitude (PA): Significant only for the moon. Moon HP (horizontal parallax) from almanac; PA = HP cos(Hs).

Ho = Hs + IC + D + R + SD + PA

4. Sight Reduction: HO 229, HO 249, and the Intercept Method

Sight reduction is the process of computing Hc and Zn from an assumed position, then comparing Hc to Ho to plot a line of position. Before calculators, navigators used pre-computed tables. Two official U.S. government publications dominate: HO 229 (Sight Reduction Tables for Marine Navigation) and HO 249 (Sight Reduction Tables for Air Navigation, also widely used at sea).

HO 229: Marine Sight Reduction

HO 229 consists of six volumes, each covering 15 degrees of latitude. For a given whole-degree latitude (assumed latitude), whole-degree LHA (assumed LHA), and tabulated declination, the tables directly give Hc (to the nearest 0.1 minute) and Z. Interpolation tables handle the remainder of declination.

Using HO 229 Step by Step:

  1. Compute GHA of body; select assumed longitude (Alon) to make LHA a whole degree.
  2. Select assumed latitude (Alat) nearest to DR latitude.
  3. Enter HO 229 with Alat, LHA, and whole degrees of declination; extract tabulated Hc, d, and Z.
  4. Interpolate for declination minutes: correction = (d / 60) times (decimal minutes of declination).
  5. Apply interpolation correction to tabulated Hc to get final Hc.
  6. Convert Z to Zn using hemisphere and LHA rules.
  7. Compute intercept a = Ho minus Hc (in minutes = nautical miles).

HO 249: Air Navigation Tables

HO 249 is organized slightly differently. Volume I contains precomputed data for seven selected stars for each degree of latitude and each degree of LHA Aries — requiring no declination interpolation for stars. Volumes II and III cover declinations 0 to 29 degrees and 30 to 89 degrees for sun, moon, planets, and other celestial bodies.

Advantage of HO 249 Vol. I for Stars:

Enter with whole-degree latitude and whole-degree LHA Aries. The table directly gives Hc and Zn for each of seven optimal stars. No declination interpolation is needed because the star's declination is already embedded in the precomputed values. This is the fastest method for a three-star fix at twilight.

The Marcq St. Hilaire (Intercept) Method

Developed by French naval officer Adolphe-Laurent-Anatole Marcq de Blond de Saint-Hilaire in 1875, this method replaced the older time-sight and noon-latitude methods with a general technique applicable at any time and for any celestial body.

The Intercept Method:

  1. Observe the body with the sextant; record GMT and sextant altitude Hs. Apply corrections to get Ho.
  2. Select an assumed position (AP) near the DR position — typically the nearest whole degree of latitude and a longitude adjusted to give a whole-degree LHA.
  3. From almanac and sight reduction tables (or the Hc formula), compute Hc and Zn for the assumed position.
  4. Compute the intercept: a = Ho minus Hc (in minutes of arc = nautical miles). Positive intercept means the LOP is toward the body (Ho greater than Hc); negative means away.
  5. On the chart, plot the assumed position, draw the azimuth line in direction Zn, measure the intercept distance along that line, and draw the LOP perpendicular to the azimuth at that point.
  6. Repeat for two or more bodies; the intersection of LOPs is the fix.

Worked Intercept Example:

  • Ho = 45 deg 38.2
  • Hc = 45 deg 22.7 (from tables, assumed position)
  • a = Ho minus Hc = +15.5 NM (toward)
  • Zn = 253.8 deg T
  • Plot: From AP, measure 15.5 NM toward 253.8 deg T;
  • draw LOP perpendicular to azimuth at that point.

5. Noon Latitude by Meridian Passage

Meridian passage (local apparent noon, or LAN) is the moment the sun crosses the observer's meridian and reaches its maximum altitude. At this instant, the sun bears due north or due south (true azimuth 000 or 180), and the navigation triangle degenerates: LHA = 0 (or 360). This simplification allows a direct latitude calculation without full sight reduction.

Determining Time of Meridian Passage

The Nautical Almanac gives the time of meridian passage at Greenwich (usually close to 1200 GMT, corrected for the equation of time). The observer's local apparent noon is:

LAN time (GMT) = almanac meridian passage time + longitude correction

Longitude correction: for west longitude, add (longitude in degrees divided by 15) hours; for east longitude, subtract. Alternatively, prepare by watching the sun's altitude increase, shoot at peak, and record the time.

The Noon Latitude Formula

With Ho at meridian passage and the sun's declination d from the almanac (for the GMT of observation):

Four Cases:

  • Observer N, sun bearing S (sun south of zenith):L = (90 deg minus Ho) + d (if d N) or L = (90 deg minus Ho) minus d (if d S)
  • Observer S, sun bearing N:L = (90 deg minus Ho) + d (if d S) or L = (90 deg minus Ho) minus d (if d N)

Simplified rule: Zenith Distance ZD = 90 deg minus Ho. Name ZD with the direction the sun bore (N or S). If ZD and d have the same name, add to get latitude; if contrary, subtract lesser from greater and keep the name of the greater.

Worked Noon Latitude Example

  • Hs at meridian passage = 63 deg 45.0
  • IC = +0.5, Dip (HE 9 ft) = -2.9
  • App Alt = 63 deg 42.6
  • Refraction = -0.7, SD = +16.0 (lower limb)
  • Ho = 63 deg 57.9
  • Sun declination = 18 deg 24.3 N
  • ZD = 90 deg minus 63 deg 57.9 = 26 deg 02.1 (sun bore S)
  • ZD = 26 deg 02.1 S, d = 18 deg 24.3 N (contrary names)
  • Latitude = 26 deg 02.1 minus 18 deg 24.3 = 7 deg 37.8 N

The observer is at latitude 7 degrees 37.8 N.

6. Star Identification and Polaris Latitude

A three-star fix at morning or evening civil twilight provides the most accurate celestial position. Stars must be identified before shooting. Two standard methods: the Star Finder (Rude Star Finder 2102-D) and the mathematical approach using LHA Aries plus the HO 249 Volume I method. Additionally, Polaris (the North Star) provides a direct latitude measurement.

Polaris Latitude Correction

Polaris is not precisely at the celestial north pole — it is currently about 0.7 degrees away. Its observed altitude Hc therefore differs from the observer's latitude by a small correction tabulated in the Nautical Almanac.

Polaris Formula:

L (latitude) = Ho (Polaris) minus 1 deg + a0 + a1 + a2

Where a0 is the main correction (function of LHA Aries), a1 is the latitude correction (function of a0 and latitude), and a2 is the month correction. All three are tabulated in the Nautical Almanac Polaris correction tables. The minus 1 degree appears because the table offset is designed so that a0 ranges from 0 to 2 degrees (not minus 1 to plus 1), keeping all values positive.

Using the Star Finder 2102-D

The Rude Star Finder 2102-D consists of a base disc printed with the positions of 57 selected stars in equatorial coordinates (declination and SHA), plus a set of transparent azimuth-altitude templates — one for each 10 degrees of latitude, printed on both sides (north latitudes on one side, south on the other).

  1. Compute LHA Aries for the time of observation.
  2. Select the template for the observer's latitude (nearest 10 degrees).
  3. Rotate the template until the LHA Aries arrow aligns with the template's meridian.
  4. Stars that fall above the altitude circles are above the horizon; read off approximate altitude and azimuth for each.
  5. Select three well-separated stars (ideally 120 degrees apart in azimuth, altitudes 20 to 60 degrees) for the best fix geometry.

LHA Aries and Star GHA

To find any star's GHA:

GHA star = GHA Aries + SHA star (reduce modulo 360 if needed)

LHA star = GHA star plus east longitude (or minus west longitude)

GHA Aries increases at approximately 15 degrees per hour (one full rotation in one sidereal day = 23 hours 56 minutes). SHA and declination of stars change extremely slowly; almanac values are valid for the year.

7. Great Circle Sailing: Shortest-Path Mathematics

On a sphere, the shortest path between two points is along a great circle — a circle whose plane passes through the center of the sphere. For transoceanic passages, a great circle route can save hundreds of miles compared to a rhumb line. However, a great circle requires constantly changing course; in practice, navigators use composite sailing or waypoints.

Great Circle Distance

Given departure point (L1, Lo1) and destination (L2, Lo2), the angular great circle distance D satisfies:

cos(D) = sin(L1) sin(L2) + cos(L1) cos(L2) cos(DLo)

Where DLo = difference in longitude = Lo1 minus Lo2 (or Lo2 minus Lo1, taken as positive, 0 to 180 degrees). D is in degrees; distance in nautical miles = 60 times D (since 1 degree of arc on Earth = 60 NM).

Initial Course Angle C

The initial course (direction of departure from the starting point) is found from the navigation triangle. The starting point plays the role of the pole, the north pole plays the role of the body, and the destination plays the role of another point:

cos(C) = [sin(L2) minus sin(L1) cos(D)] / [cos(L1) sin(D)]

C is the angle at the departure point. Convert to true course using the hemisphere and direction rules: if destination is to the east, Cn = C; if to the west, Cn = 360 minus C (for northern hemisphere departure heading north).

The Vertex of a Great Circle

The vertex is the highest latitude point of a great circle route. At the vertex, the course is due east or due west (090 or 270 degrees true). The vertex latitude Lv and the longitude difference from the departure point satisfy:

  • cos(Lv) = cos(L1) sin(C)
  • cos(LoV minus Lo1) = tan(L1) / tan(Lv)
  • sin(DLo) = cos(C) / sin(Lv) [for longitude of any waypoint]

Where C is the initial course angle. The vertex latitude is the maximum latitude reached on the route. If the vertex is within the route (between departure and destination), the route passes through it.

Composite Sailing

Composite sailing limits poleward excursion to a maximum latitude (e.g., to avoid ice). The route follows the great circle to where it reaches the limiting latitude, then follows that parallel to where the great circle to the destination also touches it, then follows the second great circle to the destination.

Composite Sailing Distance:

D(total) = D(GC1: departure to limiting lat) + D(parallel sailing segment) + D(GC2: limiting lat to destination). The parallel sailing segment uses: departure = DLo times cos(Llimit), where DLo is the difference in longitude between the two tangent points.

Great Circle vs. Rhumb Line: Worked Comparison

San Francisco (37.8 N, 122.4 W) to Yokohama (35.4 N, 139.6 E):

  • DLo = 122.4 + 139.6 = 262.0 deg (but use 360 minus 262 = 98.0 deg eastward)
  • cos(D) = sin(37.8)sin(35.4) + cos(37.8)cos(35.4)cos(98.0)
  • cos(D) = (0.6143)(0.5793) + (0.7890)(0.8151)(-0.1392)
  • cos(D) = 0.3558 minus 0.0896 = 0.2662
  • D = 74.6 deg = 4,478 NM (great circle)
  • Rhumb line distance approx 4,900 NM
  • Great circle saves approximately 422 NM

8. Mercator Sailing and Rhumb Line Mathematics

The Mercator projection preserves angles (conformal), making rhumb lines appear as straight lines. This property makes Mercator charts the standard for coastal and offshore navigation. The mathematics of Mercator sailing handles the distortion inherent in mapping a sphere onto a flat surface.

Meridional Parts (M)

On a Mercator chart, latitude lines are spaced according to the logarithm of the secant — increasingly spread apart toward the poles. The "meridional parts" M(L) for latitude L is defined as:

M(L) = 7915.7 log[tan(45 + L/2)] (approximately, for a sphere)

More precisely (spheroid): M(L) = 7915.7 log[tan(45 + L/2)] minus 23.009 sin(L)

The factor 7915.7 scales meridional parts to match nautical miles at the equator. Difference in meridional parts: m = M(L2) minus M(L1).

Rhumb Line Course and Distance

For a rhumb line from (L1, Lo1) to (L2, Lo2):

  • l = L2 minus L1 (difference in latitude, in minutes)
  • DLo = Lo2 minus Lo1 (difference in longitude, in minutes)
  • m = M(L2) minus M(L1) (difference in meridional parts)
  • tan(C) = DLo / m (course angle C from N or S)
  • D = l / cos(C) if l is nonzero (distance in NM)
  • D = DLo times cos(L) / 60 if same latitude (departure)

Departure

Departure (p) converts difference in longitude to linear east-west distance at a given latitude. It is the projection of the rhumb line onto the east-west direction:

p (departure, in NM) = DLo (in minutes) times cos(L_mid)

Where L_mid is the middle latitude. For small latitude changes, L_mid = (L1 + L2) / 2. Departure bridges the gap between angle (difference in longitude) and distance on the surface.

Worked Mercator Sailing Example

From 30 N, 70 W to 42 N, 20 W:

  • l = 42 deg minus 30 deg = 12 deg = 720 NM (northward)
  • DLo = 70 deg minus 20 deg = 50 deg = 3000 (eastward, in minutes)
  • M(42) approx 2,729.9 (from tables)
  • M(30) approx 1,876.9 (from tables)
  • m = 2,729.9 minus 1,876.9 = 853.0
  • tan(C) = 3000 / 853.0 = 3.517
  • C = 74.1 deg (measured from North toward East)
  • True course Cn = N 74.1 E = 074.1 deg T
  • D = 720 / cos(74.1 deg) = 720 / 0.2740 = 2,628 NM

9. Current, Set, and Drift: Vector Triangle Mathematics

Ocean currents affect the path of a vessel. The navigator must account for set (direction toward which current flows) and drift (speed of current) to determine actual course made good and speed made good. This involves vector addition — the current triangle.

The Three Vectors

Water Track

Course steered (heading) and speed through the water. This is what the compass and knotmeter show.

Current Vector

Set (direction current flows toward) and drift (speed of current). From tide tables, pilot charts, or observation.

Ground Track

Course made good (CMG) and speed made good (SMG) over the ground. The vector sum of water track plus current.

Vector Addition: Finding CMG and SMG

Decompose each vector into north-south and east-west components, add components, then reconstitute:

Method:

  • Water track N component: Sw cos(Cw)
  • Water track E component: Sw sin(Cw)
  • Current N component: Sc cos(Set)
  • Current E component: Sc sin(Set)
  • Ground N = Sw cos(Cw) + Sc cos(Set)
  • Ground E = Sw sin(Cw) + Sc sin(Set)
  • SMG = sqrt(Ground N squared + Ground E squared)
  • CMG = arctan(Ground E / Ground N) [adjust quadrant]

Finding Current from Two Fixes

If the navigator knows the DR position (from steered course and speed) and the observed position (from a fix), the difference reveals the net current effect:

Current vector = (Observed fix position) minus (DR position)

Divide the distance between DR and fix by the elapsed time to get drift (knots). The direction from DR to fix is the set. This method requires accurate DR and a reliable fix.

The Cocked Hat and Running Fix

When three LOPs from simultaneous observations do not intersect at a single point, they form a triangle called a "cocked hat." The navigator plots the estimated position at the center of the cocked hat (or at the corner closest to danger). A running fix uses LOPs from observations taken at different times, advancing the earlier LOP along the course and speed made good to the time of the later observation.

Advancing an LOP:

Move every point on the LOP by the course and distance made good between the two observation times. The advanced LOP is parallel to the original. The intersection of the advanced LOP with the new LOP is the running fix position. For accurate results, use CMG and SMG (not heading and speed through water) when current is significant.

10. Tide Prediction: Harmonic Constituents and Equilibrium Theory

Tides result from the gravitational forces of the moon and sun on Earth's oceans, modified by Earth's rotation and the geometry of ocean basins. Modern tide prediction uses harmonic analysis — expressing the tidal height as a sum of sinusoidal constituents, each corresponding to a specific astronomical frequency.

The Harmonic Tidal Model

The tidal height h at time t is modeled as:

h(t) = Z0 + sum[n=1 to N] of H_n cos(omega_n t minus kappa_n)

Where: Z0 = mean water level above chart datum; H_n = amplitude of nth constituent (in meters or feet); omega_n = angular speed of nth constituent (degrees per hour); kappa_n = phase lag (Greenwich phase argument) of nth constituent; t = time in hours from a reference epoch.

Principal Tidal Constituents

NameSymbolPeriodDescription
Principal Lunar SemidiurnalM212.42 hrLargest constituent for most ports; two tides per lunar day
Principal Solar SemidiurnalS212.00 hrSolar contribution; beats with M2 to produce spring/neap cycle
Larger Lunar EllipticN212.66 hrVaries tide height due to elliptical moon orbit
Lunisolar DiurnalK123.93 hrDiurnal inequality; dominant in some Pacific locations
Principal Lunar DiurnalO125.82 hrLunar diurnal; creates once-daily tidal asymmetry
Principal Solar DiurnalP124.07 hrSolar diurnal component

Spring and Neap Tides: M2 and S2 Interaction

The dominant interaction is between M2 (period 12.42 hr) and S2 (period 12.00 hr). Their beat frequency is:

Beat period = 1 / (1/T_S2 minus 1/T_M2) = 1 / (1/12.00 minus 1/12.42) days

Beat period = 14.77 days (approximately two weeks)

At spring tides (new and full moon), M2 and S2 are in phase — their amplitudes add, giving highest highs and lowest lows. At neap tides (quarter moon), M2 and S2 are 90 degrees out of phase — their amplitudes partially cancel, giving smallest tidal range.

Equilibrium Theory and Secondary Corrections

The equilibrium theory of tides (Newton) assumes the ocean instantly adjusts to the tide-generating force. It predicts: at any point, the tidal force is proportional to (3 cos squared(z) minus 1), where z is the zenith angle of the moon. This gives two tidal bulges on opposite sides of Earth, each producing semidiurnal tides as Earth rotates.

Secondary Station Corrections:

For secondary stations (not the reference port), the National Ocean Service publishes time and height differences. Corrected high water time = reference high water time + time difference. Corrected height = reference height times height ratio + height difference. These corrections account for the local basin geometry that distorts the theoretical tidal signal.

Rule of Twelfths: Approximate Tide Height

Rise/Fall in Sixths of the Tidal Period:

  • 1st sixth of tidal period: tide rises/falls 1/12 of range
  • 2nd sixth: 2/12 of range (cumulative 3/12)
  • 3rd sixth: 3/12 of range (cumulative 6/12 — half range)
  • 4th sixth: 3/12 of range (cumulative 9/12)
  • 5th sixth: 2/12 of range (cumulative 11/12)
  • 6th sixth: 1/12 of range (cumulative 12/12 = full range)

This approximation assumes a sinusoidal tide and is accurate enough for most navigation purposes when detailed harmonic predictions are unavailable.

11. Dead Reckoning: DR Plot, EP, and Leeway

Dead reckoning (DR) is the process of estimating present position by advancing a known past position using course and speed. It is the backbone of navigation between fixes. The term likely derives from "ded. reckoning" (deduced reckoning) rather than any morbid association.

The DR Position

A DR position is computed solely from the last known fix plus course steered and speed through water (no correction for current or leeway). The estimated position (EP) additionally incorporates known or estimated current and leeway.

DR Position Calculation:

  • Distance made good: D = Speed times Time
  • Change in latitude: l = D cos(C) (in minutes NM = NM)
  • Departure: p = D sin(C)
  • Change in longitude: DLo = p / cos(L_mid) (in minutes)
  • New L2 = L1 + l (N positive, S negative)
  • New Lo2 = Lo1 + DLo (E positive, W negative)

Leeway

Leeway is the sideways drift caused by wind pressure on the vessel. It is expressed as an angle between the heading and the course through the water. Wind from the starboard side causes leeway to port (the actual course through water is to leeward of the heading).

Course through water = Heading + Leeway correction

Convention: if wind is from starboard, leeway is subtracted from heading (vessel moves to port of heading). If wind is from port, leeway is added. Typical leeway is 3 to 10 degrees depending on vessel type, wind speed, and sea state.

EP with Current Correction

Steps to Compute EP:

  1. Apply leeway to heading to get course through water.
  2. Compute DR position using course through water and speed through water.
  3. Convert current set and drift to north-south and east-west components over the elapsed time.
  4. Add current displacement to DR position to get EP.

12. Compass Mathematics: TVMDC and Error

A magnetic compass points to magnetic north, not true north. Two systematic errors separate compass readings from true bearings: variation (a geographic property of the location) and deviation (a property of the individual vessel). The navigator must convert among true, magnetic, and compass with precision.

TVMDC Mnemonic

True - Variation - Magnetic - Deviation - Compass

  • True (T): Directions referenced to geographic (true) north. Used for celestial navigation, charts, and plotting.
  • Variation (V): The angle between true north and magnetic north at a given location. Printed on chart compass roses with annual rate of change. East variation: magnetic north is east of true north.
  • Magnetic (M): Directions referenced to magnetic north. True bearing corrected for variation.
  • Deviation (D): The angle between magnetic north and the direction the compass needle points on a specific vessel on a specific heading. Caused by the vessel's own magnetic field. Changes with heading; tabulated on a deviation card.
  • Compass (C): What the compass actually reads. Affected by both variation and deviation.

Conversion Rules

From True to Compass (applying errors):

  • Magnetic = True minus Variation (E is positive, so E variation subtracts)
  • Compass = Magnetic minus Deviation (E deviation subtracts)

From Compass to True (removing errors):

  • Magnetic = Compass + Deviation (E deviation adds back)
  • True = Magnetic + Variation (E variation adds back)

Memory rule:

"East is Least" — going from True to Compass, east errors are subtracted (compass reads least). "West is Best" — west errors are added. Or: Compass to True, add east ("Can Dead Men Vote Twice At Elections" = Compass, Deviation, Magnetic, Variation, True, Add East).

Compass Error and Total Correction

Compass error (CE) is the algebraic sum of variation and deviation. It relates compass reading directly to true, bypassing magnetic:

  • CE = Variation + Deviation (algebraic sum, with E positive)
  • True = Compass + CE (if CE is east, T is greater than C)
  • Compass = True minus CE

Example: V = 15 W (minus 15), D = 3 E (plus 3). CE = -15 + 3 = -12 (12 W). True course = Compass course minus 12. If compass reads 090, true = 090 minus 12 = 078 T.

Deviation Card and Swinging Ship

The deviation card records compass deviation for each compass heading (usually every 15 or 30 degrees). It is created by "swinging ship" — comparing compass readings to known magnetic bearings (from ranges, GPS, or a pelorus) on all headings. Deviation varies with heading because the vessel's permanent and induced magnetism creates a complex field that interacts differently with the earth's field at each heading angle.

Bearing Conversion Worked Example

Given: Compass bearing to lighthouse = 215 deg C

  • Deviation on vessel heading = 4 deg W (so deviation = -4)
  • Variation = 12 deg E (so variation = +12)
  • Magnetic bearing = 215 + (-4) = 211 deg M
  • True bearing = 211 + (+12) = 223 deg T

Plot 223 deg T from the vessel's position through the lighthouse to establish an LOP on the chart.

13. Practice Problems with Full Solutions

Work through these problems to consolidate your understanding of ocean navigation mathematics. Click to reveal each solution.

Problem 1: Compute LHA Sun

At 14h 22m 36s GMT, the GHA of the sun is 30 degrees 14.2 minutes. The vessel is at DR longitude 74 degrees 31.8 W. Compute LHA Sun.

Solution:

  • GHA Sun at 14h = 30 deg 14.2
  • Assumed longitude W = 74 deg 14.2 (adjusted to give whole LHA)
  • LHA = GHA minus W longitude
  • LHA = 30 deg 14.2 minus 74 deg 14.2
  • LHA = -44 + 360 = 316 deg 00.0
  • (Or: choose assumed Lo = 74 deg 14.2 W; GHA 30 deg 14.2 minus 74 deg 14.2 = -44; add 360 = 316 deg)

LHA Sun = 316 degrees. Since LHA greater than 180, sun is east of observer's meridian (morning observation, sun rising toward meridian).

Problem 2: Compute Hc and Intercept

Assumed latitude = 32 N. Declination = 14 deg 24.0 S (contrary name). LHA = 316 degrees. Sextant altitude corrected to Ho = 22 deg 14.8. Compute Hc, Z, Zn, and the intercept.

Solution:

  • L = 32 N, d = -14 deg 24.0 (S, contrary), LHA = 316
  • sin(Hc) = sin(32)sin(-14.4) + cos(32)cos(14.4)cos(316)
  • sin(32) = 0.5299, sin(-14.4) = -0.2487
  • cos(32) = 0.8480, cos(14.4) = 0.9686, cos(316) = 0.7193
  • sin(Hc) = (0.5299)(-0.2487) + (0.8480)(0.9686)(0.7193)
  • sin(Hc) = -0.1318 + 0.5906 = 0.4588
  • Hc = arcsin(0.4588) = 27 deg 18.2
  • cos(Z) = [sin(-14.4) minus sin(32)(0.4588)] / [cos(32)(0.8885)]
  • cos(Z) = [-0.2487 minus 0.2432] / [0.7533] = -0.6531
  • Z = 130.8 deg
  • N hemisphere, LHA greater than 180 (body east): Zn = Z = 130.8 T
  • Intercept a = Ho minus Hc = 22 deg 14.8 minus 27 deg 18.2 = -5 deg 03.4
  • a = -303.4 minutes... error: recheck units
  • a = -5.4 NM minus 3.4... a = Ho minus Hc in minutes = -303.4 min
  • [Corrected: a = 22 deg 14.8 minus 27 deg 18.2 = -5 deg 03.4 = -303 NM — impossible]
  • Recalc: Ho = 22 deg 14.8, Hc = 27 deg 18.2, a = -5 deg 3.4
  • In minutes: a = -(5x60 + 3.4) = -303.4 NM would be invalid
  • Note: intercept a = Ho minus Hc = -(5 deg 03.4) - too large for real sight
  • In practice, intercepts over 30 NM indicate AP choice error; re-select AP.
  • a = -303 NM means chosen AP is far from true position

This example illustrates that a large intercept signals a poor assumed position. Select a new AP closer to the estimated DR, recompute, and the intercept will be small (under 30 NM for a reliable LOP).

Problem 3: Noon Latitude

At meridian passage, sextant altitude of sun lower limb = 71 deg 24.0. Index correction = +1.5. Height of eye = 9 feet (dip = -2.9). Refraction at this altitude = -0.3. SD = +16.0. Sun declination = 23 deg 01.0 N. Sun bore south. Find latitude.

Solution:

  • Hs = 71 deg 24.0
  • IC = +1.5
  • Dip = -2.9
  • App Alt = 71 deg 22.6
  • Refraction = -0.3
  • SD (lower limb) = +16.0
  • Ho = 71 deg 38.3
  • ZD = 90 deg 00.0 minus 71 deg 38.3 = 18 deg 21.7 S
  • (sun bore south, so ZD named south)
  • d = 23 deg 01.0 N (contrary to ZD south)
  • Lat = 23 deg 01.0 minus 18 deg 21.7 = 4 deg 39.3 N

Observer latitude = 4 deg 39.3 N. Because ZD (south) and declination (north) are contrary names, the smaller is subtracted from the larger and the result takes the name of the larger — in this case north, so 4 deg 39.3 N.

Problem 4: Great Circle Distance

Find the great circle distance from Newport, RI (41.5 N, 71.3 W) to the Lizard, England (49.9 N, 5.2 W). Also find the initial true course and compare to the rhumb line course.

Solution:

  • L1 = 41.5 N, Lo1 = 71.3 W
  • L2 = 49.9 N, Lo2 = 5.2 W
  • DLo = 71.3 minus 5.2 = 66.1 degrees
  • cos(D) = sin(41.5)sin(49.9) + cos(41.5)cos(49.9)cos(66.1)
  • sin(41.5) = 0.6613, sin(49.9) = 0.7649
  • cos(41.5) = 0.7501, cos(49.9) = 0.6441, cos(66.1) = 0.4051
  • cos(D) = (0.6613)(0.7649) + (0.7501)(0.6441)(0.4051)
  • cos(D) = 0.5058 + 0.1960 = 0.7018
  • D = arccos(0.7018) = 45.5 degrees
  • GC Distance = 45.5 times 60 = 2,730 NM
  • Initial course C: cos(C) = [sin(49.9) minus sin(41.5)cos(45.5)] / [cos(41.5)sin(45.5)]
  • cos(C) = [0.7649 minus (0.6613)(0.7009)] / [(0.7501)(0.7133)]
  • cos(C) = [0.7649 minus 0.4634] / 0.5350 = 0.5636
  • C = 55.7 deg from North toward East = 055.7 deg T initial course

Great circle distance 2,730 NM, initial course 055.7 T. The course increases (becomes more northerly then easterly) as the vessel arcs toward the vertex, then curves back toward the destination. Rhumb line course is approximately 072 T for about 3,000 NM — great circle saves about 270 NM.

Problem 5: TVMDC Compass Conversion

A vessel wants to steer a true course of 215 degrees T. Variation is 14 degrees W. Deviation on this heading (compass) is 3 degrees E. Find the compass course to steer. Also: a lighthouse bears 085 degrees on the compass; find the true bearing.

Solution:

Part A: True to Compass

  • True = 215 T
  • Variation = 14 W (subtract east, add west: add 14)
  • Magnetic = 215 + 14 = 229 M
  • Deviation = 3 E (subtract east: subtract 3)
  • Compass = 229 minus 3 = 226 C
  • Steer 226 on the compass.

Part B: Compass to True

  • Compass = 085 C
  • Deviation = 3 E (add east going to true: add 3)
  • Magnetic = 085 + 3 = 088 M
  • Variation = 14 W (add west going to true: add 14)
  • True = 088 + 14 = 102 T
  • Lighthouse bears 102 degrees True.

Mnemonic check: CE = V + D = -14 + 3 = -11 (11 W). True = Compass + CE. For Part B: 085 + (-11) = 074 T. Discrepancy arises because deviation changes with heading vs. bearing. Always use the deviation for the vessel's heading, not the bearing.

Problem 6: Current Triangle

A vessel steers 090 T at 8 knots through the water. A current sets 030 T at 2 knots. Find the course made good (CMG) and speed made good (SMG).

Solution:

  • Water N component: 8 cos(90) = 0.000
  • Water E component: 8 sin(90) = 8.000
  • Current N component: 2 cos(30) = 1.732
  • Current E component: 2 sin(30) = 1.000
  • Ground N = 0.000 + 1.732 = 1.732
  • Ground E = 8.000 + 1.000 = 9.000
  • SMG = sqrt(1.732 squared + 9.000 squared) = sqrt(3.000 + 81.000) = sqrt(84.0) = 9.17 kts
  • CMG = arctan(9.000 / 1.732) = arctan(5.196) = 79.1 deg from N
  • CMG = 079.1 deg T (NE quadrant, confirmed by both N and E components positive)

The northerly current pushes the vessel north of the intended east course. CMG = 079.1 T (about 11 degrees north of 090), SMG = 9.17 knots (slightly faster than water speed due to favorable current component).

Problem 7: Tide Height at Intermediate Time

High water at a port: 1430 local, height 14.8 ft. Low water: 2042 local, height 2.4 ft. Tidal range = 12.4 ft. Using the rule of twelfths, estimate the water depth at 1730 (3 hours after HW) at a spot where chart datum shows 6.0 ft.

Solution:

  • Tidal period HW to LW = 2042 minus 1430 = 6h 12m = 6.2 hours
  • One sixth of period = 6.2 / 6 = 1.033 hours per sixth
  • Time elapsed from HW to 1730 = 3h 00m = 3.0 hours
  • Number of sixths elapsed = 3.0 / 1.033 = 2.9 sixths (approx 3 sixths)
  • Using rule of twelfths (falling tide):
  • After 1st sixth: falls 1/12 of 12.4 = 1.03 ft
  • After 2nd sixth: falls additional 2/12 = 2.07 ft
  • After 3rd sixth: falls additional 3/12 = 3.10 ft
  • Total fall after 3 sixths = 1.03 + 2.07 + 3.10 = 6.20 ft
  • Height at 1730 = 14.8 minus 6.20 = 8.60 ft above datum
  • Total depth = 8.60 ft (tide) + 6.0 ft (charted depth) = 14.6 ft

Estimated water depth at 1730 = 14.6 ft. For a vessel drawing 12 ft, this gives only 2.6 ft of underkeel clearance — proceed with caution and verify with tide tables.

Quick Reference: Key Formulas

Celestial Navigation

  • sin(Hc) = sin(L)sin(d) + cos(L)cos(d)cos(LHA)
  • cos(Z) = [sin(d) - sin(L)sin(Hc)] / [cos(L)cos(Hc)]
  • Lat (noon) = ZD +/- d (same name add, contrary subtract)
  • Polaris: L = Ho - 1 + a0 + a1 + a2
  • GHA star = GHA Aries + SHA star
  • LHA = GHA +/- longitude (E+, W-)

Sailing Formulas

  • GC: cos(D) = sin(L1)sin(L2) + cos(L1)cos(L2)cos(DLo)
  • Rhumb: tan(C) = DLo(min) / m (merid. parts diff)
  • D = l / cos(C), p = D sin(C), DLo = p / cos(L)
  • SMG = sqrt((Sn+Cn)^2 + (Se+Ce)^2)
  • TVMDC: T = C + CE, CE = V + D (E pos)

Spherical Trig

  • cos(a) = cos(b)cos(c) + sin(b)sin(c)cos(A)
  • sin(a)/sin(A) = sin(b)/sin(B) = sin(c)/sin(C)
  • Four-parts: cot(a)sin(b) = cot(A)sin(C) + cos(b)cos(C)
  • Napier sine rule: sin(mid) = prod of sines of adj parts
  • Napier cosine rule: sin(mid) = prod of cosines of opp parts

Tides & DR

  • h(t) = Z0 + sum[H_n cos(omega_n t - kappa_n)]
  • Spring/Neap beat: ~14.77 days (M2 + S2)
  • Rule of 12ths: 1-2-3-3-2-1 twelfths per sixth
  • DR: l = D cos(C), p = D sin(C), DLo = p/cos(L)
  • Leeway: course through water = Hdg +/- leeway

Frequently Asked Questions

What is the law of cosines for spherical triangles and why does it matter in navigation?

The spherical law of cosines states: cos(a) = cos(b)cos(c) + sin(b)sin(c)cos(A). For the navigation triangle PZX this becomes sin(Hc) = sin(L)sin(d) + cos(L)cos(d)cos(LHA), the master formula of celestial navigation. Every computed altitude flows from it.

What are GHA, LHA, and SHA and how do they relate?

GHA is westward angle from Greenwich to the body; LHA is westward angle from the observer's meridian to the body (LHA = GHA minus west longitude or plus east longitude); SHA is westward angle from the vernal equinox to a star (GHA star = GHA Aries + SHA star).

How does the Marcq St. Hilaire intercept method work?

Compare observed altitude Ho to computed altitude Hc. Intercept a = Ho minus Hc in nautical miles. Positive: plot toward the body along azimuth Zn. Negative: plot away. The LOP is perpendicular to the azimuth at the intercept point.

How does great circle sailing differ from rhumb line sailing?

A great circle is the shortest path (constant-radius arc through the sphere's center) but requires changing course continuously. A rhumb line is constant course but longer. For transoceanic passages, great circle routes can save hundreds of nautical miles.

What are tidal harmonic constituents?

Tides are the sum of many sinusoidal oscillations (constituents), each at a frequency derived from astronomical cycles. M2 (lunar semidiurnal, 12.42 hr) dominates most ports. S2 (solar semidiurnal, 12.00 hr) beats with M2 to produce the 14.77-day spring-neap cycle.

What is the TVMDC sequence for compass conversion?

True, Variation, Magnetic, Deviation, Compass. Going True to Compass: subtract east errors, add west (east is least). Going Compass to True: add east, subtract west. CE = V + D; True = Compass + CE.

How do you find latitude from a noon sun sight?

Correct sextant altitude Hs to Ho. Zenith distance ZD = 90 deg minus Ho, named for the direction the sun bore. If ZD and declination have the same name, add for latitude; if contrary names, subtract the lesser from the greater and the result takes the name of the larger.