Compass Calibration Example

Sample Points

Positive XYZ axes indicated with RGB colored dots
Dark Green samples used to calculate compass error

Compass Error Raw 0.0 Corrected 0.0

Full Vertical Scale +/- 10 deg. Horizontal: 0..360 deg
Red: Raw Observations, Green: Corrected

Controls for simulating magnetic errors and viewing the corrections

Offset %

Y/X Gain

Z/X Gain


Noise %




Spin Rate

The simulated magnetic field is taken to be 400 milli-Gauss. The faint gray circle in the left plot has a radius of 400. The dip angle from horizontal is 60 degrees. If calibrated on a horizontal turntable, the samples would trace a circle 30 degrees away from the negative Z azis. The dark green dots in the rotating sphere are a set of such samples.

Idealized configurations can be set by clicking the Hardiron, Softiron, or combination (Hard & Soft) buttons. The display will scale to fit all points. Hardiron exhibts a sinusoidal error with a period of 360 degrees. Softiron has TWO periods over 360. The 'Twisted' configuration applies stronger hard and soft perturbations. Additionally, the perturbations are NOT aligned with the instrument axes. The resulting ellipsoid is similar to the diagram here. In each case you can turn the noise off (Noise% = None) to get a clean view of the classic error shapes.

Corrected solutions are easy here because the perturbations have been idealized. Real-world asymmetry cannot be corrected by the least squares ellipsoid algorithm used here. Asymmetric effects are usually small. If you are aiming for fractional degree accuracy then you have to take them into account. Use the precision algorithm described here.

You may notice that with the 'twisted' ellipsoid sometimes you see the shape of a rugby ball that extends all the way across the screen from top to bottom. At other times, when nearly head-on, the dots perfectly fill the grey circular outline, more like a soccer ball. This is a warning about how you sample. If you restrict your samples to two planes you will find that your solution will not converge. See Degenerate Ellipsoids.

Source code is in javascript. You can view it by pressing F12 (FireFox, Chrome) and then pausing the script. It follows very closely the Python code in the least squares ellipsoid page. Helper math routines were taken from here . These were a Matrix inversion routine and an Eigenvalue/Eigenvector routine. I had to add a bit to the matrix inversion, because the eigenvalue routine sorted the vectors. Perfect data put zeros on the diagonal. Bad. So I added some pivoting.

I originally used a canned matrix library (Sylvester). But it didn't fit well. A constant problem was the use of indices that started with 1 (Matlab, Fortran) instead of zero (javascript, C). So, I wrote my own. Again. I considered it a learning process, since this web page is my first extended javascript application.

Offset simulates Hard iron perturbations. Offset along the Z axis does not affect the compass. The ratio of the X and Y components, which is used for heading calculations, remains the same when the origin is shifted vertically. Here the offset is at a 45 degree angle between the X and Y axes. Magnitude is a percentage of the radius

Y/X Z/X Gains simulate Soft iron perturbations. In this simulation, the X axis is kept constant (at 400 mG). A Y/X perturbation of 1.3 corresponds to an ellipsoid with X axis of 400 and Y axis of 520. The perturbation in the Z direction will not affect the compass (But see Twist below).

Twist simulates the orientation of the ellipsoid. In the real world, the perturbations on the magnetic sensors are NOT in general aligned with the instrument axes. Here we twist each axis by selected amount. When twisted, some of the Z azis term will be projected onto the XY plane. Therefore, a Z offset, or X/Z gain will now affect the heading calculations..

Noise is simulated by the addition of a vector with fixed magnitude but random direction to each sample point. The distribution is non-Gaussian (Raleigh) but has the desired effect. Noise is never zero in real life, so I set the default to a finite, but small value. Zero noise produces perfect errors which can be perfectly compensated. It verifies that the algorithms are working.

Samples A hundred samples are spread around the sphere. You can select how many are used in the calculations to correct the errors. You can see that 15 well places samples do a pretty good job. But this page shows what happens in 2D when the samples are bunched on one side. You can try it here in 3D with Coverage next.

Coverage You can bias the distribution of samples with Coverage percent. You can see that with partial coverage, low sample points, and mediocre noise you can get some large errors. Each change will resample the points and the noise, so you may get lucky. Play with it enough and you can see the general trend. If you set the noise to zero then the solutions can get solved exactly

Display You have your choice of seeing the raw observed data, or the data after the corrections are applied. Extremes in noise, low sample count and low coverage can produce quirky corrections. Quantative results are on the right. The numbers displayed at the top are zero mean RMS. Peak to peak errors are about three times these values (2√2).

©2022 Tom Judd
Carlsbad, CA