mrcal - camera calibrations and more!

Note: this is a work-in-progress documentation for a work-in-progress upcoming mrcal 2.0 release. See the mrcal 1.0.1 documentation for the current stable version.

mrcal is a toolkit (originating at NASA/JPL) for working with lens models, camera geometry, images, projections, and the various related operations such as camera calibration. Any task that produces or consumes camera models can utilize this toolkit. It was originally built to generate the high-accuracy calibrations demanded by long-range stereo, so it provides facilities to analyze calibration accuracy and to propagate and report uncertainties.

Why mrcal?

In essense: because all other tools are terrible if you care about accuracy. They make basic questions like "how much data should I gather for a calibration?" and "how good is this calibration I just computed?" and "how different are these two models?" unanswerable.

Big assumptions and obliviousness to sources of error are commonplace. This is clearly seen from the core documentation on the subject. The wikipedia article on camera calibration only talks about fitting a pinhole model to lenses, even though no real lenses follow this model (telephoto lenses do somewhat; wider lenses don't at all). OpenCV is a dominant library in this area; its calibration tutorial cheerfully says that

Re-projection error gives a good estimation of just how exact the found
parameters are. The closer the re-projection error is to zero, the more accurate
the parameters we found are.

This statement is trivially proven false: throw away most of your calibration data, and your reprojection error becomes very low. But we can all agree that a calibration computed from less data is actually worse. Right?

All the various assumptions and hacks in the existing tooling are fine as long as you don't need a whole lot of accuracy out of your models.

mrcal is meant to address these shortcomings. This toolkit tries hard to allow the user to produce calibrations that are as good as possible. It provides lots of visualization capabilities to evaluate various properties of a model solution. And it provides several powerful analysis methods, such as quantification of projection and triangulation uncertainty. And model differencing. And functions are provided to avoid making a pinhole-camera assumption where it is not appropriate, such as with wide-angle stereo.

Documentation index

Please see a tour of mrcal for a high-level overview of the capabilities of the toolkit.

First, the tools should be built or installed.

Before using the tools, it is helpful to read about the terminology and conventions used in the sources and documentation.

At the core of a calibration routine is an optimization problem. Details about its formulation are useful to be able to interpret the results.

A lens can be represented by any of a number of lens models.

A how-to-calibrate-some-cameras page describes details about how to accomplish this very common task.

After running a calibration, the camera models are written to files on disk.

We can then use these files with a number of command-line tools. In particular, we can compare the projection behaviors of different models. And we can compute the projection uncertainties of a model.

If we need to do something more than what the pre-made tools can do, there're two sets programmatic interfaces available:

Citing

To cite this work in a publication, use this bibtex stanza:

@misc{mrcal,
  author = "Dima Kogan",
  title = "mrcal",
  howpublished = "\url{http://mrcal.secretsauce.net}",
}

Dev communication

For now let's use the github issue tracker for bug reporting and for communication in general. At some point I will probably set up a mailing list as well.

Author

Dima Kogan dima@secretsauce.net

License and copyright

Copyright (c) 2017-2020 California Institute of Technology ("Caltech"). U.S. Government sponsorship acknowledged. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

There's some included external code in the sources, with their own copyrights and licenses.