mrcal-show-projection-diff - Visualize the difference in projection between N models
$ mrcal-show-projection-diff before.cameramodel after.cameramodel
... a plot pops up showing how these two models differ in their projectionsThe operation of this tool is documented at http://mrcal.secretsauce.net/differencing.html
It is often useful to compare the projection behavior of two camera models. For instance, one may want to validate a calibration by comparing the results of two different chessboard dances. Or one may want to evaluate the stability of the intrinsics in response to mechanical or thermal stresses. This tool makes these comparisons, and produces a visualization of the results.
In the most common case we're given exactly 2 models to compare. We then display the projection difference as either a vector field or a heat map. If we're given more than 2 models, then a vector field isn't possible and we instead display as a heatmap the standard deviation of the differences between models 1..N and model0.
How are we computing the differences? The details are in the docstring of mrcal.projection_diff(). Broadly, we do this:
- grid the imager - unproject each point in the grid from one camera to produce a world point - apply a transformation we compute to match up the two camera geometries - reproject the transformed points to the other camera - look at the resulting pixel difference in the reprojection
When looking at multiple cameras, their lens intrinsics differ. Less obviously, the position and orientation of the camera coordinate system in respect to the physical camera housing differ also. These geometric uncertainties are baked into the intrinsics. So when we project "the same world point" into both cameras, we must apply a geometric transformation because we want to be comparing projections of world points (relative to the camera housing), not projections relative to the (floating) camera coordinate systems. This transformation is unknown, but we can estimate it in one of several ways:
- If we KNOW that there is no geometric difference between our cameras, and we thus should look at the intrinsics differences only, we assuming that implied_Rt10 = identity. Indicate this case by passing --radius 0
- Otherwise, we fit projections across the imager: the "right" transformation would result in apparent low projection differences in a wide area.
This fitted transformation is computed by implied_Rt10__from_unprojections(), and some details of its operation are significant:
- The imager area we use for the fit - Which world points we're looking at
In most practical usages, we would not expect a good fit everywhere in the imager: areas where no chessboards were observed will not fit well, for instance. From the point of view of the fit we perform, those ill-fitting areas should be treated as outliers, and they should NOT be a part of the solve. How do we specify the well-fitting area? The best way is to use the model uncertainties: these can be used to emphasize the confident regions of the imager. This is the default behavior. If uncertainties aren't available, or if we want a faster solve, pass --no-uncertainties. The well-fitting region can then be passed using --where and --radius to indicate the circle in the imager we care about.
If using uncertainties then we utilize all the data in the imager by default. if --no-uncertainties, then the defaults are to use a more reasonable circle of radius min(width,height)/6 at the center of the imager. Usually this is sufficiently correct, and we don't need to mess with it. If we aren't guided to the correct focus region, the implied-by-the-intrinsics solve will try to fit lots of outliers, which would result in an incorrect transformation, which in turn would produce overly-high reported diffs. A common case when this happens is if the chessboard observations used in the calibration were concentrated to the side of the image (off-center), no uncertainties were used, and --where was not pointed to that area.
Unlike the projection operation, the diff operation is NOT invariant under geometric scaling: if we look at the projection difference for two points at different locations along a single observation ray, there will be a variation in the observed diff. This is due to the geometric difference in the two cameras. If the models differed only in their intrinsics parameters, then this variation would not appear. Thus we need to know how far from the camera to look, and this is specified by --distance. By default we look out to infinity. If we care about the projection difference at some other distance, pass that here. Generally the most confident distance will be where the chessboards were observed at calibration time.
models Camera models to diff -h, --help show this help message and exit
--gridn GRIDN GRIDN How densely we should sample the imager. By default we
use a 60x40 grid
--distance DISTANCE By default we compute the implied transformation for
points infinitely far away from the camera. If we want
to look closer in, the desired observation distance
can be given in this argument. We can also fit
multiple distances at the same time by passing them
here in a comma-separated, whitespace-less list. If
multiple distances are given, we fit the implied-by-
the-intrinsics transformation using ALL the distances,
but we display the best-fitting difference for each
point. Only one distance is supported if
--vectorfield. Multiple distances are especially
useful if we have uncertainties: the most confident
distance will be found, and displayed.
--where WHERE WHERE Center of the region of interest for this diff. It is
usually impossible for the models to match everywhere,
but focusing on a particular area can work better. The
implied transformation will be fit to match as large
as possible an area centered on this argument. If
omitted, we will focus on the center of the imager
--radius RADIUS Radius of the region of interest. If ==0, we do NOT
fit an implied transformation at all. If omitted or
<0, we use a "reasonable" value: the whole imager if
we're using uncertainties, or min(width,height)/6 if
--no-uncertainties. To fit with data across the whole
imager in either case, pass in a very large radius
--observations If given, I show where the chessboard corners were
observed at calibration time. These should correspond
to the low-diff regions.
--valid-intrinsics-region
If given, I overlay the valid-intrinsics regions onto
the plot
--cbmax CBMAX Maximum range of the colorbar
--extratitle EXTRATITLE
Extra title string for the plot
--vectorfield Plot the diff as a vector field instead of as a heat
map. The vector field contains more information
(magnitude AND direction), but is less clear at a
glance
--vectorscale VECTORSCALE
If plotting a vectorfield, scale all the vectors by
this factor. Useful to improve legibility if the
vectors are too small to see
--directions If given, the plots are color-coded by the direction
of the error, instead of the magnitude
--no-uncertainties By default we use the uncertainties in the model to
weigh the fit. This will focus the fit on the
confident region in the models without --where or
--radius. The computation will run faster with --no-
uncertainties, but the default --where and --radius
may need to be adjusted
--hardcopy HARDCOPY Write the output to disk, instead of making an
interactive plot
--terminal TERMINAL gnuplotlib terminal. The default is good almost
always, so most people don't need this option
--set SET Extra 'set' directives to gnuplotlib. Can be given
multiple times
--unset UNSET Extra 'unset' directives to gnuplotlib. Can be given
multiple timeshttps://www.github.com/dkogan/mrcal
Dima Kogan, <dima@secretsauce.net>
Copyright (c) 2017-2020 California Institute of Technology ("Caltech"). U.S. Government sponsorship acknowledged. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0