mrcal-show-projection-diff - Visualize the difference in projection between N models


  $ mrcal-show-projection-diff before.cameramodel after.cameramodel
  ... a plot pops up showing how these two models differ in their projections


The operation of this tool is documented at

It is often useful to compare the projection behavior of two camera models. For instance, one may want to validate a calibration by comparing the results of two different chessboard dances. Or one may want to evaluate the stability of the intrinsics in response to mechanical or thermal stresses. This tool makes these comparisons, and produces a visualization of the results.

In the most common case we're given exactly 2 models to compare. We then display the projection difference as either a vector field or a heat map. If we're given more than 2 models, then a vector field isn't possible and we instead display as a heatmap the standard deviation of the differences between models 1..N and model0.

How are we computing the differences? The details are in the docstring of mrcal.projection_diff(). Broadly, we do this:

- grid the imager - unproject each point in the grid from one camera to produce a world point - apply a transformation we compute to match up the two camera geometries - reproject the transformed points to the other camera - look at the resulting pixel difference in the reprojection

When looking at multiple cameras, their lens intrinsics differ. Less obviously, the position and orientation of the camera coordinate system in respect to the physical camera housing differ also. These geometric uncertainties are baked into the intrinsics. So when we project "the same world point" into both cameras, we must apply a geometric transformation because we want to be comparing projections of world points (relative to the camera housing), not projections relative to the (floating) camera coordinate systems. This transformation is unknown, but we can estimate it in one of several ways:

- If we KNOW that there is no geometric difference between our cameras, and we thus should look at the intrinsics differences only, we assuming that implied_Rt10 = identity. Indicate this case by passing --radius 0

- Otherwise, we fit projections across the imager: the "right" transformation would result in apparent low projection differences in a wide area.

This fitted transformation is computed by implied_Rt10__from_unprojections(), and some details of its operation are significant:

- The imager area we use for the fit - Which world points we're looking at

In most practical usages, we would not expect a good fit everywhere in the imager: areas where no chessboards were observed will not fit well, for instance. From the point of view of the fit we perform, those ill-fitting areas should be treated as outliers, and they should NOT be a part of the solve. How do we specify the well-fitting area? The best way is to use the model uncertainties: these can be used to emphasize the confident regions of the imager. This is the default behavior. If uncertainties aren't available, or if we want a faster solve, pass --no-uncertainties. The well-fitting region can then be passed using --where and --radius to indicate the circle in the imager we care about.

If using uncertainties then we utilize all the data in the imager by default. if --no-uncertainties, then the defaults are to use a more reasonable circle of radius min(width,height)/6 at the center of the imager. Usually this is sufficiently correct, and we don't need to mess with it. If we aren't guided to the correct focus region, the implied-by-the-intrinsics solve will try to fit lots of outliers, which would result in an incorrect transformation, which in turn would produce overly-high reported diffs. A common case when this happens is if the chessboard observations used in the calibration were concentrated to the side of the image (off-center), no uncertainties were used, and --where was not pointed to that area.

Unlike the projection operation, the diff operation is NOT invariant under geometric scaling: if we look at the projection difference for two points at different locations along a single observation ray, there will be a variation in the observed diff. This is due to the geometric difference in the two cameras. If the models differed only in their intrinsics parameters, then this variation would not appear. Thus we need to know how far from the camera to look, and this is specified by --distance. By default we look out to infinity. If we care about the projection difference at some other distance, pass that here. Generally the most confident distance will be where the chessboards were observed at calibration time.



  models                Camera models to diff


  -h, --help            show this help message and exit
  --gridn GRIDN GRIDN   How densely we should sample the imager. By default we
                        use a 60x40 grid
  --distance DISTANCE   By default we compute the implied transformation for
                        points infinitely far away from the camera. If we want
                        to look closer in, the desired observation distance
                        can be given in this argument. We can also fit
                        multiple distances at the same time by passing them
                        here in a comma-separated, whitespace-less list. If
                        multiple distances are given, we fit the implied-by-
                        the-intrinsics transformation using ALL the distances,
                        but we display the best-fitting difference for each
                        point. Only one distance is supported if
                        --vectorfield. Multiple distances are especially
                        useful if we have uncertainties: the most confident
                        distance will be found, and displayed.
  --where WHERE WHERE   Center of the region of interest for this diff. It is
                        usually impossible for the models to match everywhere,
                        but focusing on a particular area can work better. The
                        implied transformation will be fit to match as large
                        as possible an area centered on this argument. If
                        omitted, we will focus on the center of the imager
  --radius RADIUS       Radius of the region of interest. If ==0, we do NOT
                        fit an implied transformation at all. If omitted or
                        <0, we use a "reasonable" value: the whole imager if
                        we're using uncertainties, or min(width,height)/6 if
                        --no-uncertainties. To fit with data across the whole
                        imager in either case, pass in a very large radius
  --observations        If given, I show where the chessboard corners were
                        observed at calibration time. These should correspond
                        to the low-diff regions.
                        If given, I overlay the valid-intrinsics regions onto
                        the plot
  --cbmax CBMAX         Maximum range of the colorbar
  --extratitle EXTRATITLE
                        Extra title string for the plot
  --vectorfield         Plot the diff as a vector field instead of as a heat
                        map. The vector field contains more information
                        (magnitude AND direction), but is less clear at a
  --vectorscale VECTORSCALE
                        If plotting a vectorfield, scale all the vectors by
                        this factor. Useful to improve legibility if the
                        vectors are too small to see
  --directions          If given, the plots are color-coded by the direction
                        of the error, instead of the magnitude
  --no-uncertainties    By default we use the uncertainties in the model to
                        weigh the fit. This will focus the fit on the
                        confident region in the models without --where or
                        --radius. The computation will run faster with --no-
                        uncertainties, but the default --where and --radius
                        may need to be adjusted
  --hardcopy HARDCOPY   Write the output to disk, instead of making an
                        interactive plot
  --terminal TERMINAL   gnuplotlib terminal. The default is good almost
                        always, so most people don't need this option
  --set SET             Extra 'set' directives to gnuplotlib. Can be given
                        multiple times
  --unset UNSET         Extra 'unset' directives to gnuplotlib. Can be given
                        multiple times



Dima Kogan, <>


Copyright (c) 2017-2020 California Institute of Technology ("Caltech"). U.S. Government sponsorship acknowledged. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); You may obtain a copy of the License at