mrcal 2.5 release notes

New in mrcal 2.5

This is yet another mostly-maintenance release. Most of the implementation of some new big features is committed, but not yet completed and tested. These are thus lightly tested and documented, until they're done and finally appear in mrcal 3.0:

  • Cross-reprojection uncertainty, to be able to perform full calibrations with a splined model and without a chessboard
  • More general view of uncertainty. I want to support extrinsics and/or intrinsics uncertainty computation, in lots of scenarios
  • Implicit point solves, using the triangulation routines in the optimization cost function. Should produce much more efficient structure-from-motion solves. This is all the "triangulated-features" stuff. The _mrcal_triangulated_error() function is the core part of the cost function, and this is already demoed in test/test-sfm-triangulated-points.py

And other than that, there are lots and lots of smaller fixes and improvements.

General fixes

  • Poseutils: more careful handling of rotations near singularities. Rotations by ~ 180deg in particular weren't implemented properly, and had bugs that are now fixed.
  • Renamed "residuals" -> "measurements" to clarify the nomenclature. These are similar except for scaling and the "measurements" may also contain regularization.
    • residuals_point() -> measurements_point()
    • residuals_board() -> measurements_board()
    • show_residuals_....() functions now ask for a measurement x argument; the old residuals argument is still accepted for backwards compatibility
  • Added mrcal_traverse_sensor_links() in C and mrcal.traverse_sensor_links() in Python. These compute the best path through a graph of sensors, trying to maximize the common observations between each successive pair of sensors in the path. The general min-heap implementation at the core of this computation is available in heap.h
  • mrcal-calibrate-cameras and mrcal.compute_chessboard_corners() have a higher minimum points-per-chessboard-observation threshold (new Npoint_observations_per_board_min kwarg in the function). We now throw out any images where a chessboard detection has fewer than 2*min(gridn_width,gridn_height) points. This is irrelevant for mrgingham, which produces all-or-nothing detections, but other detectors could produce barely-adequate detections that could cause problems previously
  • mrcal.compute_chessboard_corners() has new image_path_prefix and image_directory kwargs to provide more options for interpreting paths in the given corners_cache_vnl. This is analogous to how other parts of mrcal work (for instance the mrcal-show-residuals-board-observation tool).
  • The mrcal-calibrate-cameras use that function, and now has --image-path-prefix and --image-directory. Useful to make --imagersize unnecessary and to have image overlays working with --explore. This are especially helpful if calibrating a system with different-resolution cameras. In that case --imagersize doesn't work, and finding the images on disk is required
  • Added analyses/validate-input-noise.py and analyses/validate-uncertainty.py and analyses/validate-noncentral.py tools. These are very useful to check the various assumptions made by the mrcal processing
  • The mrcal-convert-lensmodel tool has --all to write out all the models from a joint re-solve
  • Added new mrcal.model_resolution__deg_pixel() function and the mrcal-show-model-resolution tool
  • mrcal.show_distortion_off_pinhole_radial() works with all models, not just opencv ones
  • The image load/save functions no longer use libfreeimage, since that library is unmaintained. I'm using libstb for image-reading and libpng and libjpeg-turbo for image-writing. libstb can write images too, but there were several limitations that made it necessary to use the dedicated libraries. This means that currently I can write only .png and .jpg images. This can be extended if needed.
  • The mrcal-cull-corners tool has --cull-rad-off-center-board to throw out chessboard edges. Useful if we dont trust the chessboard shape
  • Added mrcal.cameramodel.optimization_inputs_reset() and mrcal.cameramodel.valid_intrinsics_region_reset() to unset these fields in a mrcal.cameramodel.
  • Added the analyses/mrcal-convert-lensmodel-from-kalibr-fov tool to fit a Kalibr "fov" model into something that mrcal supports. This isn't general-enough or tested-enough to be fully distributed yet, so run it from the mrcal source tree if you need it
  • Added mrcal_cameramodel_converter() "converter" function that can be used with O& conversions in PyArg_ParseTupleAndKeywords() calls. Can interpret either path strings or mrcal.cameramodel objects as mrcal_cameramodel_VOID_t C structures. Useful for writing Python extension modules for C code that uses mrcal_cameramodel_VOID_t.

  • Some fields in the mrcal.cameramodel type and the saved .cameramodel files and the optimization_inputs C and Python interfaces were renamed:

    \begin{aligned} &\mathrm{extrinsics\_rt\_fromref} & \rightarrow & \mathrm{rt\_cam\_ref} \\ &\mathrm{extrinsics\_Rt\_fromref} & \rightarrow & \mathrm{Rt\_cam\_ref} \\ &\mathrm{extrinsics\_rt\_toref} & \rightarrow & \mathrm{rt\_ref\_cam} \\ &\mathrm{extrinsics\_Rt\_toref} & \rightarrow & \mathrm{Rt\_ref\_cam} \\ &\mathrm{frames\_rt\_fromref} & \rightarrow & \mathrm{rt\_frame\_ref} \\ &\mathrm{frames\_Rt\_fromref} & \rightarrow & \mathrm{Rt\_frame\_ref} \\ &\mathrm{frames\_rt\_toref} & \rightarrow & \mathrm{rt\_ref\_frame} \\ &\mathrm{frames\_Rt\_toref} & \rightarrow & \mathrm{Rt\_ref\_frame} \\ &\end{aligned}

    This makes things more consistent. Compatibility logic is in place, so old code and data should keep working. New .cameramodel files write both the new and old fields, so old tools can read the new files.

  • Transform composition functions have new arguments: inverted0 and inverted1, to make it easier to use inverted transforms. The C macros mrcal_compose_Rt(), mrcal_compose_rt(), mrcal_compose_r() are unchanged, with new macros available to apply the inverses: ...._inverted0(), ...._inverted1(), ...._inverted01(). The corresponding mrcal_compose_..._full() C functions have new arguments; this breaks the API and ABI. The Python functions mrcal.compose_r() and mrcal.compose_rt() and mrcal.compose_R() have new kwargs with default values, so this API remains compatible.

Stereo

  • mrcal_rectified_system() and mrcal.rectified_system() fail if the rectified camera looks near the direction of the baseline vector. If az0_deg is being auto-detected, it will be shifted to avoid looking along the baseline
  • mrcal_rectified_system2() is a new flavor of mrcal_rectified_system() to provide a new az_edge_margin_deg argument, to set the closest the rectified view is allowed to get to \(\mathrm{az}=\pm 90^\circ\). The legacy function still exists, defaulting to az_edge_margin_deg=10. mrcal-stereo controls this via --az-edge-margin-deg
  • Similarly the Python mrcal.rectified_system() function now takes this az_edge_margin_deg argument also
  • mrcal-stereo: --equalization implies --force-grayscale
  • mrcal.stereo_range() has a better default for disparity_max to handle invalid disparities reliably in the common case where the disparity is a 16-bit signed integer

C API

  • Added C implementations of Procrustes fits: mrcal_align_procrustes_vectors_R01() and mrcal_align_procrustes_points_Rt01(). These are now the internal implementations of the Python mrcal.align_procrustes_vectors_R01() and mrcal.align_procrustes_points_Rt01() functions
  • Added mrcal_R_aligned_to_vector() C function. This is now the internal implementation of mrcal.R_aligned_to_vector()

  • Added simple math operation functions to the C API:

    • double mrcal_point3_inner(const mrcal_point3_t a, const mrcal_point3_t b)
    • double mrcal_point3_norm2(const mrcal_point3_t a)
    • double mrcal_point3_mag (const mrcal_point3_t a)
    • mrcal_point3_t mrcal_point3_add (const mrcal_point3_t a, const mrcal_point3_t b)
    • mrcal_point3_t mrcal_point3_sub (const mrcal_point3_t a, const mrcal_point3_t b)
    • mrcal_point3_t mrcal_point3_scale(const mrcal_point3_t a, const double s)
    • mrcal_point3_t mrcal_point3_cross(const mrcal_point3_t a, const mrcal_point3_t b)

    And similar for mrcal_point2_t, except there's no mrcal_point2_cross()

  • Added simple point and pose printing utilities:
    • mrcal_point2_print(p)
    • mrcal_point3_print(p)
    • mrcal_Rt_print(Rt)
    • mrcal_rt_print(rt)
  • mrcal_image_uint8_load() applies stretch equalization if given a 16-bit image. This is a reasonable default. If more specific processing is needed, call mrcal_image_uint16_load(). Applies to mrcal.load_image() also.
  • Added mrcal_image_void_t image type for generic functions that aren't meant to interface with any particular image type. The mrcal_image_anytype_load() function now uses that type.
  • All headers have C++ extern "C" wrappers for direct #include in C++ projects
  • Renamed mrcal_cameramodel_t -> mrcal_cameramodel_VOID_t to clarify that the specific lensmodel type is unknown here. The legacy alias mrcal_cameramodel_t is still available for backwards-compatibility
  • Added mrcal_intrinsics_XXX_t structures to represent camera intrinsics. These are exactly like mrcal_cameramodel_XXX_t but without the extrinsics

  • The C API can read models into a preallocated buffer intead of forcing allocation, as before. New functions: 

    bool mrcal_read_cameramodel_string_into(// out
                                        mrcal_cameramodel_VOID_t* model,
                                        // in,out
                                        int* Nintrinsics_max,
                                        // in
                                        const char* string,
                                        const int len);

bool mrcal_read_cameramodel_file_into  (// out
                                        mrcal_cameramodel_VOID_t* model,
                                        // in,out
                                        int* Nintrinsics_max,
                                        // in
                                        const char* filename);

Migration notes 2.4 -> 2.5

  • mrcal_compose_rt_full() and mrcal_compose_Rt_full() C functions have two new arguments: inverted0, inverted1. Most callers use the mrcal_..._compose() macros, so for them only the ABI has changed, and a rebuild is sufficient.
  • mrcal_compose_rt_full() C function can return dt01_dr1 and dt01_dt0. Most callers use the mrcal_..._compose() macros, so for them only the ABI has changed, and a rebuild is sufficient.
  • The C types mrcal_cameramodel_XXX_t no longer have a generic member mrcal_cameramodel_t m. If you need a generic alias of mrcal_cameramodel_XXX_t* model you now need to (mrcal_cameramodel_VOID_t*)model instead of &model->m

  • All the headers renamed to remove mrcal- from their name. Everything is intended to be included as 

    #include <mrcal/thing.h>

    So the mrcal- in the name was superfluous. If you were including either of:

    • mrcal-image.h
    • mrcal-types.h

    You should now include:

    • mrcal/image.h
    • mrcal/types.h
  • The mrcal_image_anytype_load() function uses the mrcal_image_void_t image type. The semantics are identical, but older code might need a cast or a type change to build.