mrcal 2.5.1 release notes

A number of new tools have been written that are very useful, but are still a bit rough, and need to be finished

• A new analyses/mrcal-pick-features tool is available to interactively pick matching features from a stereo pair's images, and to compute the relative extrinsics off them. This is very useful for a narrow application, and the optimal solver details are unclear, so this is not-yet released today. The assisted gui feature-picker and solver will be used for something eventually, but today it's there, waiting to be finished.
• New validation tools analyses/validate-input-noise.py and analyses/validate-uncertainty.py and analyses/validate-noncentral.py are useful to check the various assumptions made by the mrcal processing. Used for sanity-checking a solve and for algorithm development. Documented somewhat here and here.
• The mrcal library has now been used in multiple projects to align non-camera sensors with great success:
  • A geometry-only routine to align a set of cameras and a set of LIDARs is available in the camera-lidar-calibration repo. This aligns the geometry only and works for a LIDAR-only camera-less system as well. This tool assumes the camera intrinsics are perfect and it does uncertainty propagation of the output. It is decently well-tested and works well. Documentation and interfaces are still unideal, but improvements are welcome!
  • A routine to align a set of cameras and an IMU is available in analyses/calibrate-camera-imu.py. This is also geometry-only and also trusts the intrinsics 100%. It's fairly simplistic, but highly usable and on-par with existing methods. If needed a full kalibr-style sensor-bias-fitting thing will probably be implemented eventually. Improvements welcome!

• Thanks to Alan Everett, mrcal and the adjacent tooling now all build with macos. If anybody wants to see support for more OSs and distribution mechanisms (homebrew? fedora? pip?), and is willing to help with the implementation and the maintenance, please get in touch with me.
• Poseutils: more careful handling of rotations near singularities. Rotations by ~ 180deg in particular weren't implemented properly, and had bugs that are now fixed.
• Renamed "residuals" -> "measurements" to clarify the nomenclature. These are similar except for scaling and the "measurements" may also contain regularization.
  • residuals_point() -> measurements_point()
  • residuals_board() -> measurements_board()
  • show_residuals_....() functions now ask for a measurement x argument; the old residuals argument is still accepted for backwards compatibility
• Added mrcal_traverse_sensor_links() in C and mrcal.traverse_sensor_links() in Python. These compute the best path through a graph of sensors, trying to maximize the common observations between each successive pair of sensors in the path. This is a part of the seeding algorithm, and is now exposed for other usages. The general min-heap implementation at the core of this computation is available in mrcal/heap.h
• mrcal-calibrate-cameras and mrcal.compute_chessboard_corners() have a higher minimum points-per-chessboard-observation threshold (new Npoint_observations_per_board_min kwarg in the function). We now throw out any images where a chessboard detection has fewer than 2*min(gridn_width,gridn_height) points. This is irrelevant for mrgingham, which produces all-or-nothing detections, but other detectors could produce barely-adequate detections that could cause problems previously
• mrcal.compute_chessboard_corners() has new image_path_prefix and image_directory kwargs to provide more options for interpreting paths in the given corners_cache_vnl. This is analogous to how other parts of mrcal work (for instance the mrcal-show-residuals-board-observation tool).
• The mrcal-calibrate-cameras use that function, and now has --image-path-prefix and --image-directory. Useful to make --imagersize unnecessary and to have image overlays working with --explore. This is especially helpful if calibrating a system with different-resolution cameras. In that case --imagersize doesn't work, and finding the images on disk is required
• The mrcal-convert-lensmodel tool has --all to write out all the models from a joint re-solve
• Added new mrcal.model_resolution__deg_pixel() function and the mrcal-show-model-resolution tool
• mrcal.show_distortion_off_pinhole_radial() works with all models, not just opencv ones
• The image load/save functions no longer use libfreeimage, since that library is unmaintained. I'm using libstb for image-reading and libpng and libjpeg-turbo for image-writing. libstb can write images too, but there were several limitations that made it necessary to use the dedicated libraries. This means that currently I can write only .png and .jpg images. This can be extended if needed.
• The mrcal-cull-corners tool has --cull-rad-off-center-board to throw out chessboard edges. Useful if we dont trust the chessboard shape
• Added mrcal.cameramodel.optimization_inputs_reset() and mrcal.cameramodel.valid_intrinsics_region_reset() to unset these fields in a mrcal.cameramodel.
• Added the analyses/mrcal-convert-lensmodel-from-kalibr-fov tool to fit a Kalibr "fov" model into something that mrcal supports. This isn't general-enough or tested-enough to be fully distributed yet, so run it from the mrcal source tree if you need it
• Added mrcal_cameramodel_converter() "converter" function that can be used with O& conversions in PyArg_ParseTupleAndKeywords() calls. Can interpret either path strings or mrcal.cameramodel objects as mrcal_cameramodel_VOID_t C structures. Useful for writing Python extension modules for C code that uses mrcal_cameramodel_VOID_t.

• Some fields in the mrcal.cameramodel type and the saved .cameramodel files and the optimization_inputs C and Python interfaces were renamed:

  \begin{aligned} \mathrm{extrinsics\_rt\_fromref} & \to \mathrm{rt\_cam\_ref} \\ \mathrm{extrinsics\_Rt\_fromref} & \to \mathrm{Rt\_cam\_ref} \\ \mathrm{extrinsics\_rt\_toref} & \to \mathrm{rt\_ref\_cam} \\ \mathrm{extrinsics\_Rt\_toref} & \to \mathrm{Rt\_ref\_cam} \\ \mathrm{frames\_rt\_fromref} & \to \mathrm{rt\_frame\_ref} \\ \mathrm{frames\_Rt\_fromref} & \to \mathrm{Rt\_frame\_ref} \\ \mathrm{frames\_rt\_toref} & \to \mathrm{rt\_ref\_frame} \\ \mathrm{frames\_Rt\_toref} & \to \mathrm{Rt\_ref\_frame} \\ \end{aligned}

  This makes things more consistent. Compatibility logic is in place, so old code and data should keep working. New .cameramodel files write both the new and old fields, so old tools can read the new files.

• Transform composition functions have new arguments: inverted0 and inverted1, to make it easier to use inverted transforms. The C macros mrcal_compose_Rt(), mrcal_compose_rt(), mrcal_compose_r() are unchanged, with new macros available to apply the inverses: ...._inverted0(), ...._inverted1(), ...._inverted01(). The corresponding mrcal_compose_..._full() C functions have new arguments; this breaks the API and ABI. The Python functions mrcal.compose_r() and mrcal.compose_rt() and mrcal.compose_R() have new kwargs with default values, so this API remains compatible.

• mrcal_rectified_system2() is a new flavor of mrcal_rectified_system() to provide a new az_edge_margin_deg argument, to set the closest the rectified view is allowed to get to \(\mathrm{az}=\pm 90^\circ\). The legacy function still exists, defaulting to az_edge_margin_deg=10. mrcal-stereo controls this via --az-edge-margin-deg. If az0_deg is being auto-detected, it will be shifted to avoid looking along the baseline
• Similarly the Python mrcal.rectified_system() function now takes this az_edge_margin_deg argument also
• The C function mrcal_rectified_system2() now supports pinhole rectification, something only the Python mrcal.rectified_system() call did previously. This is strictly worse than the LENSMODEL_LATLON rectification used by default, but is good to have for testing, since this is the traditional scheme used by every other tool.
• mrcal-stereo: --equalization implies --force-grayscale
• mrcal.stereo_range() has a better default for disparity_max to handle invalid disparities reliably in the common case where the disparity is a 16-bit signed integer

• Added C implementations of Procrustes fits: mrcal_align_procrustes_vectors_R01() and mrcal_align_procrustes_points_Rt01(). These are now the internal implementations of the Python mrcal.align_procrustes_vectors_R01() and mrcal.align_procrustes_points_Rt01() functions
• Added mrcal_R_aligned_to_vector() C function. This is now the internal implementation of mrcal.R_aligned_to_vector()

• Added simple math operation functions to the C API:

  • double mrcal_point3_inner(const mrcal_point3_t a, const mrcal_point3_t b)
  • double mrcal_point3_norm2(const mrcal_point3_t a)
  • double mrcal_point3_mag (const mrcal_point3_t a)
  • mrcal_point3_t mrcal_point3_add (const mrcal_point3_t a, const mrcal_point3_t b)
  • mrcal_point3_t mrcal_point3_sub (const mrcal_point3_t a, const mrcal_point3_t b)
  • mrcal_point3_t mrcal_point3_scale(const mrcal_point3_t a, const double s)
  • mrcal_point3_t mrcal_point3_cross(const mrcal_point3_t a, const mrcal_point3_t b)

  And similar for mrcal_point2_t, except there's no mrcal_point2_cross()

• Added simple point and pose printing utilities:
  • mrcal_point2_print(p)
  • mrcal_point3_print(p)
  • mrcal_Rt_print(Rt)
  • mrcal_rt_print(rt)
• mrcal_image_uint8_load() applies stretch equalization if given a 16-bit image. This is a reasonable default. If more specific processing is needed, call mrcal_image_uint16_load(). Applies to mrcal.load_image() also.
• Added mrcal_image_void_t image type for generic functions that aren't meant to interface with any particular image type. The mrcal_image_anytype_load() function now uses that type.
• All headers have C++ extern "C" wrappers for direct #include in C++ projects
• Renamed mrcal_cameramodel_t -> mrcal_cameramodel_VOID_t to clarify that the specific lensmodel type is unknown here. The legacy alias mrcal_cameramodel_t is still available for backwards-compatibility
• Added mrcal_intrinsics_XXX_t structures to represent camera intrinsics. These are exactly like mrcal_cameramodel_XXX_t but without the extrinsics

• The C API can read models into a preallocated buffer intead of forcing allocation, as before. New functions:

  bool mrcal_read_cameramodel_string_into(// out
                                          mrcal_cameramodel_VOID_t* model,
                                          // in,out
                                          int* Nintrinsics_max,
                                          // in
                                          const char* string,
                                          const int len);
  
  bool mrcal_read_cameramodel_file_into  (// out
                                          mrcal_cameramodel_VOID_t* model,
                                          // in,out
                                          int* Nintrinsics_max,
                                          // in
                                          const char* filename);