# mrcal Python API

## Table of Contents

- Geometry
- Lens models
- Projections
- Visualization
- Calibration helpers
- Image transforms
- Model analysis
- Stereo, triangulation, feature-matching
- Synthetic data
- CHOLMOD interface
- Layout of the measurement and state vectors
- State packing
- Optimization
- Camera model reading/writing
- Image reading/writing
- Miscellaneous utilities

A Python API is available to go beyond what the pre-made commandline tools can do. These tools themselves are written using the Python API, so their sources are are a good guide.

All the Python functions have complete docstrings, so the `pydoc3`

tool is
effective at displaying the relevant documentation. For convenience, all the
docstrings have been extracted and formatted into the Python API reference.

The available functions, by category:

## Geometry

### Primitives

Functions to manipulate rotations and poses.

`mrcal.identity_R()`

: Return an identity rotation matrix`mrcal.identity_r()`

: Return an identity Rodrigues rotation`mrcal.identity_Rt()`

: Return an identity Rt transformation`mrcal.identity_rt()`

: Return an identity rt transformation`mrcal.r_from_R()`

: Compute a Rodrigues vector from a rotation matrix`mrcal.R_from_r()`

: Compute a rotation matrix from a Rodrigues vector`mrcal.rt_from_Rt()`

: Compute an rt transformation from a Rt transformation`mrcal.Rt_from_rt()`

: Compute an Rt transformation from a rt transformation`mrcal.invert_R()`

: Invert an (3,3) matrix rotation. This is a transpose`mrcal.invert_Rt()`

: Invert an Rt transformation`mrcal.invert_rt()`

: Invert an rt transformation`mrcal.compose_r()`

: Compose Rodrigues rotations`mrcal.compose_Rt()`

: Compose Rt transformations`mrcal.compose_rt()`

: Compose rt transformations`mrcal.rotate_point_r()`

: Rotate point(s) using a Rodrigues vector`mrcal.rotate_point_R()`

: Rotate point(s) using a rotation matrix`mrcal.transform_point_rt()`

: Transform point(s) using an rt transformation`mrcal.transform_point_Rt()`

: Transform point(s) using an Rt transformation`mrcal.R_from_quat()`

: Convert a rotation defined as a unit quaternion rotation to a rotation matrix`mrcal.quat_from_R()`

: Convert a rotation defined as a rotation matrix to a unit quaternion`mrcal.Rt_from_qt()`

: Compute an Rt transformation from a qt transformation`mrcal.qt_from_Rt()`

: Compute a qt transformation from an Rt transformation

### Alignment

`mrcal.align_procrustes_points_Rt01()`

: Compute a rigid transformation to align two point clouds`mrcal.align_procrustes_vectors_R01()`

: Compute a rotation to align two sets of direction vectors`mrcal.R_aligned_to_vector()`

: Compute a non-unique rotation to map a given vector to [0,0,1]

## Lens models

Routines for manipulating lens models.

`mrcal.supported_lensmodels()`

: Returns a tuple of strings for the various lens models we support`mrcal.lensmodel_num_params()`

: Get the number of lens parameters for a particular model type`mrcal.lensmodel_metadata_and_config()`

: Returns meta-information about a model`mrcal.knots_for_splined_models()`

: Return a tuple of locations of x and y spline knots

## Projections

`mrcal.project()`

: Projects a set of 3D camera-frame points to the imager`mrcal.unproject()`

: Unprojects pixel coordinates to observation vectors`mrcal.project_pinhole()`

: Projects a set of 3D camera-frame points using a pinhole model`mrcal.unproject_pinhole()`

: Unprojects a set of 2D pixel coordinates using a pinhole model`mrcal.project_stereographic()`

: Projects a set of 3D camera-frame points using a stereographic map`mrcal.unproject_stereographic()`

: Unprojects a set of 2D pixel coordinates using a stereographic map`mrcal.project_latlon()`

: Projects a set of 3D camera-frame points using a transverse equirectangular projection (used primarily for wide-angle stereo)`mrcal.unproject_latlon()`

: Unprojects a set of 2D pixel coordinates using a transverse equirectangular projection (used primarily for wide-angle stereo)`mrcal.project_lonlat()`

: Projects a set of 3D camera-frame points using an equirectangular projection (useful for representing 360-degree panoramas)`mrcal.unproject_lonlat()`

: Unprojects a set of 2D pixel coordinates using an equirectangular projection (useful for representing 360-degree panoramas)

## Visualization

### Driver routines

These are all backends for the corresponding commandline tools.

`mrcal.show_geometry()`

: Visualize the world represented by some number of cameras and chessboards and points. For instance, result of a calibration run.`mrcal.show_projection_diff()`

: Visualize the difference in projection between N models`mrcal.show_projection_uncertainty()`

: Visualize the uncertainty in camera projection`mrcal.show_projection_uncertainty_vs_distance()`

: Visualize the uncertainty in camera projection along one observation ray`mrcal.show_distortion_off_pinhole()`

: Visualize a lens's deviation from a pinhole projection: examine the difference across the imager`mrcal.show_distortion_off_pinhole_radial()`

: Visualize a lens's deviation from a pinhole projection: examine the radial distortion curve`mrcal.show_valid_intrinsics_region()`

: Visualize a model's valid-intrinsics region`mrcal.show_splined_model_correction()`

: Visualize the projections corrections represented by a splined model`mrcal.show_residuals_board_observation()`

: Visualize calibration residuals for a single observation`mrcal.show_residuals_histogram()`

: Visualize the distribution of the optimized residuals`mrcal.show_residuals_vectorfield()`

: Visualize the optimized residuals as a vector field`mrcal.show_residuals_magnitudes()`

: Visualize the optimized residual magnitudes as color-coded points`mrcal.show_residuals_directions()`

: Visualize the optimized residual directions as color-coded points`mrcal.show_residuals_regional()`

: Visualize the optimized residuals, broken up by region

### Utilities

`mrcal.annotate_image__valid_intrinsics_region()`

: Annotate an image with a model's valid-intrinsics region`mrcal.imagergrid_using()`

: Get a 'using' gnuplotlib expression for imager colormap plots`mrcal.fitted_gaussian_equation()`

: Get an 'equation' gnuplotlib expression for a gaussian curve fitting some data`mrcal.sample_imager()`

: Returns regularly-sampled, gridded pixels coordinates across the imager`mrcal.sample_imager_unproject()`

: Reports 3D observation vectors that regularly sample the imager`mrcal.plotoptions_state_boundaries()`

: Return the 'set' plot options for gnuplotlib to show the state boundaries`mrcal.plotoptions_measurement_boundaries()`

: Return the 'set' plot options for gnuplotlib to show the measurement boundaries`mrcal.apply_color_map()`

: Color-code an array`mrcal.write_point_cloud_as_ply()`

: Write a point cloud to a .ply file

## Calibration helpers

These are used by routines implementing a camera calibration system. Most users
will run the `mrcal-calibrate-cameras`

tool instead of calling these.

`mrcal.compute_chessboard_corners()`

: Compute or read the chessboard observations, and return them in a usable form`mrcal.estimate_monocular_calobject_poses_Rt_tocam()`

: Estimate camera-referenced poses of the calibration object from monocular views`mrcal.estimate_joint_frame_poses()`

: Estimate world-referenced poses of the calibration object`mrcal.seed_stereographic()`

: Compute an optimization seed for a camera calibration

## Image transforms

`mrcal.scale_focal__best_pinhole_fit()`

: Compute the optimal focal-length scale for reprojection to a pinhole lens`mrcal.pinhole_model_for_reprojection()`

: Generate a pinhole model suitable for reprojecting an image`mrcal.image_transformation_map()`

: Compute a reprojection map between two models`mrcal.transform_image()`

: Transforms a given image using a given map

## Model analysis

`mrcal.implied_Rt10__from_unprojections()`

: Compute the implied-by-the-intrinsics transformation to fit two cameras' projections`mrcal.worst_direction_stdev()`

: Compute the worst-direction standard deviation from a 2x2 covariance matrix`mrcal.projection_uncertainty()`

: Compute the projection uncertainty of a camera-referenced point`mrcal.projection_diff()`

: Compute the difference in projection between N models`mrcal.is_within_valid_intrinsics_region()`

: Which of the pixel coordinates fall within the valid-intrinsics region?

## Stereo, triangulation, feature-matching

`mrcal.rectified_system()`

: Generate rectified stereo models, which we can use to rectify images for stereo matching`mrcal.rectified_resolution()`

: Compute the resolution to be used for the rectified system. Usually this is called by`mrcal.rectified_system()`

, but it's available standalone as well`mrcal.rectification_maps()`

: Construct pixel mappings to transform captured images into rectified images`mrcal.stereo_range()`

: Compute ranges from observed disparities`mrcal.stereo_unproject()`

: Compute a point cloud from observed disparities`mrcal.match_feature()`

: Find a pixel correspondence in a pair of images`mrcal.triangulate()`

: Triangulate N points with uncertainty propagation. This is a higher-level function than the other`mrcal.triangulate_...()`

routines`mrcal.triangulate_geometric()`

: Simple geometric triangulation`mrcal.triangulate_lindstrom()`

: Triangulation minimizing the 2-norm of pinhole reprojection errors`mrcal.triangulate_leecivera_l1()`

: Triangulation minimizing the L1-norm of angle differences`mrcal.triangulate_leecivera_linf()`

: Triangulation minimizing the infinity-norm of angle differences`mrcal.triangulate_leecivera_mid2()`

: Triangulation using Lee and Civera's alternative midpoint method. Recommended.`mrcal.triangulate_leecivera_wmid2()`

: Triangulation using Lee and Civera's inverse-depth-weighted alternative midpoint method. Recommended in favor of`mrcal.triangulate_leecivera_mid2()`

if we're looking at objects very close to either camera.

## Synthetic data

`mrcal.ref_calibration_object()`

: Return the geometry of the calibration object`mrcal.synthesize_board_observations()`

: Produce synthetic chessboard observations`mrcal.make_perfect_observations()`

: Write perfect observations with perfect noise into the optimization_inputs

## CHOLMOD interface

The mrcal solver is an optimization routine based on sparse nonlinear least
squares. The optimization loop is implemented in `libdogleg`

, which uses the
CHOLMOD solver to compute the Cholesky factorization. With a Cholesky
factorization we can efficiently solve the linear system \(J^T J \vec a = \vec b\)
where the jacobian matrix \(J\) is large and sparse.

CHOLMOD is a C routine, and mrcal provides a Python interface. This is used
internally for the projection uncertainty computations, and is convenient for
general analysis. The sparse \(J\) matrix is available from the optimizer via the
`mrcal.optimizer_callback()`

function, as a `scipy.sparse.csr_matrix`

sparse
array.

The factorization can be computed by instantiating a
`mrcal.CHOLMOD_factorization`

class, and the linear system can then be solved by
calling `mrcal.CHOLMOD_factorization.solve_xt_JtJ_bt()`

. See these two
docstrings for usage details and examples.

## Layout of the measurement and state vectors

Functions to interpret the contentes of the state and measurement vectors.

`mrcal.state_index_intrinsics()`

: Return the index in the optimization vector of the intrinsics of camera i`mrcal.state_index_extrinsics()`

: Return the index in the optimization vector of the extrinsics of camera i`mrcal.state_index_frames()`

: Return the index in the optimization vector of the pose of frame i`mrcal.state_index_points()`

: Return the index in the optimization vector of the position of point i`mrcal.state_index_calobject_warp()`

: Return the index in the optimization vector of the calibration object warp`mrcal.num_states_intrinsics()`

: Get the number of intrinsics parameters in the optimization vector`mrcal.num_states_extrinsics()`

: Get the number of extrinsics parameters in the optimization vector`mrcal.num_states_frames()`

: Get the number of calibration object pose parameters in the optimization vector`mrcal.num_states_points()`

: Get the number of point-position parameters in the optimization vector`mrcal.num_states_calobject_warp()`

: Get the number of parameters in the optimization vector for the board warp`mrcal.num_intrinsics_optimization_params()`

: Get the number of intrinsics parameters to describe*one*camera`mrcal.measurement_index_boards()`

: Return the measurement index of the start of a given board observation`mrcal.measurement_index_points()`

: Return the measurement index of the start of a given point observation`mrcal.measurement_index_regularization()`

: Return the index of the start of the regularization measurements`mrcal.num_measurements_boards()`

: Return how many measurements we have from calibration object observations`mrcal.num_measurements_points()`

: Return how many measurements we have from point observations`mrcal.num_measurements_regularization()`

: Return how many measurements we have from regularization`mrcal.num_measurements()`

: Return how many measurements we have in the full optimization problem`mrcal.num_states()`

: Get the total number of parameters in the optimization vector

## State packing

The optimization routine works in the space of scaled parameters, and several functions are available to pack/unpack the state vector \(\vec b\).

`mrcal.pack_state()`

: Scales a state vector to the packed, unitless form used by the optimizer`mrcal.unpack_state()`

: Scales a state vector from the packed, unitless form used by the optimizer`mrcal.ingest_packed_state()`

: Read a given packed state into optimization_inputs

## Optimization

Direct interfaces to the mrcal optimizer.

`mrcal.optimize()`

: Invoke the calibration routine`mrcal.optimizer_callback()`

: Call the optimization callback function

## Camera model reading/writing

The `mrcal.cameramodel`

class provides functionality to read/write models
from/to files on disk. Both the `.cameramodel`

and `.cahvor`

file formats are
supported, choosing the proper one, depending on the given filename. When
reading a pipe (no filename known), both formats are tried. If writing to a
pipe, the `.cameramodel`

format is chosen, unless `.cahvor`

is requested via the
arguments. The available methods:

`mrcal.cameramodel.__init__()`

: Read a model from a file on disk, or construct from the data given in the arguments.`mrcal.cameramodel.write()`

: Write out this camera-model to a file`mrcal.cameramodel.intrinsics()`

: Get or set the intrinsics in this model`mrcal.cameramodel.extrinsics_rt_toref()`

: Get or set the extrinsics in this model`mrcal.cameramodel.extrinsics_rt_fromref()`

: Get or set the extrinsics in this model`mrcal.cameramodel.extrinsics_Rt_toref()`

: Get or set the extrinsics in this model`mrcal.cameramodel.extrinsics_Rt_fromref()`

: Get or set the extrinsics in this model`mrcal.cameramodel.imagersize()`

: Get the imagersize in this model`mrcal.cameramodel.valid_intrinsics_region()`

: Get or set the valid intrinsics region`mrcal.cameramodel.optimization_inputs()`

: Get the original optimization inputs. Used for uncertainty evaluation or other analysis`mrcal.cameramodel.icam_intrinsics()`

: Get the camera index indentifying this camera at optimization time. Used in conjunction with`mrcal.cameramodel.optimization_inputs()`

## Image reading/writing

mrcal includes simple functions for reading/writing images. These aren't interesting, or better than any other functions you may have already. These exist because they're faster than loading the opencv module and to make life easy for those that don't already have other functions handy.

`mrcal.load_image()`

: load an image from a file on disk into a numpy array`mrcal.save_image()`

: save an image in a numpy array to a file on disk

## Miscellaneous utilities

`mrcal.hypothesis_board_corner_positions()`

: Reports the coordinates of chessboard points, as predicted by the optimization state`mrcal.polygon_difference()`

: Return the difference of two closed polygons`mrcal.mapping_file_framenocameraindex()`

: Parse image filenames to get the frame numbers`mrcal.close_contour()`

: Close a polyline, if it isn't already closed`mrcal.apply_homography()`

: Apply a homogeneous-coordinate homography to a set of 2D points`mrcal.corresponding_icam_extrinsics()`

: Return the icam_extrinsics corresponding to a given icam_intrinsics`mrcal.residuals_board()`

: Compute and return the chessboard residuals`mrcal.residuals_point()`

: Compute and return the discrete point residuals`mrcal.sorted_eig()`

: Compute eigenvalues, eigenvectors; sorted results returned