Posted on by Martin H. Trauth

MATLAB/LEGO MINDSTORMS Exercise #8 Image Stitching with MATLAB

This exercise is intended to help students to understand how to merge multiple images taken with a webcam using MATLAB.

  1. Build a rotation platform with a single large motor and an RGB camera mounted on top facing to the side.
  2. Acquire images of an object of your choice with the RGB camera rotated by the device .
  3. Correct the images for differences in brightness, color and contrast.
  4. Merge the images to a single picture.

Solution

  1. First you have to build the device according to the LEGO building instructions. The building instructions are in a LXF file which can be opened with the free LEGO Digital Designer software, which is available for computers running macOS or Windows. After launching the software you can use the view menu to switch from the construction mode to the building instructions mode.
  2. You need the Computer Vision Toolbox and a high-resolution RGB webcam, which is supported by MATLAB, such as the Logitech C922 Pro Stream Webcam. To acquire images with your webcam using MATLAB you need the USB Webcam Support with MATLAB.
  3. To generate the panorama you have to take two or more pictures at approximately the same distance, with an image overlap of about 40–60 percent. For the best results you should rotate the camera at a fixed location rather than moving it on rails. We obtained good results from rotating the camera for 0.4 seconds with 10% power.
  4. Since the camera uses automatic settings for brightness, contrast and color, the images are slightly different. The images therefore need to be corrected for differences in these parameters to get the best possible results.
  5. To merge the images the algorithm has to find objects which are present in both images to match them. This can be done with the detectSURFFeatures. With more than two images you have to iterate through all the images.
  6. Then estimate the geometric transform from the matched point pairs using estimateGeometricTransform for each image pair.
  7. Create an empty panorama with the dimension similar to the one expected from the merged images.
  8. Create an spatial object with world coordinates and the dimension of the panorama. This can be done with the imref2d class. This is used to place every single image correctly within the panorama.
  9. Create a blender for overlapping the images, warp the images, and merge the images into the panorama using the created blender, panorama, warped images and mask.

LEGO building instructions and MATLAB script to run the experiment

Download the building instructions and MATLAB script. The LXF file contains the building instructions to be used with the free LEGO Digital Designer software available for macOS and Windows.