MATLAB® and LEGO® MINDSTORMS®

I am currently working on a MATLAB/LEGO® MINDSTORMS® course for undergraduate and graduate students, aiming to improve their skills to build efficient teams to solve typical problems in earth sciences in acquiring, processing and analyzing typical multispectral (visible, infrared, thermal), geophysical (seismic, magnetic) and geometric (2D, 3D) data.

Using the LEGO® MINDSTORMS® Education and Extension EV3 kits, together with LEGO® SPEED® rail tracks and wheels, connected by using the classic LEGO Cross Axle, groups of 4–5 students will design the model of a moving satellite in a ~20 hour lab class. After a series of preparatory exercises to become familiar with the MATLAB Support Package for LEGO MINDSTORMS EV3 Hardware, communicating with the EV3, controlling the motors and reading the sensors, each of the four groups is asked to first create the rover, which moves by a servo motor on rails using MINDSTORMS and SPEED technology. The four groups use different sensors mounted on the rover: (1) a red,green,blue camera to acquire series of overlapping images, similar to those of a moving satellite; (2) a visible-near infrared (IR) sensor to measure normalized difference vegetation index (NDVI) of nearby vegetation; (3) a thermal infrared sensor to measure the heat radiation of objects; (4) a infrared, a ultrasonic sensor and/or a 3D laser to determine the 3D shape of objects using the Light Detection And Ranging (LIDAR) concept.

The LEGO MINDSTORMS Education EV3 kits include several sensors, including a gyro sensor, a ultrasonic sensor, a color sensor and an infrared sensor. The MATLAB Support Package for LEGO MINDSTORMS EV3 Hardware provides MATLAB functions to control the motors and interface with the hardware input sensors and output capabilities. In addition to those, there are a number of low-priced but high-quality sensors on the market, which are in some cases equipped with LEGO-compatible mounting systems. There are a number of companies that offer such sensors for the MINDSTORMS series. The challenge for all groups is to design the rover-sensor system, acquiring the data using the MINDSTORMS technology, transferring the data using USB, Bluetooth and WiFi technology, determine the accuracy and the precision of the system, calculate system characteristics such as impulse impulse, step and frequency response.

The next step, with the use of MATLAB, is to rectify, merge and reference images of objects of interest from multiple images taken by the moving sensors. After these geometric corrections, the groups will apply classification algorithms to identify types of vegetation and other materials, and calculate 3D shapes of objects such as fossils or geological outcrops. This application of image processing is very similar to the acquisition and processing of moving IR and LIDAR sensors on satellites or aircrafts. The data acquired with the LEGO MINDSTORMS EV3 model of a satellite will be processed and analyzed with the same MATLAB-based tools to be used with satellite images, such as the ones included in the Image Processing Toolbox, the Signal Processing Toolbox, the Mapping Toolbox, as well as in the textbooks MATLAB Recipes for Earth Sciences, 4th Edition (Springer 2015) and MATLAB and Design Recipes for Earth Sciences, 1st Edition (Springer 2013, 2nd Edition in preparation).

Project Goals: These proposed course will be designed for undergraduate and graduate students, aiming to improve their skills to build efficient teams to solve typical problems in earth sciences in the acquiring, processing and analyzing typical multispectral (visible, infrared, thermal) and geometric (2D, 3D) data. The highlight of the course proposed here is the consistent use of modern teaching concepts, such as flipped classroom and constructionist learning concepts. In the implementation of the course, lectures and demonstrations at the beginning of each module are gradually replaced by the group challenges, team teaching and joint learning, which is expected to generate important synergies between participants and instructors. This learning approach helps participants not only in designing their own project work, but also provides new teaching concepts for the implementation of their own teaching.

The consistent use of MATLAB as a market-leading 4GL programming language in a state-of-the-art numerical computing environment, together with the computer-controlled electromechanical parts of LEGO MINDSTORMS allows students to plan, build and program models of an earth science acquisition system, such as multi- and hyperspectral satellites and acquisition systems for the analysis of geometric and structural design. In addition to this comprehensive goals, students will be familiarized with various aspects of data acquisition (sensors and their specs, such as bit depth, sampling rate, accuracy), data transfer (USB, Wifi, Bluetooth), code optimization and visualization of the acquired data in 2D and 3D.

The course material will be made available soon in the membership area.