The Greenteam Stuttgart counts on Nerian’s stereo vision technology in the Formula Student Driverless. Autonomous motor sports places high demands on the reliable detection and precise localization of track markings so that the vehicle can safely complete the course. With the SceneScan Pro stereo vision sensor, these goals are achieved through fast and efficient stereo matching
About Formula Student and the Greenteam
Formula Student is the largest student engineering competition in the world. Every year 600 teams compete in competitions all over the world with their specially designed and self-built racing cars. To be successful in the competition, a team must master a variety of disciplines, from acceleration sprints and winding endurance tracks to design presentations and business plans.
The Greenteam Stuttgart, based at the University of Stuttgart, has managed to keep its place among the top teams in the world ranking list since its foundation through great commitment and continuous innovation. Every year the team with about 40 to 50 members sets out to build a new racing car to surpass the one of the previous year.
Since 2017, the team has taken on another new challenge, the Formula Student Driverless, a new class of autonomous racing cars. For this purpose, the racing cars are upgraded in order to complete the various racing disciplines fully autonomously, i.e. completely without external influence.
The D0711-4
D0711-4 is the name of the Greenteam’s autonomous racing car for the 2019/20 season and is based on the E0711-9, an earlier manually controlled Formula Student car. The chassis and aerodynamic components of the car are made almost entirely of carbon fiber to keep the critical vehicle weight as low as possible. Four wheel hub motors deliver 80 kW of power from the 600 V high-voltage battery. The highly optimized aerodynamics ensure grip even in fast tight corners. A perception system consisting of cameras and LiDAR sensors, together with odometry, provides information on the status and position of the vehicle. A powerful ECU (Electronic Control Unit) calculates the resulting trajectory in real time and controls the vehicle. A remote-controlled braking system brings the vehicle to a halt in seconds in an emergency.
The Perception System
The Perception System is one of the most important components for the autonomous race. The demands on the system are manifold, so that the car can orientate itself quickly and safely on the track. The colored track markings must be reliably recognized and precisely located. At the same time, the system must be as efficient as possible, unnecessary data processing costs both critical reaction time and weight in the form of additionally required hardware and battery capacity. A wide variety of modalities are used in Formula Student, with very different strengths and weaknesses.
- Monocular cameras offer a high information density including color information and are very easy to implement. However, the localization of the track markings based on the 2D images is very imprecise.
- LiDAR offers excellent accuracy in localization, but hardly provides any contextual information. Because of the diverging laser beams, the range is limited here as well.
- Stereo cameras combine many of the advantages of the two modalities mentioned above. The high information density of the images is maintained, in addition to accurate depth information. By selecting appropriate lenses, stereo cameras can even exceed the ranges of the other modalities.
The Greenteam uses all the modalities mentioned here to make use of their different strengths. Two monocular cameras cover the close-up range. A LiDAR sensor provides accurate position information in the medium distance range. A stereo vision system is ideal for covering longer distances. However, it is much more difficult to implement than the other technologies. To obtain a stereo image, the images of two cameras must first be synchronized, correctly overlayed and then the depth information calculated. This means a higher development and computing effort, especially for a real-time application. The Greenteam overcomes these obstacles with the SceneScan Pro stereo vision sensor from Nerian Vision Technologies.
The Stereo Vision Solution from Nerian: SceneScan Pro
SceneScan Pro is the stereo vision application that enables the greenteam to use 3D stereo vision competitively and accurately in real-time. Using hardware-based image processing using an FPGA, SceneScan Pro is significantly more efficient, faster and more reliable than traditional solutions using data processing in a GPU. The small form factor and low power consumption simplify integration especially in the space-constrained and mobile application of a racing car. SceneScan Pro is even capable of outputting multiple trigger signals, which can be used to synchronize a variety of sensors. In the case of the Greenteam, this triggers two additional monocular cameras.
Integration into the existing software stack of the D-0711-4 is also easy. With the available ROS node and interface, SceneScan Pro can be configured with great flexibility and integrates seamlessly into the overall system. The output depth maps are processed to identify and precisely locate the required route markings.
Very high-resolution images are required to detect objects at greater distances. However, due to the fast movement of the vehicle, it is also particularly important to be able to process images quickly. With SceneScan Pro’s extremely high processing power, high frame rates can be achieved with low latency. As a result, the D-0711-4 is able to locate itself safely and precisely on the racetrack, further increasing the possible speed. This makes the SceneScan Pro 3D stereo vision sensor from Nerian ideal for the demanding application in autonomous motorsports of Formula Student Driverless.
