Stereo Vision IP Core for FPGAs

Stereo Vision IP Core

The stereo vision core is what drives our SP1 stereo vision sensor. If you require stereo vision capabilities for your own FPGA-based products, you can now license our IP core.

The stereo vision core performs stereo matching on two grayscale input images. The images are first rectified to compensate for lens distortions and camera alignment errors. Stereo matching is then performed by applying a variation of the Semi Global Matching (SGM) algorithm. Various post-processing methods are applied to improve the computed depth data. The output of the stereo vision core is a subpixel accurate and dense disparity map (an inverse depth map), which is streamed over an AXI4-Stream interface.

To simplify use of the stereo vision core on devices with a shared system memory, such as the Xilinx Zynq SoC, an auxiliary core for direct memory access (DMA) is provided. This DMA core reads input data from memory through AXI3, and converts it into a data stream that is suitable for the stereo vision core. Likewise, the DMA core collects the processing results from the stereo vision core and writes them back to memory.

If you are interested in licensing our IP cores then please contact us.

Features

  • General processing architecture
    • Processing of grayscale images with a bit depth of 8 bits per pixel
    • Stream-based processing of input images using either AXI4-Stream or AXI3
    • Configuration through AXI4-Lite interface
    • Output of disparity map starts before receiving the last pixel of both input images
    • Support for variable image sizes
    • Multi-clock design with faster clock for performance critical tasks
  • Image rectification
    • Rectification using a pre-computed compressed rectification map
    • Bi-linear interpolation for subpixel accurate rectification
  • Stereo matching
    • Stereo matching through a variation of the Semi-Global Matching (SGM) algorithm
    • Configurable disparity range form 32 to 256 pixels
    • Configurable disparity offset
    • Configurable penalties P1 and P2 for small and large disparity variations
    • Pre-processing of input images for improved robustness against illumination variations and occlusions
  • Post-processing
    • Subpixel optimization
    • Consistency check with configurable threshold
    • Uniqueness check with configurable threshold
    • Filling of small gaps through interpolation
    • Noise reduction
    • Speckle filtering
    • Filtering of untextured image areas

Data Sheet / Documentation

Previous versions of the listed data sheet can be found in our download archive.

Depth Measurement Examples

Example left camera image Example depth measurements
Close
Example left camera image
Example left camera image.
Close
Example depth measurements
Depth measurements (disparity map) computed by SP1.
Example left camera image Example depth measurements
Close
Example left camera image
Example left camera image.
Close
Example depth measurements
Depth measurements (disparity map) computed by SP1.
Example left camera image Example depth measurements
Close
Example left camera image
Example left camera image. Taken from the Middlebury 2006 dataset.
Close
Example depth measurements
Depth measurements (disparity map) computed by SP1.

See also