Automotive smart cameras use deep learning
Following a collaboration with, StradVision, Renesas Electronics announces the joint development of a deep learning-based object recognition solution for smart cameras. StradVision’s software has been optimised to run on Renesas Electronics’ R-Car SoCs.
The deep learning-based object recognition system is for smart cameras used in next-generation advanced driver assistance system (ADAS) applications and cameras for ADAS Level 2 and above.
Next-generation ADAS implementations require high-precision object recognition capable of detecting vulnerable road users (VRUs) such as pedestrians and cyclists. These systems must also consumer very low power for mass-market mid-tier to entry-level vehicles.
According to Naoki Yoshida, vice president of Automotive Technical Customer Engagement, at Renesas, StradVision is a leader in vision processing technology, with “abundant experience developing ADAS implementations using Renesas’ R-Car SoCs”. The collaboration has produced production-ready solutions “that enable safe and accurate mobility in the future,” said Yoshida. The deep learning based camera system is expected to contribute to the widespread adoption of next-generation ADAS implementations and support the escalating vision sensor requirements expected to arrive in the next few years.
StradVision’s deep learning–based object recognition software delivers high performance in recognising vehicles, pedestrians and lane marking. The high-precision recognition software has been optimised for Renesas R-Car automotive SoCs R-Car V3H and R-Car V3M. These R-Car devices incorporate a dedicated engine for deep learning processing called CNN-IP (Convolution Neural Network Intellectual Property), enabling them to run StradVision’s SVNet automotive deep learning network at high speed with minimal power consumption. The object recognition characteristic realises deep learning–based object recognition while maintaining low power consumption, suitable in mass-produced vehicles, encouraging ADAS adoption.
StradVision’s SVNet deep learning software is an AI perception solution for the mass production of ADAS systems. It is characterised by recognition precision in low-light environments and its ability to deal with occlusion when objects are partially hidden by other objects. The basic software package for the R-Car V3H performs simultaneous vehicles, person and lane recognition, processing the image data at a rate of 25 frames per second. Developers can customise the software, adding signs, markings and other objects as recognition targets. StradVision provides support for deep learning-based object recognition covering all the steps from training through the embedding of software for mass-produced vehicles.
In addition to the CNN-IP dedicated deep learning module, the Renesas R-Car V3H and R-Car V3M feature the IMP-X5 image recognition engine. The on-chip image signal processor (ISP) is designed to convert sensor signals for image rendering and recognition processing. This makes it possible to configure a system using inexpensive cameras without built-in ISPs, reducing the overall bill-of-materials (BoM) cost, says Renesas.
The R-Car SoCs featuring the new joint deep learning solution, including software and development support from StradVision, are scheduled to be available to developers by early 2020.
