Nvidia unveils Isacc ROS GEMS with AI perception
At the robotics event, ROS World 2021, Nvidia unveiled Isaac ROS GEMS, graphic processing unit (GPU) programming examples. They provide packages that include image processing and computer vision, including deep neural network (DNN) -based algorithms that are optimised for Nvidia GPUs and Jetson.
As autonomous machines move around their environments, they must keep track of where they are using visual odometry which estimates where a camera is relative to its starting position. The Isaac ROS GEM for stereo visual odometry provides this functionality to ROS developers. According to Nvidia, this GEM offers the best accuracy for a real-time stereo camera visual odometry solution. In addition to being highly accurate, this GPU-accelerated package runs extremely fast. It is now possible to run SLAM on HD resolution (1280 x 720) in real-time (more than 60 frames per second) on an Nvidia Jetson AGX Xavier.
Developers can use any of Nvidia’s inference models available via Nvidia’s website or provide their own DNN with DNN Inference GEM. Pre-trained models or optimisations of developers’ own models can be tuned via Nvidia’s TAO toolkit.
After optimisation, TensorRT or Triton, Nvidia’s inference server, deploy these packages. Optimal inference performance will be achieved with the nodes leveraging the inference software development kit, TensorRT, advises Nvidia. If TensorRT does not support the desired DNN model, then Nvidia Triton should be used to deploy the model, says the company.
The GEM includes native support for U-Net and DOPE. The U-Net package, based on TensorRT, can be used for generating semantic segmentation masks from images; the DOPE package can be used for 3D pose estimation for all detected objects.
According to Nvidia, this tool is the fastest way to incorporate performant artificial intelligence (AI) inference into a ROS application.
The GA release of Isaac Sim, which will be available in November 2021, with improvements in the user interface, performance and useful building blocks to improve simulations which are built faster. There are also planned to be an improved ROS bridge and more ROS samples.
Nvidia has a virtual booth at ROS World 21, with technical presentations on Isaac.
