NVIDIA has released a new version of its Isaac Software Development Kit (SDK), providing robots with updated AI perception and simulation capabilities.
NVIDIA founder and CEO Jensen Huang announced the news at NVIDIA's latest GPU Technology Conference (GTC CHINA 2019). The Isaac SDK represents a significant milestone in establishing a unified robotics development platform for AI, simulation, and manipulation capabilities.
The Isaac SDK includes the Isaac Robotics Engine (providing the application framework), the Isaac GEM (pre-built deep neural network models, algorithms, libraries, drivers, and APIs), a reference application for indoor logistics, and the first version of Isaac Sim (providing navigation functionality).
The new Isaac SDK can significantly accelerate the development and testing of robots by researchers, developers, startups, and manufacturers. It enables robots to acquire AI-driven perception and training capabilities through simulation, allowing for testing and validation of robots in a variety of environments and situations.
This can save costs.
Starting with AI-based perception
Every autonomous machine begins with perception.
To accelerate the development of AI robots, the new Isaac SDK includes various camera-based perceptual deep neural networks. Among them:
Object detection—identifying objects used for navigation, interaction, or manipulation.
Free space segmentation—detecting and segmenting the external world, such as determining where sidewalks are and where robots can move.
3D pose estimation—understanding the position and orientation of a target to enable tasks such as robotic arms picking up objects.
2D Human Pose Estimation – Applying pose estimation to humans is crucial for robots that interact with humans (e.g., delivery robots) and collaborative robots (specifically designed to work with humans).
The SDK's object detection has also been updated with ResNet deep neural networks, which can be trained using NVIDIA's transfer learning toolkit. This makes it easier to add new objects for detection and train new models that can start and run with high accuracy.
Launch Isaac Sim
The new version introduces a significant feature: training robots using Isaac Sim and deploying the resulting software onto real robots operating in the real world. This is expected to greatly accelerate robot development, enabling training with comprehensive data.
Through simulation, developers can test the robot under extreme conditions (i.e., difficult or abnormal situations) to further enhance its training. These results are then fed into the training pipeline, allowing the neural network to improve accuracy based on both real and simulated data.
Multi-robot Sim is here
The new SDK also provides multi-robot simulation. This allows developers to place multiple robots in a simulation environment for testing, so that they learn to work in relation to each other. Individual robots can run independent versions of the Isaac navigation software stack while moving within a shared virtual environment.
Therefore, manufacturers seeking to operate multiple robots in large-scale logistics management can first test robot interactions and debug issues before deploying them in the real world.
Isaac Integration with DeepStream
The new SDK also integrates support for NVIDIA DeepStream software, which is widely used for processing analytics. Developers can deploy DeepStream and NVIDIA GPUs at the edge AI to support robotics applications for processing video streams.
Developers can now build a wide variety of robots that require analysis of camera video sources, for both onboard applications and remote positioning.
Programming using the Isaac SDK
Finally, for robot developers who have already developed their own code, the new SDK can also integrate their work and adds new APIs based on the C programming language.
This allows developers to connect their software stacks to the Isaac SDK and minimize programming language switching—providing users with access to Isaac functionality via the C API.
C-API access also allows developers to use the Isaac SDK in other programming languages.
