Meet the Omnivore: Developers create bots using NVIDIA Omniverse and Isaac Sim

Editor’s Note: This post is part of ours Meet the omnivore Series showcasing individual creators and developers using NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Antonio Serrano Muñoz

While still in graduate school, Antonio Serrano-Muñoz helped author articles on planetary gravity, AI-assisted diagnosis of rheumatoid arthritis, and robots that precisely track millimeter-sized walkers like ants.

Well, the Ph.D. Applied robotics student at Mondragon University in northern Spain, uses robotics and AI in remanufacturing to address climate change and pollution issues.

In short, Serrano-Muñoz is busy discovering unique ways to apply technology to complex, real-world problems — and in his spare time he creates extensions for NVIDIA Omniverse, a real-time 3D design collaboration and world simulation platform.

Omniverse Extensions are core building blocks that allow anyone to build and extend the functionality of Omniverse apps to meet the specific needs of their workflows with just a few lines of Python code.

Serrano-Muñoz has created six open-source Omniverse extensions accessible on GitHub, one of which extends the capabilities of NVIDIA Isaac Sim, an Omniverse-based application framework for creating photorealistic, physically accurate virtual environments for development, training and simulate AI robots.

“Since I started my PhD, I decided to work with Isaac Sim to simulate robots,” said Serrano-Muñoz. “It offers a powerful real-time simulation platform with an ultra-realistic physics engine and graphics – as well as a clean, simple application programming interface that makes it easy to interact with the entire system.”

Omniverse for robotics simulation

Serrano-Muñoz has been involved with robotics for as long as he can remember. Growing up in Cuba, he was always playing around with electronics and fell even more in love with technology when he started programming in college.

“Robots can assist humans in strenuous, high-precision, repetitive, and sometimes dangerous tasks,” Serrano-Muñoz said. “They have the potential to improve our lives, and I hope my work will advance robotics in a way that allows us to build a better present and achieve a better future.”

He believes Omniverse is critical to his PhD in applied robotics.

“Performing graphically realistic simulations of robot environments in real-time was not possible before Omniverse,” he said. “The platform opens the door to a new era of revolutionary changes in robotics, simulation and real-time collaboration.”

Omniverse connects specialists of all kinds – engineers, designers, content creators – for the development of simulation systems, he added. Key to this is Universal Scene Description (USD), an open-source 3D scene description and extensible file framework that serves as a common language for virtual worlds.

“USD plays an important role in authoring, composing, and reading a hierarchically organized scene to create and manipulate its rendering elements and objects,” said Serrano-Muñoz.

Expansion of the omniverse for Isaac Sim

Using NVIDIA Omniverse, the developer has created robot simulations for industrial use cases and a digital twin of the Mondragon University lab for robotics prototyping.

A digital twin of a bank in Mondragon University’s robotics lab, created with NVIDIA Omniverse.

And while working on such projects, Serrano-Muñoz wanted to integrate a tool into Isaac Sim with which he was already familiar: Robot Operating System, or ROS, a set of software libraries for building robotic applications. So he created an Omniverse Extension to do just that.

The extension allows users to manipulate simulated robotic systems in the Omniverse powered Isaac Sim application via ROS control interfaces. ROS MoveIt, a motion planning framework for robots, can be used in conjunction with Isaac Sim’s dynamic control extension and PhysX capabilities that bring physical accuracy to high-fidelity robotics simulations.

“It’s easy to develop code without leaving the Omniverse Kit,” said Serrano-Muñoz. “Omniverse Extensions feature a system-wide integration API, install, enable, and reload mechanisms to extend the functionality of Omniverse Apps.”

This particular extension for ROS, he added, encourages agile prototyping for robotics applications – which is further accelerated by its NVIDIA RTX 3080 laptop GPU – and makes its workflow faster than ever.

Learn more from Serrano-Muñoz about using digital twins for industrial robotics by watching his on-demand NVIDIA GTC session. And check out his community spotlight on the Omniverse Twitch channel, taking place August 3 at 11:00 am PT.

Join in the creation

Creators and developers around the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Developers like Serrano-Muñoz will be joining NVIDIA at SIGGRAPH, a global computer graphics conference taking place August 8-11. Watch the Omniverse community livestream at SIGGRAPH on Tuesday, August 9 at 12:00 PM PT to learn how Omniverse and other design and visualization technologies are driving breakthroughs in graphics and GPU-accelerated software.

Also, anyone can register for the first #ExtendOmniverse developer contest until Friday, August 19th. Build an Omniverse Expansion with Omniverse Code to win an NVIDIA RTX GPU.

View artworks by other omnivores and submit projects to the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.

Follow NVIDIA Omniverse on Instagram, Twitter, youtube and Middle for additional resources and inspiration. Look at the omniverse forumsand join our Discord server and Twitch Channel to chat with the community.

About Willie Ash

Check Also

SambaNova Systems brings a massive performance boost to its integrated AI hardware and software platform

Artificial intelligence startup SambaNova Systems Inc. today announced an overhauled version of its flagship system …