Meet the Omnivore: Developer Creates Bots with NVIDIA Omniverse and Isaac Sim

Editor’s Note: This article is part of our Meet the omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.

Antonio Serrano-Munoz

While still a student, Antonio Serrano-Muñoz helped write papers on planetary gravities, AI-based rheumatoid arthritis diagnosis, and robots that accurately track millimeter-sized walkers, like ants. .

Now the Ph.D. an applied robotics student at the University of Mondragon in northern Spain uses robotics and AI in refurbishment to tackle climate change and pollution issues.

In short, Serrano-Muñoz is busy discovering unique ways to apply technology to complex real-world problems – and in his spare time, he’s creating extensions for NVIDIA Omniverse, a 3D design collaboration platform. real-time and global simulation.

Omniverse Extensions are basic building blocks that allow anyone to create and extend the functionality of Omniverse applications to meet their specific workflow needs with just a few lines of Python code.

Serrano-Muñoz has created six open-source Omniverse extensions accessible on GitHub, one of which enhances the capabilities of NVIDIA Isaac Sim, an Omniverse-powered application framework for creating photorealistic and physically accurate virtual environments in which to develop, train, and simulate. AI robots.

“Since the beginning of my doctorate. studies, I decided to work with Isaac Sim to simulate robots,” said Serrano-Muñoz. “It offers a powerful real-time simulation platform with an ultra-realistic physics engine and graphics, as well as a clean and simple application programming interface that makes it easy to interact with the entire system.”

Omniverse for robotics simulation

Serrano-Muñoz has been pursuing robotics for as long as he can remember. Growing up in Cuba, he always played with electronics and fell in love with technology when he started coding in college.

“Robots can help humans with strenuous, high-precision, repetitive and sometimes dangerous tasks,” Serrano-Muñoz said. “They have the potential to improve our lives, and I hope my work advances robotics in a way that allows us to build a better present and realize a better future.”

He thinks Omniverse is crucial for his doctoral studies in applied robotics.

“Achieving real-time, graphically realistic simulations of robotic environments was not possible before Omniverse,” he said. “The platform opens the door to a new era of groundbreaking changes in robotics, simulation, and real-time collaboration.”

Omniverse connects specialists of all kinds – engineers, designers, content creators – for the development of simulation systems, he added. The key to this is Universal Scene Description (USD), an open source 3D scene description and extensible file framework serving as a common language for virtual worlds.

“USD plays an important role in the process of creating, compositing, and playing back a hierarchically organized scene to create and manipulate its render elements and objects,” Serrano-Muñoz said.

Omniverse Expansion for Isaac Sim

Using NVIDIA Omniverse, the developer created robot simulations for industrial use cases and a digital twin of the Mondragon University lab for robotic prototyping.

A digital twin of a bench at Mondragon University’s robotics lab, made with NVIDIA Omniverse.

And while working on such projects, Serrano-Muñoz wanted to integrate into Isaac Sim a tool he already knew: Robot Operating System, or ROS, a set of software libraries for building robotic applications. So he created an Omniverse extension to allow this.

The extension allows users to manipulate simulated robotic systems in the Omniverse-powered Isaac Sim app through ROS control interfaces. ROS MoveIt, a motion planning framework for robots, can be used in conjunction with Isaac Sim’s dynamic control extension and PhysX capabilities, which bring physical precision to high-fidelity robot simulations.

“It’s easy to develop code without leaving the Omniverse Kit,” Serrano-Muñoz said. “Omniverse extensions come with a system-wide integration API, install, activate, and reload mechanisms to augment the functionality of Omniverse apps.”

This particular extension for ROS, he added, boosts agile prototyping for robotics applications – which is further accelerated by its NVIDIA RTX 3080 laptop GPU – making its workflow faster than ever.

Learn more about Serrano-Muñoz’s use of digital twins for industrial robotics by watching his NVIDIA GTC on-demand session. And watch his Community Spotlight on the Omniverse Twitch channel which will run on August 3 at 11:00 a.m. PT.

Take part in the creation

Creators and developers around the world can download NVIDIA Omniverse for free, and enterprise teams can use the platform for their 3D projects.

Developers like Serrano-Muñoz will join NVIDIA at SIGGRAPH, a global computer graphics conference taking place August 8-11. Watch the Omniverse Community Live Stream on SIGGRAPH on Tuesday, August 9 at noon PT to learn how Omniverse and other design and visualization technologies are driving breakthroughs in GPU-accelerated graphics and software.

Additionally, anyone can participate in the first #ExtendOmniverse Developer Contest until Friday, August 19. Create an Omniverse expansion using the Omniverse code for a chance to win an NVIDIA RTX GPU.

Discover the works of other “Omnivores” and submit projects in the gallery. Connect your workflows to Omniverse with software from Adobe, Autodesk, Epic Games, Maxon, Reallusion and more.

Follow NVIDIA Omniverse on instagram, Twitter, Youtube and Medium for additional resources and inspiration. Discover the Omniverse forumsand join our Discord Server and Twitch Channel to chat with the community.

Comments are closed.