Nvidia upgrades Omniverse to for OpenUSD framework and generative AI

“Nvidia AI and Omniverse are two are distinct platforms,” Lebaredian said. “But they’re linked. They support each other. We can’t predict exactly when AI is going to be smart enough to do some of the things we want. There’s no way to know. It’s all research. We’re pleasantly surprised that the large language models and things that they’re doing with ChatGPT show the world happened a little bit earlier than most people expected. And so now we’re harnessing that for for Omniverse.”

The marriage of AI and Omniverse

Lebaredian said that with AI today, the big change is that large language models are encapsulating a lot of knowledge and it seems like AI is understanding what humans are searching for.

“This changes everything for USD and what we’re doing with 3D,” he said. “One of the fundamental changes is the ability to discern natural language and a human’s intent. Not a lot of people in the world have a deep understanding of how a computer works and how computer languages work. Not a lot of people in the world can write software programs. But what we’re seeing with ChatGPT, and in all these other models, is that they’re actually quite good at writing software, which democratizes the ability to program. It feels like it’s not going to be long until virtually everyone who has access to a computer is essentially going to be a programmer. You just tell it what to do.”

“With the creation of 3D virtual worlds, this is invaluable,” Lebaredian said. Being able to write programs that that assist you in generating the worlds is awesome. So what we announced there is where we’re working on training a model that specialized with USD with code that calls into the USD API’s. We call those code snippets. So that’s so that anyone could potentially become a developer for USD.”

He said you can use prompts to do things like find all objects in the scene. You can tell it to find all objects in the scene that are larger than a certain amount or have metallic material. Normally, you’d have to write a Python script or some C++ to do something like that.”

Omniverse upgrades

Omniverse, Nvidia’s platform for building and connecting 3D tools, received a major upgrade. New connectors and advancements showcased in Omniverse foundation applications enhance the platform’s efficiency and user experience. Notable updates include Omniverse USD Composer, which allows users to assemble large-scale, OpenUSD-based scenes, and Omniverse Audio2Face, which provides generative AI APIs for realistic facial animations and gestures.

Popular applications such as Cesium, Convai, Move AI, SideFX Houdini, and Wonder Dynamics are now seamlessly connected to Omniverse via OpenUSD, expanding the platform’s capabilities.

Rev Lebaredian, vice president of Omniverse and Simulation Technology at Nvidia, said in an interview with GamesBeat that there’s growing demand for connected and interoperable 3D software ecosystems among industrial enterprises.

He emphasized that the latest Omniverse update empowers developers to harness generative AI through OpenUSD, enhancing their tools. Moreover, the update enables enterprises to build larger, more complex world-scale simulations, serving as digital testing grounds for industrial applications.

Key improvements to the Omniverse platform include enhancements to Omniverse Kit, which serves as the engine for developing native OpenUSD applications and extensions. Moreover, Nvidia introduced the Omniverse Kit Extension Registry, a central repository for accessing, sharing, and managing Omniverse extensions.

This registry empowers developers to easily customize their applications by enabling them to turn functionalities on and off. Additionally, extended-reality developer tools were introduced, enabling users to incorporate spatial computing options into their Omniverse-based applications.

With over 600 core Omniverse extensions provided by Nvidia developers can now build custom applications with greater ease, enabling modular app building.

The Omniverse update also brings new developer templates and resources that provide a headstart for developers starting with OpenUSD and Omniverse, requiring minimal coding to get started. Rendering optimizations have been implemented to fully leverage the capabilities of Nvidia’s Ada Lovelace architecture enhancements in Nvidia RTX GPUs. The integration of DLSS 3 technology into the Omniverse RTX Renderer and the addition of an AI denoiser enables real-time 4K path tracing of massive industrial scenes.

Another key highlight of the update is the native RTX-powered spatial integration, which allows users to build spatial-computing options directly into their Omniverse-based applications. This integration provides users with flexibility in experiencing their 3D projects and virtual worlds.

The Omniverse platform update includes upgrades to various foundation applications, which serve as customizable reference applications for creators, enterprises, and developers. One such application is Omniverse USD Composer, which enables users to assemble large-scale OpenUSD-based scenes. Omniverse Audio2Face, another upgraded application, offers access to generative AI APIs for creating realistic facial animations and gestures from audio files, now including multilingual support and a new female base model.

New Omniverse partners

Several customers have already embraced Omniverse for various digitalization tasks. Boston Dynamics AI Institute is utilizing Omniverse to simulate robots and their interactions, facilitating the design of novel robotics and control systems.

Continental, a leading company in automotive and autonomous systems, leverages Omniverse to generate physically accurate synthetic data at scale for training computer-vision AI models and performing system-integration testing in its mobile robots business.

Volvo Cars has transitioned its digital twin to be OpenUSD-based, providing immersive visualizations to aid customers in making online purchasing decisions. Marks Design, a brand design and experience agency, has adopted Omniverse and OpenUSD to streamline collaboration and enhance animation, visualization, and rendering workflows.

The latest release of the Omniverse platform is currently available in beta for free download and testing. Nvidia plans to launch the commercial version of Omniverse in the coming months, offering subscription plans and enterprise support to meet the needs of businesses and organizations.

With the major update to Omniverse, Nvidia aims to empower developers, creators, and industrial enterprises with advanced 3D pipelines and generative AI capabilities.

The platform’s integration with popular applications, improved developer resources, and expanded ecosystem partnerships are set to drive innovation in the fields of industrial digitalization, robotics, autonomous systems, computer graphics, and more

More announcements

In a major leap towards unlocking the next era of 3D graphics, design, and simulation, Nvidia formally announced its participation in the Alliance for OpenUSD, joining forces with industry giants Pixar, Adobe, Apple, and Autodesk. This collaboration ensures compatibility in 3D tools and content across industries, paving the way for digitalization. (Intel and Advanced Micro Devices have yet to join this effort).

Nvidia also launched three new desktop workstation Ada Generation GPUs: the NVIDIA RTX 5000, RTX 4500, and RTX 4000. And Shutterstock, a creative platform for image creators, unveiled its utilization of generative AI in 3D scene backgrounds.

Leveraging Nvidia Picasso, a cloud-based foundry for building visual generative AI models, Shutterstock trained a foundation model that can generate photorealistic, 8K, 360-degree high-dynamic-range imaging (HDRi) environment maps.

This development enables quicker scene development for artists and designers. Additionally, Autodesk announced its integration of generative AI content-creation services, developed using foundation models in Picasso, with its popular software Autodesk Maya.

Nvidia Studio Driver releases, which provide optimal performance and reliability for artists, creators, and 3D developers, also received an update. The August Nvidia Studio Driver includes support for updates to Omniverse, XSplit Broadcaster, and Reallusion iClone, ensuring peak reliability for users’ favorite creative applications.

Adobe and Nvidia strengthened their collaboration across Adobe Substance 3D, generative AI, and OpenUSD initiatives. They announced plans to make Adobe Firefly, Adobe’s family of creative generative AI models, available as APIs in Omniverse, thereby enhancing the design processes of developers and creators.

The new Nvidia RTX 5000, RTX 4500, and RTX 4000 Ada Generation professional desktop GPUs, featuring Nvidia’s Ada Lovelace architecture, deliver improved rendering, real-time interactivity, and AI performance in 3D applications.

These GPUs incorporate third-generation RT Cores for enhanced image processing and fourth-generation Tensor Cores for AI training and development. With large GPU memory and advanced video encoding capabilities, the GPUs are tailored to meet the demands of high-end creative workflows.

During the event, Nvidia artist Andrew Averkin showcased his work, “Natural Coffee,” which exemplified the fusion of art and technology. Averkin utilized AI to generate visual ideas and employed Nvidia’s GPUs and Omniverse USD Composer for efficient 3D modeling, lighting, and scene composition. The integration of these tools significantly reduced the time required for creating immersive 3D scenes.