NVIDIA Cosmos expands with new data and training tools for world models

NVIDIA has expanded the Cosmos platform with new data preparation, training and simulation tools for teams developing world models. The company published the announcement on its blog on 22 October; some of the updates were presented as part of announcements at GTC Berlin. According to NVIDIA, Cosmos brings together data, model and optimization tools to help physical AI systems understand and simulate the real world.

Five workflow blueprints for data preparation

Cosmos Data Factory focuses on scaling up data labeling, curation, diversification and evaluation. Cosmos Curator, which uses GPUs to curate video training datasets, also supports new components for processing multimodal and multisensory inputs.

Data Factory includes five cloud-native blueprints available through GitHub, Microsoft Azure, Nebius and CoreWeave. These ready-made workflows target different data preparation needs:

• For autonomous vehicle projects, driving logs can be curated and labeled, while additional examples can be generated for rare or hard-to-capture situations.

• The visual attribute augmentation workflow changes the appearance of objects to create more diverse content for visual search.

• The event video blueprint turns still images into moving examples that depict rare or safety-critical situations.

• Defect image generation aims to create inspection data showing flaws in circuit boards, metal and glass.

• The video augmentation workflow changes weather and lighting conditions in footage while preserving the scene’s overall layout.

According to NVIDIA, Porsche is using Cosmos Data Factory through Voxel51 and Microsoft Azure to develop a shared autonomous vehicle data loop across Volkswagen Group brands. The company says this approach is expected to reduce repeated physical test drives by up to 90% and development time by up to 85%. These figures are projections shared by NVIDIA, not independent measurements reported in the announcement.

Model training and optimization

Cosmos Framework is described as an open, modular structure for model training and optimization. NVIDIA says developers can adapt open models such as GR00T, Alpamayo, QWEN and Pi0.5 through fine-tuning or reinforcement learning. Compatible Hugging Face models can also be used with these tools. After training, models can be optimized for deployment through distillation and quantization.

In the MicroAGI example included in the announcement, Cosmos 3 was retrained using videos showing people interacting with objects. The 32-billion-parameter reasoning model was then distilled into an 8-billion-parameter Nano model, reportedly with no loss in accuracy. This result pertains to a specific application shared by the company; it does not show that the same outcome will be achieved with other models or tasks.

Implications for simulation and design workflows

The new simulation variants of Cosmos 3 Nano are intended for testing policies developed for autonomous vehicles and dual-arm robots in repeatable, multi-view and multisensory scenarios. NVIDIA also reports that, in validated configurations, Cosmos 3 NIM microservices deliver up to twice the inference speed without loss of quality. Metropolis Blueprint 3.3, for video search and summarization, is said to help monitor 40% more video streams on the same GPU.

For architecture and interior design teams, the announcement does not introduce a new modeling or visualization feature. However, projects working with real-world data, such as building and facility inspection, could explore synthetic defect or event data generation. The announcement does not specify software integrations or hardware requirements for archviz artists. Before adopting a workflow, teams should therefore verify compatibility with their tools, GPU requirements and access conditions for the relevant services.