Overview of 3D Asset Processing and Generation

R2026b

The 3D Asset Processing Library for MATLAB^® support package enables you to create, import, process, and export 3D mesh assets for use in engineering simulation environments. You can generate textured 3D meshes from images or text prompts using AI-based deep learning models, import existing assets from standard 3D file formats, apply spatial transformations, segment meshes into subparts, and export the results to formats compatible with simulation tools such as RoadRunner, Unreal Engine, and other 3D applications.

The typical workflow for 3D asset processing and generation involves these steps, which you can perform programmatically using MATLAB functions and objects, or interactively by using the Asset Generator app.

Import or Generate 3D Assets

You can acquire 3D mesh assets by importing existing files or by generating new assets using AI models.

Import from file — Use the asset3d.read function to read 3D mesh data from these standard file formats: STL, OBJ, PLY, OFF, DAE, 3MF, FBX, USD, USDA, USDC, USDZ, glTF, and GLB. The function returns a Mesh or Scene object depending on the file contents.

Generate from image — Use the trellis object to generate a textured 3D mesh from one or more reference images. The generateFromImage object function accepts image data and generates a textured mesh.

Generate from text — Use the generateFromText object function of the trellis object to generate a textured 3D mesh from a natural language description.

Interactive generation — Use the Asset Generator app to interactively generate 3D assets from images or text prompts. The app provides a graphical interface for uploading images, entering text prompts, previewing generated assets, and adjusting generation parameters such as guidance strength, sampling steps, and texture resolution.

Inspect and Visualize Meshes

After importing or generating a 3D asset, you can inspect its properties and visualize the geometry in an interactive 3D viewer.

A Mesh object stores vertex positions, face connectivity, vertex normals, UV coordinates, and material assignments. Use the read-only properties of the mesh to query geometric information such as bounds, centroid, extents, surface area, and volume.

Use the show object function to visualize a mesh or scene in an interactive 3D viewer. The viewer supports orbit, pan, and zoom controls for inspecting the asset from any angle.

Use the snapshot object function to render a mesh or scene to an image. The function returns an RGB image array and optionally writes the result to a file. For custom rendering, you can control the camera view, image size, and background color using name-value arguments.

For multi-mesh scenes, the Scene object organizes meshes in a hierarchical transform graph. Use the dispGraph object function to display the scene hierarchy and inspect parent-child relationships between nodes.

Process and Transform Geometry

You can apply spatial transformations to meshes and scenes to adjust their position, orientation, and size for use in driving simulation.

Use these object functions to transform geometry:

• transform — Apply a 4-by-4 homogeneous transformation matrix or a rigidtform3d object.

• rotate — Rotate around a specified axis by a given angle.

• translate — Translate by a specified offset.

• scale — Scale uniformly or along individual axes.

• resize — Resize to target dimensions.

• setUpAxis — Change the up axis convention. For example, from Y-up to Z-up.

For multi-mesh scenes, use the addMesh and removeMesh object functions to build and modify the scene hierarchy. Use the subscene object function to extract a portion of the scene graph, and the flatten object function to bake node transforms into vertex positions and produce individual meshes.

Use the copy object function to create an independent deep copy of a mesh or scene for non-destructive editing.

Define or Modify Materials and Textures

Physically based rendering (PBR) materials control the visual appearance of a mesh in simulation. Use a Material object to define surface properties such as base color, metallic factor, roughness factor, emissive color, and opacity. Assign texture maps to material properties using a Texture object.

A Texture object holds image data along with sampler settings such as wrap mode and interpolation, and per-texture UV transform parameters including offset, scale, and rotation. Use the sample object function to query texture color values at specific UV coordinates.

Segment and Modify Meshes

You can segment a mesh into semantically meaningful parts based on geometric properties. Use the asset3d.segmentMeshSDF function to partition a mesh into segments based on shape thickness and local geometry. Visualize the segmentation results using the asset3d.showSegments function.

Segmentation is useful for isolating parts of a complex asset so that you can apply different materials or transformations to individual segments. For example, separating and labeling the body, wheels, and windows of a vehicle mesh.

To modify an existing asset using AI, use the modify object function of the trellis object. This object function accepts a mesh and a text prompt describing the desired change, and produces a new mesh with the requested changes applied.

In the Asset Generator app, use the Segment and Modify buttons on the Tools tab to interactively segment and modify meshes.

Export Assets

After processing your 3D assets, export them to standard file formats for use in driving simulation tools, game engines, or other 3D applications.

Use the asset3d.write function to write a mesh or scene to file. Supported output formats include: STL, OBJ, PLY, OFF, DAE, 3MF, FBX, USD, USDA, USDC, USDZ, glTF, GLB, and URDF. The function preserves color, material, and texture information for formats that inherently support these types.

These formats enable use of exported 3D assets in a wide range of applications:

• Driving simulation — RoadRunner, CARLA, and Unreal Engine^® based driving simulators (glTF/GLB, FBX, USD).

• Game engines — Unreal Engine and Unity^® for real-time visualization and virtual environments (FBX, glTF/GLB, USD).

• Robotics simulation — Robotics System Toolbox™, Robotics System Toolbox Offroad Autonomy Library, Simscape™, ROS Toolbox and Gazebo for robot model descriptions (URDF).

• Engineering simulation — Satellite Tool Kit^®, Antenna Toolbox™ (glTF/GLB, STL).

• 3D printing and manufacturing — Slicing software and additive manufacturing workflows (STL, 3MF, OBJ).

• Web and AR/VR — Web-based 3D viewers, Apple AR Quick Look, and mixed reality applications (glTF/GLB, USDZ).

• Film and visual effects — VFX pipelines and animation tools using Pixar Universal Scene Description (USD, FBX, DAE).

• CAD and engineering — Geometry exchange with CAD tools and computational geometry applications (STL, OBJ, PLY, OFF).

In the Asset Generator app, use the Export button on the Home tab to export assets to file or to the MATLAB workspace. When you export to the workspace, the app creates a Mesh object or Scene object that you can further process programmatically.