News · Apple
Vision Pro's Next Leap: visionOS 2 Unlocks 3D Object Tracking
Apple's latest software update elevates spatial computing by enabling app developers to integrate real-world 3D objects with digital content.
A developer working on a Mac with code related to visionOS 2 and a 3D model icon
While not a headline feature during WWDC24, visionOS 2 introduces a profound capability for Apple Vision Pro: 3D object tracking. This expands the headset's understanding of its environment.
Previously, Vision Pro could only track predefined 2D images. Now, developers can designate physical 3D objects for the system to recognize and overlay digital information.
The process involves a developer pre-supplying 3D models in USDZ format. Apple's Create ML tool then trains a neural network specific to each object.
This means a coffee mug, a board game, or a piece of furniture could become a dynamic element within an AR application. The potential for interactive tutorials, gaming, and utility is immense.
Beyond 3D object recognition, visionOS 2 also brings notable improvements to hand tracking and overall scene understanding.
Apple is clearly laying the groundwork for increasingly sophisticated AR applications, moving beyond flat interfaces into truly interactive mixed reality.
Share this story
