CES 2024: Sony announces development of spatial content creation system

Sony Corporation has announced the development of an immersive spatial content creation system, which includes an extended reality (XR) head-mounted display equipped with high-quality 4K OLED microdisplays and video see-through function, and a pair of controllers optimised for intuitive interaction with 3D objects and precise pointing.

The latest spatial reality displays are now providing highly realistic, three-dimensional (3D) content without the use of special glasses or virtual reality (VR) headsets, for spatial content creation and review process. By introducing this system and collaborating with 3D creation software developers, Sony aims to further empower spatial content creators to transcend boundaries between the physical and virtual realms for more immersive creative experiences.

The system is aimed at supporting creators in sophisticated 3D content creation. Sony plans to collaborate with developers of a variety of 3D production software, including in the entertainment and industrial design fields.

In addition to video see-through functionality and spatial recognition, with six cameras and sensors in total, the system features a ring controller that allows users to manipulate objects in virtual space, as well as a controller that enables precise pointing, allowing creators to craft in virtual space with controllers and keyboards, all while wearing the head-mounted display.

Sony said users would not only be able to see real-scale 3D models in an XR environment with the high-definition display, but also create and modify 3D models in it. Through integration with third-party 3D creation applications, it also supports remote, real-time review between locations, making it an immersive tool for the entire spatial content creation process.

Sony has released technologies that support workflow and quality of spatial content creation and facilitate use of 3D computer graphics in various creative fields. Sony’s mobile motion capture system “mocopi”, with small and lightweight sensors and a dedicated smartphone app, enables full-body motion tracking using Sony’s proprietary technologies.

The device features the Snapdragon XR2+ Gen 2 Platform processor by Qualcomm Technologies. Said to be able to unlock a next generation of mixed reality (MR), the purpose-built platform places premium MR and VR technology into a single chip architecture for thinner and more comfortable headsets.

Qualcomm said the platform is designed to “unleash” a new tier of more realistic, detailed experiences in MR and VR, with 4.3K spatial computing at 90 frames per second. It also boasts higher GPU frequency by 15% and CPU frequency by 20%. Supporting 12 or more concurrent cameras with on-device artificial intelligence (AI), Snapdragon XR2+ Gen 2-powered devices can track the user, their movements and the world around them for navigation and experiences that merge physical and digital spaces.

At launch, Sony is exclusively partnering with industrial technology leader Siemens to introduce a solution for immersive design and collaborative product engineering using software from the Siemens Xcelerator open digital business platform. The system will be available later in 2024.

“Siemens is partnering with Sony to enable immersive engineering, a critical building block for the industrial metaverse. Together, we are creating an environment where it will be possible to experience the realities of physics, without the bounds of time to profoundly improve how our customers work and collaborate,” said Cedrik Neike, member of the managing board of Siemens AG and CEO of Siemens Digital Industries.