Apple has been exploring virtual reality and augmented reality technologies for more than 10 years based on patent filings. However, with virtual and augmented reality exploding in popularity with the launch of ARKit, Apple’s dabbling is growing more serious and could lead to an actual dedicated AR/VR product in the not too distant future, such as Apple smart glasses.

Apple is rumoured to have a secret research unit with hundreds of employees working on AR and VR and exploring ways the emerging technologies could be used in future products such as apple smart glasses. VR/AR hiring has ramped up over the last few years, and Apple has acquired multiple AR/VR companies as it furthers its work in the AR/VR space.

Former Apple hardware engineering Chief Dan Riccio in January 2021 transitioned to a new role in which he is overseeing Apple’s work on an AR/VR headset. The project has faced development challenges, and Apple executives believe that Riccio’s focus may help. Though he is overseeing the overall project, Mike Rockwell continues to lead the day-to-day efforts.

Apple is said to be working on multiple virtual and augmented reality headset prototypes as engineers search for the “most compelling application” for such a device, and current rumours indicate that Apple’s first product will be an augmented/virtual reality headset and/or apple smart glasses.

Apple is rumoured to be working on at least two AR projects that include an augmented reality headset set to be released in 2022 followed by a sleeker pair of augmented reality glasses coming at a later date. Many rumours have focused solely on the glasses, however, leading to some confusion about Apple’s plans, but it appears the headset will be the first product launched.

Apple analyst Ming-Chi Kuo believes Apple’s “mixed reality” headset will come out in 2022, with the apple smart glasses to follow in 2025. The headset is AR/VR, while Apple’s Glasses are augmented reality.

AR/VR headset

Websites The Information and Bloomberg have both said that the company is working on apple smart glasses and an AR/VR headset, with the headset to come out in first followed by the glasses. The headset is rumoured to be similar to Facebook’s Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable.

It’s said to feature two high-resolution 8K displays and eye-detecting cameras that will let users “read small type” and “see other people standing in front of and behind virtual objects.” The headset will be able to map surfaces, edges, and dimensions of rooms with “greater accuracy than existing devices on the market”. It will have a more advanced display and a chip that’s even faster than the M1 processor in 2020 Macs.

Many of the initial rumours suggested the headset would be an AR headset, but Bloomberg in January 2021 reported that it would be a mixed reality headset positioned as a “pricey, niche precursor” to a more ambitious augmented reality project. It will mostly focus on virtual reality functionality, but will have some augmented reality capabilities.

The Information in February said that it will be equipped with more than a dozen cameras for tracking hand movements in addition to cameras for seeing the outside world. They also reported that the design blocks out the user’s peripheral vision to prevent light from leaking into the wearer’s field of view. There could also be an outward-facing visor built into the display of apple smart glasses to allow the wearer to show graphics to others.

Apple analyst Ming-Chi Kuo has said that Apple’s upcoming mixed reality headset will feature 15 optical camera modules in total. Eight of the 15 camera modules will be used for see-through augmented reality experiences, six modules will be used for “innovative biometrics,” and one camera module will be used for environmental detection.

Kuo has also said that Apple’s headset will be equipped with an advanced eye-tracking system used to provide an intuitive visual experience that interacts seamlessly with the external environment, along with more intuitive operation that can be controlled with eye movements and reduced computational burden in the form of a reduced resolution where the user is not looking.

There is no word yet on whether apple smart glasses will be able to support iris recognition, but Kuo says that based on hardware specifications, an iris recognition feature should be possible. If this can be implemented, Kuo expects users will be able to use it for a “more intuitive Apple Pay method” when using headsets.

Design-wise, the headset has been described as having a “sleek, curved visor attached to the face by a mesh material and swappable headbands.” One headband reportedly features spatial audio technology similar to the one used in AirPod Pros for a surround-sound-like experience, while another provides additional battery life while on the go. Apple is working on multiple control methods, including a “thimble-like device to be worn on a person’s finger.” The headset would also be able to respond to the wearer’s eye movements and hand gestures, while one prototype of the headset also featured a physical dial on the visor’s side. Apple has also been testing Siri input.

Apple wants to create an App Store for the headset, with a focus on gaming, streaming video content, and video conferencing. It has been described by Bloomberg as an “all-encompassing 3-D digital environment” designed for gaming, video watching, and communicating.

Current headset prototypes are said to weigh nearly 200 to 300 grams, but Apple is aiming to reduce the final weight to 100 to 200 grams if technical problems can be solved, which would make the headset lighter than existing VR devices. The headset will be portable with independent power and storage, but it won’t be “mobile” like an iPhone. It will provide an “immersive experience” that surpasses currently available VR products, and it could be integrated with Apple TV+ and Apple Arcade.

Augmented reality functionality for the headset is “more limited,” and while a release is planned for 2022, Apple is said to have hit several development hurdles and has “conservative” sales estimates for the device. Apple expects to sell approximately 180,000 units, which is on par with pricier devices like the Mac Pro. The headset is expected to be much more expensive than headsets from other companies, with The Information reporting that Apple has discussed pricing the headset at nearly $3,000.