After back-to-back years of bringing unique, compelling concepts to the floor of the Consumer Electronics Show, BMW is turning inward in 2024 and focusing on the interior of the vehicle — namely, the types of technology and media content the automaker thinks its customers will want to interact with as they drive their high-priced, high-powered vehicles into the future.
In a statement, Frank Weber, chief technology officer and member of the board of management responsible for BMW Group development, said that the company was showing “more content, more customization, and more gaming.” And coming soon, the company’s operating system will feature “perfectly aligned” augmented reality and “strong, reliable” AI to allow better communication between the car and the driver.
Practically, this means that BMW is bringing more streaming video content and gaming choices to its vehicles. For its future lineup, the automaker claims that its customers will be able to use previously unavailable technologies, including augmented and mixed reality and in-car voice assistants driven by generative AI. No barf bags needed.
BMW Operating System 9, also known as iDrive 9, rests on Google’s Android Automotive software. Customers who purchase the optional “BMW Digital Premium” package will get access to a range of third-party apps from the company’s ConnectedDrive store, including gaming on their vehicle’s infotainment display while stopped or charging at an EV station. Assumedly, back passengers will be able to play as well using massive screens such as the BMW i7’s 32-inch drop-down display.
And now, for the first time, BMW drivers will be able to use real video game controllers when playing. The automaker says “any standard gaming controller with Bluetooth capability” can be used as long as the customer’s vehicle supports Operating System 9. Over-the-air software updates that will allow video game controllers are coming in 2024, BMW says. One such game that can be played using controllers is Beach Buggy Racing 2, a mobile game offered in both the iOS App Store and the Google Play Store.
BMW is now the latest carmaker to bring gaming to its cars. The trend was kicked off by Tesla, which has gone faster and further (sometimes too far) in giving gaming options to its customers. Tesla users can use game controllers or even the car’s own native controls, like the steering wheel, to play certain games.
Using the company’s preexisting AirConsole app, vehicle users can choose between single-player and multiplayer games. “Casual” games in which users can use their smartphones as controllers will also be available.
Of course, in-car distractions are at an all-time high, and BMW will have to put guardrails in place to ensure that vehicle owners aren’t able to abuse these features while driving or that passenger gaming isn’t distracting the driver.
If you’re not a hardcore or even casual gamer, perhaps you’ll find BMW’s expanded video streaming choices more up your alley. The company is updating its in-car video app for cars running Operating System 8.5 and 9, including on-demand and live offerings. BMW is bringing these expanded choices to more vehicles in 2024, such as the BMW X2 and all models in the new Mini lineup.
BMW’s i7 sedan — the one with the huge 31-inch drop-down movie screen for rear passengers — includes embedded Amazon Fire TV. The company is now trying to integrate something called “DTS AutoStage Video Service powered by TiVo,” which it says includes “both live channels and on-demand media libraries covering the categories news, sport, entertainment for kids, movies and TV series.” That system was first launched with BMW’s 5 series and will be coming to more “executive and luxury” cars via an over-the-air software update in 2024.
Augmented and mixed use reality
Automakers have been trying to make AR apps for their cars for a very long time. But now, BMW is claiming to be among the first to get it right.
The company says it will show this capability “for the first time” at CES in Las Vegas. (Our producer Becca Farsace has heroically offered to try it out at the show, so look out for that. Prayers up she doesn’t get too sick.)
Passengers will be asked to don a pair of Xreal Air 2 AR glasses, which use Micro OLEDs to project a 1920 x 1080 pixel display in front of your eyes. In this way, they will “see how navigation instructions, hazard warnings, entertainment content, information on charging stations and supporting visualizations in parking situations are embedded perfectly into the real-world environment.”
How perfectly? TBD. You’ll remember that BMW is also working with Meta to alleviate any issues of interacting with virtual objects while moving through the real world at high speed. Normally, AR and VR apps would have an extremely difficult time working in a moving vehicle, but BMW says it has figured it out. (I think it’s worth quoting the company in full so you can properly understand the level of trust we’re talking about):
The BMW Group has been working on the development of AR/MR technology since 2008 and investigating possible uses for head-mounted displays in research projects. The BMW engineers, together with their development partners, have now achieved in showing stable augmented reality and mixed reality content to passengers in a dynamically moving vehicle for the first time. The displayed picture stays securely embedded in the environment even in challenging driving situations, such as when turning, driving over large bumps or accelerating. To achieve this, the tracking system of the AR/MR device in question has to be tied to the vehicle’s sensor data.
BMW says it wants to build a “industry standard” for how AR and MR can be used in vehicles, pointing to its work with digital keys as an example of how it would approach such work.
The company thinks that rear passengers, such as luxury vehicle owners who are often chauffeured to work, will want to wear Xreal’s AR glasses to do work or join in virtual meetings. Gaming is another possible application.
Generative AI and LLMs
BMW says it is making progress on using Amazon’s Alexa technology to build its own custom voice helper, which it first announced back in 2022. The company says it’s ready to show off its work so far in advance of “a potential series rollout.” (In other words, still under development and no sure product to announce yet.)
Using Alexa’s large language model as its foundation, BMW’s Intelligent Personal Assistant will be able to respond to “quick instructions and answers about vehicle functions in a much more human, conversation-like manner, while at the same time being able to control some vehicle functions.”
To be sure, Alexa has been available as an in-car voice assistant in BMW cars since 2018. But BMW says its current lineup lacks “complex processing capabilities” and “human-like interactions and dialogs,” but that could be coming soon thanks to LLMs.
Automakers across the industry are scrambling to build stronger links between their voice assistants and core vehicle functions. What’s interesting here is that BMW is sticking with its plan to use Amazon’s Alexa as its basis rather than shifting to Google’s AI products, which could possibly complement the automaker’s Android-based operating system.
Lastly, BMW says it’s working with car supplier Valeo on a “automated valet parking” feature that also could work via teleoperation.
According to the automaker, customers would leave their cars in designated drop-off areas. The vehicle would then search for a vacant parking spot itself, maneuver into the space, and later drive back to the pick-up area for collection by the driver. “Either an autonomous driving system or a teleoperated valet system can be used as the underlying technology for this,” BMW says.
The challenges of teleoperation are myriad, largely around keeping a strong connection between the remote operator and the vehicle without any latency. Most autonomous car companies include some form of teleassistance, but full-blown remote operation remains a mixed bag.
Like most automakers, BMW has been working on self-parking technology for a bit, even getting credit from the US Department of Transportation for it at one time. But self-parking is tricky, and while all companies have some self-parking or even automated parallel parking system available, it would be a stretch to say that it’s entirely bug-free.