6 AI-powered Applications for Providing In-Vehicle Comfort

This article was published as a part of the Data Science Blogathon.

Introduction to AI-powered Applications

Many a time while driving, we need to adjust things like in-vehicle seat relaxation function,  interior lighting, music, air-conditioning, fragrances, etc, manually while attempting to keep our eyes on the road. However, it is very inconvenient and sometimes it can potentially put the driver and the co-passengers at the risk. Furthermore, it becomes even more complex when a previous driver has different preferences for cabin temperature, audio playlist, and so on. In light of this, would not it be convenient if our vehicle’s heating, ventilation, illumination, music, visor, and air conditioning (HVAC) system could learn our individual preferences, and automatically make these adjustments for us?

Well, it’s possible!

In this post, we will take a look at six AI-powered applications for in-vehicle comfort offered by various companies. These AI-powered applications will not only provide comfort but will also improve the driver’s and the vehicle’s safety.

AI-powered Applications for Providing In-Vehicle Comfort

1.  TOYOTA’s LQ Car Delivers a Personalized Experience

                                                         Source: BusinessInsider

• Toyota came up with a concept LQ car that can deliver a personalized experience, and meet each driver’s unique mobility needs.  This concept car employs an AI agent called Yui which interacts with the user via interactive voice communication and detects the emotional and physical state of the driver.

• Further, the detected state is used to control the features of the car such as configuring a seat relaxation function, altering the interior lighting, music, air-conditioning, illumination intensity, and fragrances.

• To keep the car emissions in check, the automaker collaborated with Aisin Chemical Company and Cataler Corporation has also included an air purification coating that decomposes some of the car’s emissions while driving.

• Yui also regulates seat functions via inflatable “air bladders” and air conditioning systems that are embedded into the seats. The goal is to either keep drivers awake or relaxed. For instance, if the Yui detects that the driver is fatigued, it will inflate the back portion of the seat to make the driver sit straighter and pump chilly air from the seat’s ventilation system.

• The LQ car also has an autonomous driving function, which enables a self-driving valet parking system. This means that after dropping the driver off, the automobile can drive itself to an assigned parking space in a parking lot.

• Furthermore, the Augmented Reality Head’s Up Display (AR HUD) employs AR to expand the HUD’s information display. The display projects driving-related information such as lane warnings and the GPS semi-transparently on the windshield, mitigating eye movement.

                                                        Source: BusinessInsider

Roof and floor mats are also utilized as a means of communication between the car and its passengers. Yui, the AI agent, can alter the color displays to indicate whether the vehicle is in automatic or manual driving mode, and it can also light up different sections of the footwells to indicate which passenger Yui is communicating with. 

                                                          Source: BusinessInsider

The headlamps include a Digital Micromirror Device that displays road surface information to passengers both inside and outside the vehicle.

Read here| https://www.businessinsider.in/business/news/toyota-has-created-a-car-that-uses-ai-to-sense-a-drivers-state-and-can-even-wake-them-up-using-inflatable-cushions-or-cold-air-if-theyre-nodding-off/articleshow/71603360.cms

2. Continental AG partnered with Amazon to develop Personalized Algos

                                                             Source: Continental

• Continental AG partnered with Amazon Machine Learning Solutions Lab to develop personalization algorithms that learn from user behavior and automatically adjust the temperature for the driver to offer optimal in-vehicle thermal comfort.

• Considering people have different thermal preferences which can also vary dramatically depending on external environmental factors and other factors, offering optimal in-vehicle thermal comfort becomes quite challenging to accomplish.

• The multi-contextual personalization system developed during the ML Solutions Lab engagement utilizes Amazon SageMaker to create and offer an innovative suite of onboard personalization features for drivers.

• To prototype this solution, Continental Automotive Systems gathered several hours of real-world data using a test vehicle equipped with multiple sensors that measured, among other things, the outside temperature, cabin temperature and humidity, and sunlight. The goal was to capture the key variables and dynamics governing thermal comfort.  

Fig. 1: Diagram depicting the Simulation Scheme for adjusting the in-vehicle temperature (Source: Amazon)

• As shown in fig. 1, the simulation scheme includes a closed-loop system where the temperature-adjusting behavior influences the cabin temperature, which consequently indirectly affects the temperature-setting behavior. Both exogenous factors (orange) and endogenous factors (green) play a crucial role, and exponential temporal dynamics govern how these impact the cabin air temperature, near-skin temperature, and eventually the set temperature.

• While building the personalization system, the commonalities across users, as well as the differences among them, are leveraged in the following two stages:

1.  First, a neural network is trained using all data in an undifferentiated way. The inputs comprised both exogenous variables and demographic data.
2.  Second, the final layer of the neural network is fine-tuned for each individual, using only their data.

• To scale this solution to production roll-out, more real-world data is required.  Additionally, the simulation environment can be used to produce estimates on the amount of data required for collection. Ancillary modules are also required to identify the driver (and other occupants), as well as their clothing— based on these attributes a personalized profile is loaded.

• These modules can utilize a hybrid cloud-edge architecture, where model training workloads are run in the cloud, while inference and minor updates can be performed at the edge in a privacy-preserving way.

Read here| https://aws.amazon.com/blogs/machine-learning/achieve-in-vehicle-comfort-using-personalized-machine-learning-and-amazon-sagemaker/

3. Infotainment System by Dentsu and Hyundai in collaboration with the Audioburst

                                                           Source: Autocarpro

• Audioburst is a startup that utilizes AI technology to extract the best bits from podcasts and talk shows to create new personalized news briefs. Dentsu and Hyundai invested $10 million in the Audioburst project to develop an AI-powered infotainment system. Using automatic speech recognition (ASR) and other natural language techniques, this system enables the passengers to search music/audio libraries, enjoy personalized audio playlists and news briefs, etc.

• The startup ingests and indexes millions of audio segments each day, then employs AI techniques like Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) — to create a searchable music library, personalized audio feeds and news briefs, notifications, etc.

• Through an API, partners can integrate Audioburst’s tailored feeds into their smart speakers, mobile, in-car infotainment systems, and other products. Brands can also leverage the technology to target listeners with more personalized advertising based on an improved understanding of customers’ interests.

• However, Audioburst is not alone in working to create tailored listening experiences as a channel for advertising. For example, Spotify is also doing this with music. Pandora also allows advertisers to personalize their messages by leveraging listener data.

• Audioburst partnered with LGE as well, this partnership focused on developing new infotainment systems for automakers. This deal was the first to utilize Audioburst’s Deep Analysis API, which employs an additional level of metadata categorization to provide a deeper understanding of the content users searched for. Now the Hyundai deal will also leverage this API.

Read here| https://techcrunch.com/2019/04/22/audioburst-raises-10m-to-build-a-i-powered-infotainment-systems-for-cars-ad-solutions/

4. MBUX offers Personal Predictions and Recommendations

                                                          Source: Mercedes

• Mercedes-Benz has devised a User Experience infotainment system called Mercedes-Benz User Experience (MBUX) which is response based on the vocal or touch input from the driver.

• Through deep learning, the MBUX software learns from the driver’s routine to offer personal predictions and appropriate recommendations. Through deep learning, it travels through layers of ideas until it has learned from these concepts and begins to independently seek out new knowledge.

• Rather than simply submitting suggestions, MBUX software also takes driving behavior, tone of voice, and other external components into consideration and offers appropriate recommendations.

• MBUX also converges a touchscreen, thumbpads, and several touch-sensitive buttons to upgrade to a new UI on the steering wheel. Furthermore, it also has other capabilities like receiving messages about bad roads, petrol prices, weather information, vehicle location, and even allows changing of climate control settings. Furthermore, the company is also working to reduce software’s response time.

• Additionally, to make driver experiences more comfortable, Mercedes’ AI-powered OS also offers smart home integration, movie ticket purchasing as well as food delivery capabilities. While some of these verticals might require further iterations, the automakers are focusing on diversifying the range of these offerings to a wide range of countries.

Read here| https://www.cnet.com/roadshow/news/mercedes-benz-mbux-infotainment-voice-control/

5. Bosch – Virtual Visor

                                                              Source: Bosch

• Bosch developed a Virtual Visor, which can not only improve driver comfort but also aid in the reduction of accidents caused by sun glares. According to a study, sun glare causes thousands of automobile accidents each year, which is more than double the number of accidents caused by any other weather-related condition. Hence Bosch’s solution may be quite useful in avoiding accidents.

• This Virtual Visor comprises a transparent LCD panel and RGB driver-facing camera. It also utilizes intelligent algorithms to intuitively block the sun’s glare while allowing the driver to see the road ahead.

• Virtual Visor links an LCD panel (that could be fixed to a car’s windscreen) with a driver or occupant-monitoring camera to track the sun’s casted shadow on the driver’s face.

• The system then utilizes artificial intelligence to locate the driver within the image captured by the driver-facing camera. It also utilizes AI to determine the landmarks on the face ‒ including where the eyes, nose, and mouth are located ‒ so that it can detect shadows on the face.

• Furthermore, the algorithm analyzes the driver’s field of view, darkening only the section of the display where the light strikes the driver’s eyes. The rest of the display remains transparent, no longer obscuring a large section of the field of vision.

• Although it is still in the prototype stage, it has the potential to help avert innumerable traffic accidents.

Read here| https://www.bosch-presse.de/pressportal/de/en/safe-eyes-save-lives-how-bosch-engineers-are-innovating-the-95-year-old-sun-visor-204928.html

6. Affectiva-Smart Eye Interior Sensing

                                                             Source: Affectiva

• Many drivers have had the experience of attempting to alter settings in their vehicles while keeping their eyes on the road. This is often quite uncomfortable and can also put the driver at risk. To circumvent this, there is a pressing requirement to anchor a solution around this. In light of this, car manufacturers need to have a thorough understanding of what happens inside a vehicle, which is often termed, Interior Sensing.

• Affectiva’s Emotion AI evolved to measure not only the complex and subtle emotions and other cognitive processes, but also the in-vehicle activities: how they are interacting with the in-vehicle environment and with each other, what their activities are, and what objects they are utilizing? The deep learning-based software, which is powered by massive amounts of real-world data, employs in-vehicle cameras to measure the state of the cabin, as well as the driver and occupants, in real-time.

• In 2021, Smart Eye, the leading provider of Driver Monitoring Systems, high-fidelity eye-tracking technology, and interior-sensing, acquired Affectiva to integrate the two companies into a global AI powerhouse. 

• The deep expertise of Affectiva in machine learning, data acquisition & annotation, and AI ethics enable Smart Eye to provide Human Insight AI that understands, supports and predicts human behavior in complex contexts.

• In collaboration, these companies are working on dashboard technology that uses cameras to monitor the driver’s face for fatigue, and if detected, it can respond accordingly by perhaps playing music to wake them up. 

Conclusion to AI-powered Applications 

To sum it up, in this article we explored the following six AI-powered applications for providing in-vehicle comfort:

1. Toyota developed a concept LQ car which employs an AI agent called Yui that interacts with the user via interactive voice communication and detects the emotional and physical state of the driver. Based on the detected state, the LQ car delivers a personalized experience.

2. Continental AG partnered with Amazon ML Solutions Lab and developed a multi-contextual personalization system that utilizes Amazon SageMaker to create and offer an innovative suite of onboard personalization features for drivers.

3. Dentsu and Hyundai collaborated with Audioburt to develop an AI-powered infotainment system that utilizes AI techniques like Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) — to create a searchable music library, personalized audio feeds and news briefs, notifications, etc.

4. Mercedes-Benz has devised a User Experience infotainment system called Mercedes-Benz User Experience (MBUX). Its software takes driving behavior, tone of voice, and other external components into consideration and offers appropriate recommendations.

5. Bosch developed a [rotoypr for Virtual Visor that utilizes an algorithm that analyzes the driver’s field of view (FoV), and darkens only the section of the display where the light strikes the driver’s eyes.

6. Affectiva in collaboration with Smart Eye is working on a dashboard technology that can measure not only the complex and subtle emotions and other cognitive processes but also the in-vehicle activities to enhance road safety and make the mobility experience enjoyable.

I hope you liked my article on AI-powered applications. Share your feedback with me below in the comments section.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Drishti

I'm a Researcher who works primarily on various Acoustic DL, NLP, and RL tasks. Here, my writing predominantly revolves around topics related to Acoustic DL, NLP, and RL, as well as new emerging technologies. In addition to all of this, I also contribute to open-source projects @Hugging Face.
For work-related queries please contact: [email protected]