The growing demand for advanced vehicle features and zero-emission vehicles puts automotive engineering teams under increasing pressure to incorporate new technologies at an ever-faster pace. To meet these demands, more and more teams are moving to AI.
The rise of AI over the past decade has produced technologies that can be used at each stage of the engineering workflow. For example, machine learning models can be used to mine historical fleet data to inform key decisions at the design stage; they can also be deployed on the vehicle as part of an advanced driver assistance system (ADAS).
Many recent innovations would not be possible without AI techniques. But not all AI techniques are new to automotive engineers. For example, advanced statistical models (machine learning models in today’s parlance) have long been used to characterize test cell data in calibration workflows.
This paper shows how automotive teams use AI to:
- Build innovative features
- Enhance existing products
- Improve development workflows
- Interpret real-world data
- Perform process improvement and service enhancement
Case studies provide real-world examples of how engineering organizations are using AI in automated driving, powertrain and controls, fault diagnosis and prediction, and manufacturing.
Building Innovative Features
Machine learning is best known for its role in automated driving, but it has also made its way into powertrain and controls applications. In these applications, machine learning models can be used to provide estimates of hard-to-measure states such as driving style or component wear. These estimates can be used in feedback controllers to improve vehicle performance or efficiency.
This section features case studies from:
- Mahindra
- Caterpillar
- Continental
- PathPartner
- Autoliv
Mahindra Truck and Bus: Using Machine Learning to Estimate Road Surface Conditions
Mahindra Truck and Bus Division developed a tool that uses data from sensors on the vehicle, applies signal filtering and sensor fusion to generate features from the data, and then applies a machine learning model to predict the road surface condition. This prediction is incorporated into other algorithms and is used to improve the fuel economy of the vehicle.
Modeling a road condition estimator using machine learning at Mahindra.
Modeling a road condition estimator using machine learning at Mahindra.
Modeling a road condition estimator using machine learning at Mahindra.
Modeling a road condition estimator using machine learning at Mahindra.
Automated Driving
Perhaps the field where AI techniques have had the largest impact is in automated driving. The ability of machine learning and deep learning models to perceive the surrounding environment has made them an essential tool in the push to higher levels of autonomy. These models are used to translate data from cameras, lidar, radar, and other vehicle sensors into meaningful representations of the environment. Building such models requires a few foundational pieces: labeled ground-truth data that can be used to train models, algorithms for fitting models to data, and a framework for validating the performance of the models. Because real-world data plays an important role in building such models, much of the work involved relates to obtaining and curating the data sets.
Caterpillar: Managing Ground-Truth Data
Caterpillar, in collaboration with MathWorks, developed a big data infrastructure with a web front end to leverage external labelers; a database for searching and retrieving labeled ground truth; and an interface that enables function developers to use the labeled ground truth for training, validating, and deploying classifiers.
Ground-truth labeling infrastructure at Caterpillar.
Continental: Traffic Sign Recognition from Camera Data
Applying AI techniques to camera data requires a complete workflow from data curation through model development and deployment. Tools are also needed to visualize the performance of the model to understand where its strengths and weaknesses are. Engineers at Continental developed a tool chain to label ground-truth data, inspect recorded scenes, and develop and validate machine learning algorithms for traffic sign recognition.
Traffic sign recognition for driver assistance systems at Continental.
PathPartner: Classifying Objects in Radar Point Clouds
Engineers at PathPartner applied machine learning to radar data in an application that can detect pedestrians and other vulnerable road users in conditions where cameras may be insufficient, such as at night, in inclement weather, and at a distance. They used the Classification Learner app to quickly assess multiple machine learning algorithms and determine prediction accuracy.
Developing machine learning algorithms for radar-based automotive applications at PathPartner.
Autoliv: Object Detection from Lidar Point Clouds
Engineers at Autoliv used deep learning to detect objects in lidar point clouds, enabling them to identify objects in zones that are undetectable by other sensors. With deep learning, the team significantly reduced the time to manually label and analyze lidar data.
Lidar-based sensor verification at Autoliv.
Enhancing the Performance of Existing Products
The use of AI in a project need not exclude other techniques. AI can be used to augment classical techniques, such as those based on first principles or physics. For example, a machine learning model might be integrated into a control strategy only in a region where it is known to be more accurate than existing methods. In fact, for testing purposes, it is common to have AI models running alongside established algorithms so as to capitalize on the advantages of each approach.
Using AI to enhance existing products means that teams can rely on the strong foundation that they have built over many years while augmenting their offerings with new technology.
This section features case studies from:
- BMW
- Cummins
- Vitesco
BMW: Detecting Oversteer
BMW developed a machine learning model for detecting oversteer, a situation in which a vehicle’s rear tires lose their grip while navigating a turn. Using data collected from a driver on a track, the team analyzed the data from various sensors on the vehicle to identify the ones that would be most useful for detecting oversteer. They then trained a machine learning model that could perform this detection on new data and deployed it to the ECU for on-vehicle testing. This on-vehicle testing not only proved the machine learning workflow was viable, but also enabled the team to assess the performance of the overall traction control system and understand the improvements compared with the previous design.
Detecting when a vehicle is in an oversteer situation using machine learning at BMW.
Cummins: Developing System Identification Models for Control Design
Cummins is researching the use of machine learning to enhance classical control design. Their investigation includes the use of machine learning to improve system identification models for model predictive control and combining data-driven techniques such as deep learning and reinforcement learning with PID-based control.
Deploying reinforcement learning to augment classic control design at Cummins.
Vitesco: Reinforcement Learning for Emissions Reduction
Vitesco applied reinforcement learning in an emissions application. After creating a detailed model of the plant (consisting of the engine and exhaust gas system), they prototyped, generated, and optimized reinforcement learning agents to improve the control strategy for an exhaust gas aftertreatment system.
Applying deep reinforcement learning in powertrain control at Vitesco.
Improving Development Workflows
As products become more complex and timelines shrink, it is increasingly difficult for engineers to explore the complete design space. Computerized models (whether physics based or data based) can significantly reduce the amount of real-world testing that needs to be performed. Although models developed from physics and first principles are often preferred for their white-box nature, they can be too computationally inefficient to be used in large tradeoff studies.
In this case, a machine learning model can serve as a surrogate that captures the dynamics of the physics-based model but is more computationally efficient. For example, a deep learning model can be tuned to emissions data generated from a high-fidelity engine model and then used to develop an aftertreatment controller.
One of the most mature uses of surrogate modeling is in modeling engine behavior for model-based calibration. Experiments performed in a test cell generate data that captures the engine’s response to various input conditions. Machine learning models such as Gaussian process models or radial basis functions are then “fitted” to this data. The resulting model captures the engine behavior and can be executed quickly on a computer, enabling optimization of engine calibration tables.
This section features case studies from:
- Mazda
- GM
- Daihatsu
- Renault
Mazda: Using Model-Based Calibration to Reduce Testing and Increase Model Accuracy
Model-based calibration consists of using traditional statistics methods to define a test plan, capturing data from an experimental or simulated setup, fitting a data-based model to the data, and then optimizing system performance using the data-based model as a proxy for the actual system. Using model-based calibration techniques, Mazda engineers minimized the testing workload, reduced the engine model complexity, and increased the accuracy of their model for predicting smoke from the exhaust pipe.
Speeding next-generation engine development of SKYACTIV TECHNOLOGY at Mazda.
GM: Using Model-Based Calibration for Electric Drive Systems
The same model-based calibration techniques are also being used to calibrate electric drive systems. General Motors calibrated their current reference generation tables to determine the optimal current command for each speed/torque/voltage combination in an electric drive system. They used design of experiments to come up with an optimal testing plan, then fit a variety of machine learning models to the experimental data to model the current as a function of the inputs. These current models supported rapid design exploration, enabling the team to perform an optimization that generated the current reference tables to be used in the vehicle software.
Using model-based calibration to calibrate the current reference generation tables for an electric drive at GM.
AI-Assisted Testing
AI techniques can be used to reduce the amount of human oversight needed during testing. Using historical test data, models can be trained to identify desirable or undesirable behavior while a test is running. Because AI models can be trained on sensor data such as images, audio, or video, they can be used to perform real-time classification of testing criteria.
Daihatsu: Identifying Engine Knock
Identifying knocking is a task that has traditionally been performed by skilled testers. Daihatsu engineers developed an approach that used acoustic analysis to extract features from audio signals. These features are then fed into a deep learning model that classifies engine knock. This approach can judge the knocking sound with the same accuracy as skilled workers.
Classifying engine sounds using AI at Daihatsu.
Reducing Emissions
Legacy automakers are under increasing pressure to make zero-emissions vehicles (powered by batteries or fuel cells) while continuing to develop conventional vehicles equipped with internal combustion engines (ICEs). Stringent clean air regulations are driving the development of more complex ICE- and catalytic converter–based systems and on-board diagnostics (OBD) systems. These systems tend to be nonlinear, and developing, tuning, and testing them requires considerable resources. AI approaches to developing data-driven models offer a viable alternative by enabling the increased use of virtual environments for development.
Renault: Developing an Exhaust Gas Estimator
Traditional NOX estimates come from lookup tables or combustion models. However, lookup tables lack accuracy and combustion models are computationally expensive. Engineers at Renault used a deep learning network to model engine-out NOX. They collected experimental data from an actual engine with common drive cycles, and iterated over many deep neural network configurations to find one that suitably modeled the engine-out NOX. They then used this model as part of a nonlinear observer for control design and for desktop simulation.
Developing an NOX emissions simulator at Renault.
Interpreting Real-World Data
Machine learning algorithms recognize patterns in large data sets, making them a natural fit for the ever-increasing amount of real-world driving data that is available. The trends and patterns extracted from this data can be used for evaluating engineering designs, vehicle calibration, infrastructure planning, and developing new products and services.
This section features case studies from:
- Honda
- Volkswagen
- Ford
Honda: Evaluating Emissions Aftertreatment Systems
Engineers at Honda analyzed 1 million kilometers of driving data from 1,000 vehicles to inform development activities for meeting emission regulations. They developed a pipeline to preprocess and filter the raw data then extract features including speed, mileage, gear selection, AC utilization, and traffic conditions. The statistical distributions derived from this analysis enabled the engineers to ensure that their emissions aftertreatment systems met performance requirements.
Using fleet analytics and MATLAB to build strategies for BS-VI development at Honda.
Volkswagen Data Lab: Analyzing Driving Patterns
Volkswagen Data Lab analyzed how telemetry data varied from driver to driver and developed a classifier that could automatically determine who was driving based on data from the vehicle sensors. They used CAN bus data from the vehicle and extracted various statistical features from the time series using sliding windows. This approach enabled them to do statistical recognition of the driver, providing a basis for potential business models such as “pay-as-you-drive.”
Classification of individual driving behavior at Volkswagen Data Lab.
Real-World Fleet Data
Connected vehicles generate real-world driving data that can be used in many aspects of vehicle design. The amount of data can be so large that specialized IT systems are required to store it.
Ford: Accessing and Analyzing Fleet Data
A team at Ford connected MATLAB to their fleet data stored on Apache Spark™ so that engineers can access and analyze this data with familiar tools. This data enables ADAS engineers to understand how a feature is performing in the real world and to generate scenarios that can be used for simulation and validation.
Using MATLAB on Apache Spark for ADAS feature usage analysis and scenario generation at Ford.
Process Improvement and Service Enhancement
AI techniques are unlocking new opportunities for process improvement and service enhancement. Predictive maintenance and anomaly detection techniques are being adopted by manufacturing groups for early detection of issues with production lines. AI algorithms can provide an early indication of manufacturing quality (reducing future scrap rates), and predict failures in manufacturing equipment before they occur.
This section features case studies from:
- Mercedes-Benz
- Baker Hughes
Mercedes-Benz: Detecting Machine Cycle Errors
Engineers at Mercedes-Benz applied anomaly detection techniques to detect error cycles in manufacturing production parameters. They used time-series techniques to identify characteristics of machine cycles and then performed statistical analysis to characterize normal and anomalous cycles.
Pattern matching for time-series manufacturing data at Mercedes-Benz.
Predictive Maintenance
Predictive maintenance techniques are also being used for on-vehicle applications. Especially in the commercial vehicle space, unplanned downtime can be costly for equipment operators. The traditional way to address this problem is to schedule routine maintenance far in advance of a failure occurring. However, this approach requires extra labor for the maintenance and may result in the scrapping of parts that have remaining useful life.
Baker Hughes: Predicting Pump Failure
Baker Hughes used machine learning techniques to develop a predictive maintenance algorithm that can predict the failure of pumps on oil extraction equipment, enabling them to reduce unplanned failures and save an estimated $10 million a year.
Baker Hughes’ predictive maintenance software for gas and oil extraction equipment uses data analytics and machine learning.
AI with MATLAB and Simulink
MATLAB® has long been used for data analysis and algorithm development. In many of the examples described above, MATLAB was chosen for its ease of use and the variety of domain-specific tools it provides. The use of machine learning techniques on automotive sensor data often requires domain-specific preprocessing to extract the right information from the raw data before it can be passed into a machine learning algorithm. For example, frequency domain content may be extracted from filtered signals in order to create features that a machine learning model acts upon.
In MATLAB, engineers can combine domain-specific techniques such as controls, signal processing, image processing, and lidar processing with machine learning techniques to create models that are more intuitive and robust than completely black-box models.
With Simulink®, engineers can understand and analyze complex systems by simulating block diagrams. Simulink provides blocks for machine learning and deep learning, enabling engineers to combine AI techniques with Model-Based Design.
AI presents many opportunities for automotive engineering teams. While there are still unexplored areas of automotive engineering where AI could provide value, the applications shared in this paper provide ideas for teams looking to get started with AI.
