“If I had a chauffeur”

100 years ago, refrigerators were not widely used in society. Back then it was not always possible to eat cold and sweet tropical fruit. 

Today, thanks to the invention of refrigerators, food can be stored fresh and safe at home. The evolution of technology has made something that was considered rare or special into something that many people now take for granted.

What will become “common” in the next wave of technological evolution? What about having a chauffeur? This is a value that only a small fraction of people can enjoy today.

With a chauffeur service, travel will be very convenient and comfortable. You will be taken to your destination using the route you prefer with comfortable and careful driving.

A chauffeur also has the flexibility to monitor the weather or the physical condition of the vehicle’s passengers and make driving adjustments.

We would like to make this luxury that only a few people currently enjoy into something more “common”. Through reliable automated driving, everyone can experience the pleasure of using a special means of transportation.

How can automated driving become more widely used?

  • self-driving-02

Technology evolves rapidly as various enterprises around the world work to shape the future of automated driving. How do we integrate these evolving technologies into our daily lives? By building trust between people and automated driving technology. 

In general, there are three aspects that we must consider in order for automated driving technology to be accepted and more widely used in society. 

① Further increase in user value
Automated driving must offer users convenience, security and economic benefits.

② Reduction of social loss 
It must offer solutions to problems that are commonly attributed to automobiles, such as the reduction of the number of automobile accidents and the resolution of traffic jams.

③ Locus of responsibility
It must be clear where the responsibility lies in the event of a car accident.

There are challenges to be tackled for each difficulty.
All of these challenges have to be addressed to make automated cars less of a concern and more enjoyable for people. In particular, the factors that affect technological development are the increase in user value and the improvement of infrastructure. 

Technology developments to solve these problems have been conducted in the past, but we believe it is necessary to develop technologies that understand human characteristics more deeply. People will not choose automated driving cars simply because they are safer. People feel uneasy when too much priority is given to safety while the behavior of a car becomes unstable. For example, in the situation like this, people become sensitive to small changes in gravity (acceleration).

To be trusted, automated driving must have a deep understanding of human characteristics, such as how people feel and act in certain situations. That way, people will feel comfortable while leaving the driving task completely to automated driving cars. To this end, DENSO is also focusing on research to understand human characteristics.

Simulating human senses using technology for automated driving

Hiroshi Ino, General Manager of the Vehicle System R&D Department, Advanced Mobility Systems Business Development Division of DENSO CORPORATION says, "For safe and comfortable automated driving, it's important to understand what people are feeling when driving, and make sure to simulate it through engineering."

  • self-driving-03

It is said that people understand how fast they are going by the flow of the scenery. This concept is called "optical flow" in the field of computer vision.

Drivers judge the distance from the car in front of them not based on distance or speed, but based on a perceived speed that is different from the actual speed. They are actually calculating, "How many seconds will it take for this car to hit the car in front?" 

The technology that utilizes this research into human characteristics is called Adaptive Cruise Control (ACC) which enables the vehicle to automatically adjust acceleration and deceleration within a preset speed to control the distance between vehicles. This is not the only area DENSO has researched human characteristics and simulated them through engineering.

  • self-driving-04

There is a concept called "friction circle" in tires. When a car is driving on a curve, friction occurs between the tires and the road surface when braking or steering while accelerating. Friction occurs in both the front, back, and side directions, but the maximum friction force remains the same. The friction circle shows it as a conceptual diagram.

When the entire friction circle is represented by a bowl and the gravity produced by a tire is represented by a ball, the ball drifts out or spins out of the bowl. In the conventional safety system, if the tire is loaded until the ball reaches the end of the bowl, the tire condition is forcibly returned to ensure safety.

When safe and secure driving is performed, a state called “ball in bowl” is realized in which gravity is controlled efficiently within the limit of vehicle dynamics. 
We analyze these mechanisms from the viewpoint of automotive engineering and reproduce the human senses by engineering to develop control systems.
In this way, DENSO is trying to realize comfortable and safe automated driving technology, as if it were a professional driver taking the wheel.

As a matter of fact, the results of experiments on a mountainous test course using this technology showed that even though it was a difficult course with a series of curves, we were able to control the gravity within the limit of the friction circle, giving the occupants peace of mind.

  • self-driving-05

These images show how gravity is applied to the car body when it is driven by different drivers.
According to the “ball in bowl” concept, the closer the drawn lines are to the shape of a small circle, the more comfortable the driving is. The upper left is the case of automated driving. It can be said that the driving is more comfortable compared to other drivers since the shape of the drawn lines is closer to a small circle.

The strength of DENSO, which conducts R&D with a thorough knowledge of automobiles, is its ability to verify technology on the spot, just like the mountainous test course. We take advantage of this strength to realize automated driving that incorporates human senses.

Ino explains, "As automated driving becomes more widespread in society, humans and cars must work together to drive. How can we seamlessly transfer the driving control to machines? This is a pretty interesting area of research." 

"As a researcher, there is no area that is more exciting." Ino says with a smile on his face, explaining why he's involved in the development of automated driving technology. 

Humans naturally predict motion

In order for machines to drive in the place of humans, we are also working on adapting AI technologies such as deep learning and machine learning to cars. In our daily lives, we anticipate the movement of objects and make adjustments to our own actions. The AI-driven "image recognition" is superior to humans in some respects, but it is not stable in some circumstances, such as irregular environments, and still has many challenges.


  • self-driving-06

Naoki Ito, General Manager of the Applied AI R&I Dept., AI R&D Div., Advanced Research and Innovation Center of DENSO CORPORATION says, "Even actions that human beings perform unconsciously can still be difficult in the case of a machine. In order to realize reliable automated driving, we need to tackle these challenges."

People recognize their surroundings unconsciously in their daily lives. For example, we recognize buildings, trees, pedestrians, and oncoming cars, and naturally understand whether objects will move or not. For many people, it is easy to recognize objects in a moving scenery. However, in order for AI to do the same thing, it is necessary to continue to follow (track) the trajectory of the object while detecting it in every frame. In situations where objects are hidden, current technology can make mistakes in detecting objects.

In order to solve this problem, DENSO has developed an algorithm for image recognition technology that refers to the past tracking information to estimate and detect the current frame position and improve tracking performance. This technology enables accurate detection of hidden objects.


This technology makes it possible to identify the positions of objects and obtain information about their past trajectories. Then we are motivated to predict the trajectories as the next step. For example, when pedestrians come into view, humans naturally predict "Where is this person going?" If AI can make this prediction, it could make automated driving more trustable.

DENSO is also using this technology to predict trajectories of pedestrians. In order for AI to predict "Where will this person go after this?" from past tracking data, it is important to predict the destination. To do this, we first use information such as the shape of the road and the location of buildings to predict multiple destinations. Then we estimate the routes that can be reached for each destination, and then predict the most likely route to be chosen.

  • self-driving-07-en

The video above shows the "pedestrian warning demonstration" which utilizes the results of pedestrian trajectory prediction. This defines the hazardous area around you and warns you if a pedestrian might enter the area. A red boundary box is indicated as a warning when the probability of a pedestrian entering the defined hazardous area is higher than a certain threshold. 

In the demo video, no warning has been issued yet, but since the pedestrian in the boundary box (green) is moving to the left of the screen, the result is shown in red. It is also making a prediction considering the possibility of this pedestrian going straight.

“Will the pedestrians go left or walk straight?” Both cases are predicted. In fact, the pedestrian turns left and the prediction converges to the left as well. AI technology are utilized to predict the trajectory of these pedestrians.

Ito says, "It's not easy for AI to reproduce what people do naturally. However, there are areas where AI is better. We have a lot of problems to solve for the social implementation of automated driving, so I think it's a challenging and interesting field of research."

For completely reliable automated driving technologies

  • self-driving-08

As automated driving becomes widespread in society, drivers will be the passengers. In that case, safety and comfort as well as trust will be required.

For automated driving to be implemented in society, it must be recognized as safe and secure for people. No matter how far technology evolves, acceptance will be difficult unless it is recognized as a reliable technology.

People can totally rely on a dependable automated driving car, as if it were a chauffeur at the wheel, and spend their free time on the road. To make that future a reality, DENSO is pursuing research and development into automated driving that takes human characteristics into full consideration.