Connecting 130 Factories Worldwide with IoT :Behind the Scenes at DENSO’s Dedicated Efforts to Bring Development In-house
Apr. 3, 2020
On December 9, 2019, DENSO Corporation hosted DENSO Tech Links Tokyo #5, with the main theme for the fifth incarnation of the event being “DENSO’s In-house Software Development Starting from Its Factories.” DENSO’s factories, which have manufactured 12 billion automotive components of 44,000 different types, are supported by a wide range of IoT technologies. Technical advisor Takuya Oikawa and three other DENSO employees held presentations exploring what goes on behind the scenes of the company’s IoT technologies. This presentation featured Dan Yakabe, Yuta Kuroda and Hiroo Sawada from the DENSO Production Eng. Div. Factory IoT Development Team, who explained why, by whom, and in what ways they were moving development of factory IoT software in-house while providing specific case examples.
■Speakers
Dan Yakabe, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Yuta Kuroda, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Hiroo Sawada, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Why DENSO Pursued In-house Software Development Starting from Its Factories
Dan Yakabe: Hello everyone, my name is Dan Yakabe and I work for the DENSO Production Eng. Div.
The Factory IoT Platform we are creating is used to collect data on equipment, people and other aspects of factory operations provided by various devices, then convert this data into valuable information and provide it to users through applications.
This is our in-house system, and today I’m going to talk about why DENSO is pursuing in-house software development starting from its factories, and adopting the “Make It Your Own” approach. Yuta Kuroda will then explain who is carrying this out, and Hiroo Sawada will tell us how it is being carried out.
But first, allow me to begin with a brief personal introduction. In 2002, I joined a systems integrator company where I developed smart meter systems, HEMS, and other M2M systems. Later on, I made a mid-career change and joined one of the nation’s biggest banks where I developed security-related systems and APIs (application programming interfaces) for Fintech companies. My experience developing M2M technologies, security systems and APIs proved to be very useful when developing DENSO’s Factory IoT Platform for 130 different factories.
Manufacture of 44,000 Different Automotive Components
Yakabe: Let’s begin by talking about the development concept for our Factory IoT Platform. We often talk about CASE (connected, autonomous, shared and electric). Times have changed, from a focus on mass-producing high-quality products to an era in which organizations are required to tie together both equipment and software and provide services at increasingly rapid speeds in order to stay competitive. It is said that the automotive industry undergoes a once-in-a-century transformation.
In order to achieve the speed and revolutionary changes needed, “autonomy” is required to control progress. “Speed” means nimbly and quickly attempting new things, while the particularly important “co-creation” entails getting help from other organizations. These are the three concepts I want to focus on today.
On December 16, DENSO will celebrate its 70th anniversary [editor’s note: anniversary event was on December 9, 2019].
(Audience applause.)
I want to point out our R&D expenditure figures, which show that DENSO spent ¥510 billion on R&D to achieve ¥5.3 trillion in sales. Startups often celebrate reaching the ¥1 billion or ¥2 billion mark, but in DENSO’s case our numbers are two entire digits above that level, which enables us to pursue an extremely wide range of different things. DENSO employs 172,000 people, a majority of whom are based outside of Japan, making our work environments very international in character.
We have 130 factories—150 by some counts—which are located in almost all of the world’s continents. Furthermore, DENSO has pledged to grow its software engineer numbers to 12,000 by the year 2025—an ambitious goal which has led many to expect great things from our company.
This year we celebrate the 25th anniversary of the QR Code, a well-known technology we invented, and over the years we have also made a wide range of automotive components including air conditioners, motors for electric cars, and ADAS millimeter-wave radar sensors and monocular cameras, to name a few. We have produced 44,000 different types of automotive components, numbering 12 billion units in total. As a result, we possess manufacturing facilities capable of outputting large numbers of highly varying products, and this results in a highly diverse array of data sent in from our factories. This makes the development of an IoT platform quite difficult.
Using Digital Technologies to Achieve Co-creation Between People and Equipment, and to Accelerate Improvements
Yakabe: Now let’s look at our goals in creating the Factory IoT Platform. Our base concept is “using digital technologies to achieve co-creation between people and equipment, and to accelerate improvements.” The basic approach is to take processed results data from equipment related to operating conditions, product quality and so forth, and have people analyze this information via applications and feed the results back to the equipment.
The applications are generally expected to include machine learning, deep learning and other such analytical models along with sections providing visual displays of information, notification functions, and other standard features. Applications will be connected to a cloud computing network so the 130 factories around the world can make full use of them, after which we start making gradual improvements. In reality, though, this is very hard to pull off. Personally, I would call it bari-muzu.
I’m originally from Kyushu, where we say bari-muzu, which means “very difficult” in English. The reason it’s so difficult is because we want to achieve a lean production system at our facilities using the platform I just discussed, which takes in and manages data before outputting it again. DENSO is not the only one striving for lean manufacturing—a lot of Japanese companies want the same thing—but this approach does lead to huge numbers of demands for improvements at worksites.
I have implemented numerous rounds of PoC (proof of concept) testing and created a number of IoT platforms. At one factory where I provided only the data visualization and notification features, I received 400 improvement requests in just half a year’s time. The demands for new data input were endless: more data categories for equipment, a voice-recording feature, analysis of workers’ paths of movement, integration with other systems’ data, and so on.
Regarding application interfaces, users asked for a voice-operated search feature, more application subsections, slight changes to the search button location, data analysis cycle compatibility, and notifications via smartwatches, among others. Then there were the requests for coordination with upstream design information, and applications for mobility services, sharing and after-market areas—the list goes on.
Achieving Autonomy, Speed and Co-creation
Yakabe: Feeling a bit overwhelmed, I found myself relying on package-style product suites purchased from outside companies. I tried a number of these outside product packages, but things never went smoothly.
As I mentioned, I was taking in data from a number of different devices in relation to equipment information, sensor information, worker paths of movement and actions, and so forth. I wanted to get data via the platform and create new applications, but the need to consult the product vendor every time I input or output data was detracting from our autonomy.
A similar problem was occurring in regard to speed: every time I wanted to make a new application, I found myself being told that according to the roadmap, it was still a year or so away from being finished. Regarding co-creation, our partners had specific requirements and conditions for releasing databases to the public, and many of these partners were unable to work together with us as a result. In the end, DENSO was unable to move forward with development as anticipated.
An IoT platform requires three things in regard to data: collection, storage and usage. In order to achieve the necessary speed and co-creation during improvement cycles for these three areas while maintaining autonomy, these three things are necessities.
I’ll say it again: what we needed was autonomy, speed and co-creation. In the area of autonomy, for example, we used Java at the start but later replaced it with Golang, which led to greatly improved performance. Regarding speed, we began to use a public cloud network as well as Kubernetes.
So how was DENSO to move forward with making software on their own? How could we achieve effective co-creation? The answer to these was to move development in-house.
Yuta Kuroda will now talk about who it is that carried out this in-house transition.
Thank you for your time.
(Audience applause.)
Behind the Scenes at DENSO’s Full-Out Efforts Toward Moving Development In-house
Yuta Kuroda: Hello everybody. I’m going to talk about how a group of DENSO production engineering experts pursued full-out efforts toward internal, in-house development of the Factory IoT Platform.
I joined DENSO in 2008 and have worked here for 12 years, starting in production engineering and later moving on to core systems SE. Currently, I work in the Factory IoT Development Team, which is where I had always wanted to be. I’m really interested in Kubernetes, Ansible, CI (continuous integration) and the other infrastructure-related areas in which I'm involved.
In the slides, I used the example of Dejima to describe a special place when talking about agile development at large companies. Many people think that such development is carried out at special places in specialized organizations, and that only product owners are company employees while everything else is outsourced. Others may think what we do is just PoC without a specific goal. I want to explain how serious we are about this project.
I will go over the details during Cross Talk, so in this presentation I want to talk about who is carrying out our efforts, and just how dedicated those people are.
Breakthroughs to Remedy Production Engineering Trade-offs
Kuroda: To start with, what exactly is “production engineering”? Allow me to provide a bit of an overview. “Product design” entails deciding what we’re going to make; “production engineering” focuses instead on how we’re going to make it. Some people refer to this as “process design,” as it is carried out by the division or department in charge of designing the processes.
In real-life production operations, there are manual production lines where workers use their hands, and automated lines which utilize robots. In general, manual lines are less expensive and robotic lines are more expensive.
Robotic lines are faster and more precise, whereas human-operated manual lines are more flexible in terms of process changes. Making decisions regarding trade-offs between quality, cost and, delivery, as well as other factors, is one of the main goals of production engineering development efforts.
When it comes to production engineering at DENSO, the Production Eng. Div. not only thinks about trade-offs related to quality, cost and delivery, but also pursues technological breakthroughs in order to succeed to a high degree in all three of these areas.
We harness the power of software in order to build new paradigms.
DENSO Corporate Culture: If It Doesn’t Exist, Create It
Kuroda: Talking about breakthroughs is easy, but actually performing at such high levels is something else entirely, and here at DENSO we have pursued that end through our long-established “Make It Your Own” corporate culture.
The company’s former Vice President Aoki was the one who first suggested moving things in-house. He also likened “Make It Your Own” to the flavor of a home-cooked meal—creating something that cannot be found outside one’s home, which is highly useful because it cannot be bought anywhere else.
Since 1955, the Machinery & Tools Div. [editor’s note: division in charge of machine tools] has created robots and various equipment in-house, and in the field of software as well DENSO has been designing their own in-house CAD systems and production simulators for a long time now.
As Dan mentioned, standards such as the QR Code were created in-house here at DENSO. In other words, when we find that something we need does not yet exist, our approach is to make that thing. Ever since our founding, we have focused strongly on creating our own versions of things in areas where other companies would not.
Under our current system, we handle everything from quality control through to production management and embedded software. Therefore, our staff come from various backgrounds and bring together a wide range of knowledge in embedded software design, equipment design, and even other areas such as in the case of Dan, who has experience in banking, research and IT. Our members possess a diverse range of individualistic strengths not seen in any other organization.
Of course, not everything is carried out by in-house personnel from the planning phase through to operation and maintenance, but we do have personnel involved in all of these phases. The Production Eng. Div. is an extremely important part of the company in terms of fostering competitive manufacturing strength, and currently about one-fourth of the division is involved in factory IoT efforts.
In other words, the division is at the very center of all our manufacturing operations—it’s the place where we are focusing a majority of our efforts. I want to emphasize just how serious we truly are.
Next up is Hiroo Sawada, who will talk specifically about what he has created.
Technical Requirements for a Factory IoT Platform
Hiroo Sawada: Hello, my name is Hiroo Sawada. I’ll start with a short personal introduction.
I’m one of DENSO’s mid-career hires. I previously worked at a major electronic components firm where I gained experience in various things including quality assurance and production engineering. I was eventually assigned to autonomous control technologies coordinated with MES (manufacturing execution systems). DENSO’s project aimed at linking together their current 130 factories really impressed me, and it was an honor to be assigned to one of the coveted data scientist positions at the company in October 2017.
However, DENSO did not yet have a data platform and there was no data to analyze so just three days after joining the company I was removed from my data scientist position and tasked with transitioning platform production in-house. Therefore, I’m not actually a data scientist at DENSO; rather, I’m an engineer creating a data collection platform.
Here we see the main requirements for an IoT-related factory platform.
The platform must allow for scaling out and in via equipment connection increases and decreases, saving of data from all types of sources to a database, and provide loose coupling via middleware to enable co-creation with a wide range of partners.
We were aware of the Kubernetes system, which is widely known throughout the industry, and it was necessary for us to connect DENSO’s 130 factories. So we have been using this system as a platform to achieve unhindered scaling out over short time frames.
As an example, let’s examine a scaling out from 4 to 5 pods. (Examines some documents.) We have units of time on the Y axis and numbers of messages processed per unit of time on the X axis. Four pods are being used, and by entering a simple command we can increase the number of pods by one, thus easily increasing the number of messages processed. In this way, Kubernetes enables scaling out to compensate when current performance is not sufficient.
Method for Converting from Various Factory Data Formats
Sawada: Umm, so I’m guessing that not many of you are familiar with PLCs.
(Audience laughs.)
PLCs, or programmable logic controllers, are devices used in manufacturing operations to control equipment. At DENSO, we use PLCs produced by various companies including Omron, Rockwell Automation, Mitsubishi, Keyence, Siemens and others, and these PLCs have differing communication-related specifications.
(Examines some documents.) According to one example, 16 different bits are assigned to the address, with each bit having different meaning. The first bit, for example, indicates that the equipment is online, and the next bit shows that the equipment is in automated operation mode. Address bit assignment varies by PLC manufacturer, so bit definitions are different at each DENSO production center. This wide diversity in data structuring makes it very difficult to carry out ETL (extract, transform, load) operations and save data to the database, which is why we use Kafka.
(Examines some documents.) These are hex dumps of the memory whose decoding rules are stored in the RDB (relational database). The decoding rule is referenced during the streaming processes in Kafka streaming which enables the hex dumps to be inserted into the database.
Loose Coupling Between Middleware via In-house Applications
Sawada: Things are organized as shown here.
The factory is on the left side, and GCP (Google Cloud Platform) on the right. Within GCP, middleware such as Mosquitto and Kafka are in pods within the GKE (Google Kubernetes Engine), and through these data from the factory is input to BigQuery. These pieces of middleware are loosely coupled by having in-house applications in between them, so even if changes are made to the middleware, swapping out the in-house applications accordingly enables end to end data delivery to BigQuery.
We initially used Java for creation of in-house applications, but later decided that Golang was better for container applications.
The Java docker image size had grown to 195 megabytes, whereas a Golang image size as low as 15 megabytes was possible. Therefore, we switched from Java to Golang. I believe that this sort of change could not be done without in-house development.
All that was left was to create our own Golang framework which enabled a patchwork-style approach for introducing different modules. This made greater module reusability and short development times possible.
To conclude, here’s a summary of what I talked about today.
We used Kubernetes to enable scaling out for 130 factories in a short time; applied rules to the RDB for converting from various factory data formats, and then carry out streaming ETL; utilized loose coupling between middleware through in-house applications; and created our own framework to achieve a system for building various applications in short times for provision to worksites. Thank you for your time.
■Speakers
Dan Yakabe, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Yuta Kuroda, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Hiroo Sawada, Factory IoT Development Team, Production Eng. Div., DENSO Corporation
Why DENSO Pursued In-house Software Development Starting from Its Factories
Dan Yakabe: Hello everyone, my name is Dan Yakabe and I work for the DENSO Production Eng. Div.
The Factory IoT Platform we are creating is used to collect data on equipment, people and other aspects of factory operations provided by various devices, then convert this data into valuable information and provide it to users through applications.
This is our in-house system, and today I’m going to talk about why DENSO is pursuing in-house software development starting from its factories, and adopting the “Make It Your Own” approach. Yuta Kuroda will then explain who is carrying this out, and Hiroo Sawada will tell us how it is being carried out.
But first, allow me to begin with a brief personal introduction. In 2002, I joined a systems integrator company where I developed smart meter systems, HEMS, and other M2M systems. Later on, I made a mid-career change and joined one of the nation’s biggest banks where I developed security-related systems and APIs (application programming interfaces) for Fintech companies. My experience developing M2M technologies, security systems and APIs proved to be very useful when developing DENSO’s Factory IoT Platform for 130 different factories.
Manufacture of 44,000 Different Automotive Components
Yakabe: Let’s begin by talking about the development concept for our Factory IoT Platform. We often talk about CASE (connected, autonomous, shared and electric). Times have changed, from a focus on mass-producing high-quality products to an era in which organizations are required to tie together both equipment and software and provide services at increasingly rapid speeds in order to stay competitive. It is said that the automotive industry undergoes a once-in-a-century transformation.
In order to achieve the speed and revolutionary changes needed, “autonomy” is required to control progress. “Speed” means nimbly and quickly attempting new things, while the particularly important “co-creation” entails getting help from other organizations. These are the three concepts I want to focus on today.
On December 16, DENSO will celebrate its 70th anniversary [editor’s note: anniversary event was on December 9, 2019].
(Audience applause.)
I want to point out our R&D expenditure figures, which show that DENSO spent ¥510 billion on R&D to achieve ¥5.3 trillion in sales. Startups often celebrate reaching the ¥1 billion or ¥2 billion mark, but in DENSO’s case our numbers are two entire digits above that level, which enables us to pursue an extremely wide range of different things. DENSO employs 172,000 people, a majority of whom are based outside of Japan, making our work environments very international in character.
We have 130 factories—150 by some counts—which are located in almost all of the world’s continents. Furthermore, DENSO has pledged to grow its software engineer numbers to 12,000 by the year 2025—an ambitious goal which has led many to expect great things from our company.
This year we celebrate the 25th anniversary of the QR Code, a well-known technology we invented, and over the years we have also made a wide range of automotive components including air conditioners, motors for electric cars, and ADAS millimeter-wave radar sensors and monocular cameras, to name a few. We have produced 44,000 different types of automotive components, numbering 12 billion units in total. As a result, we possess manufacturing facilities capable of outputting large numbers of highly varying products, and this results in a highly diverse array of data sent in from our factories. This makes the development of an IoT platform quite difficult.
Using Digital Technologies to Achieve Co-creation Between People and Equipment, and to Accelerate Improvements
Yakabe: Now let’s look at our goals in creating the Factory IoT Platform. Our base concept is “using digital technologies to achieve co-creation between people and equipment, and to accelerate improvements.” The basic approach is to take processed results data from equipment related to operating conditions, product quality and so forth, and have people analyze this information via applications and feed the results back to the equipment.
The applications are generally expected to include machine learning, deep learning and other such analytical models along with sections providing visual displays of information, notification functions, and other standard features. Applications will be connected to a cloud computing network so the 130 factories around the world can make full use of them, after which we start making gradual improvements. In reality, though, this is very hard to pull off. Personally, I would call it bari-muzu.
I’m originally from Kyushu, where we say bari-muzu, which means “very difficult” in English. The reason it’s so difficult is because we want to achieve a lean production system at our facilities using the platform I just discussed, which takes in and manages data before outputting it again. DENSO is not the only one striving for lean manufacturing—a lot of Japanese companies want the same thing—but this approach does lead to huge numbers of demands for improvements at worksites.
I have implemented numerous rounds of PoC (proof of concept) testing and created a number of IoT platforms. At one factory where I provided only the data visualization and notification features, I received 400 improvement requests in just half a year’s time. The demands for new data input were endless: more data categories for equipment, a voice-recording feature, analysis of workers’ paths of movement, integration with other systems’ data, and so on.
Regarding application interfaces, users asked for a voice-operated search feature, more application subsections, slight changes to the search button location, data analysis cycle compatibility, and notifications via smartwatches, among others. Then there were the requests for coordination with upstream design information, and applications for mobility services, sharing and after-market areas—the list goes on.
Achieving Autonomy, Speed and Co-creation
Yakabe: Feeling a bit overwhelmed, I found myself relying on package-style product suites purchased from outside companies. I tried a number of these outside product packages, but things never went smoothly.
As I mentioned, I was taking in data from a number of different devices in relation to equipment information, sensor information, worker paths of movement and actions, and so forth. I wanted to get data via the platform and create new applications, but the need to consult the product vendor every time I input or output data was detracting from our autonomy.
A similar problem was occurring in regard to speed: every time I wanted to make a new application, I found myself being told that according to the roadmap, it was still a year or so away from being finished. Regarding co-creation, our partners had specific requirements and conditions for releasing databases to the public, and many of these partners were unable to work together with us as a result. In the end, DENSO was unable to move forward with development as anticipated.
An IoT platform requires three things in regard to data: collection, storage and usage. In order to achieve the necessary speed and co-creation during improvement cycles for these three areas while maintaining autonomy, these three things are necessities.
I’ll say it again: what we needed was autonomy, speed and co-creation. In the area of autonomy, for example, we used Java at the start but later replaced it with Golang, which led to greatly improved performance. Regarding speed, we began to use a public cloud network as well as Kubernetes.
So how was DENSO to move forward with making software on their own? How could we achieve effective co-creation? The answer to these was to move development in-house.
Yuta Kuroda will now talk about who it is that carried out this in-house transition.
Thank you for your time.
(Audience applause.)
Behind the Scenes at DENSO’s Full-Out Efforts Toward Moving Development In-house
Yuta Kuroda: Hello everybody. I’m going to talk about how a group of DENSO production engineering experts pursued full-out efforts toward internal, in-house development of the Factory IoT Platform.
I joined DENSO in 2008 and have worked here for 12 years, starting in production engineering and later moving on to core systems SE. Currently, I work in the Factory IoT Development Team, which is where I had always wanted to be. I’m really interested in Kubernetes, Ansible, CI (continuous integration) and the other infrastructure-related areas in which I'm involved.
In the slides, I used the example of Dejima to describe a special place when talking about agile development at large companies. Many people think that such development is carried out at special places in specialized organizations, and that only product owners are company employees while everything else is outsourced. Others may think what we do is just PoC without a specific goal. I want to explain how serious we are about this project.
I will go over the details during Cross Talk, so in this presentation I want to talk about who is carrying out our efforts, and just how dedicated those people are.
Breakthroughs to Remedy Production Engineering Trade-offs
Kuroda: To start with, what exactly is “production engineering”? Allow me to provide a bit of an overview. “Product design” entails deciding what we’re going to make; “production engineering” focuses instead on how we’re going to make it. Some people refer to this as “process design,” as it is carried out by the division or department in charge of designing the processes.
In real-life production operations, there are manual production lines where workers use their hands, and automated lines which utilize robots. In general, manual lines are less expensive and robotic lines are more expensive.
Robotic lines are faster and more precise, whereas human-operated manual lines are more flexible in terms of process changes. Making decisions regarding trade-offs between quality, cost and, delivery, as well as other factors, is one of the main goals of production engineering development efforts.
When it comes to production engineering at DENSO, the Production Eng. Div. not only thinks about trade-offs related to quality, cost and delivery, but also pursues technological breakthroughs in order to succeed to a high degree in all three of these areas.
We harness the power of software in order to build new paradigms.
DENSO Corporate Culture: If It Doesn’t Exist, Create It
Kuroda: Talking about breakthroughs is easy, but actually performing at such high levels is something else entirely, and here at DENSO we have pursued that end through our long-established “Make It Your Own” corporate culture.
The company’s former Vice President Aoki was the one who first suggested moving things in-house. He also likened “Make It Your Own” to the flavor of a home-cooked meal—creating something that cannot be found outside one’s home, which is highly useful because it cannot be bought anywhere else.
Since 1955, the Machinery & Tools Div. [editor’s note: division in charge of machine tools] has created robots and various equipment in-house, and in the field of software as well DENSO has been designing their own in-house CAD systems and production simulators for a long time now.
As Dan mentioned, standards such as the QR Code were created in-house here at DENSO. In other words, when we find that something we need does not yet exist, our approach is to make that thing. Ever since our founding, we have focused strongly on creating our own versions of things in areas where other companies would not.
Under our current system, we handle everything from quality control through to production management and embedded software. Therefore, our staff come from various backgrounds and bring together a wide range of knowledge in embedded software design, equipment design, and even other areas such as in the case of Dan, who has experience in banking, research and IT. Our members possess a diverse range of individualistic strengths not seen in any other organization.
Of course, not everything is carried out by in-house personnel from the planning phase through to operation and maintenance, but we do have personnel involved in all of these phases. The Production Eng. Div. is an extremely important part of the company in terms of fostering competitive manufacturing strength, and currently about one-fourth of the division is involved in factory IoT efforts.
In other words, the division is at the very center of all our manufacturing operations—it’s the place where we are focusing a majority of our efforts. I want to emphasize just how serious we truly are.
Next up is Hiroo Sawada, who will talk specifically about what he has created.
Technical Requirements for a Factory IoT Platform
Hiroo Sawada: Hello, my name is Hiroo Sawada. I’ll start with a short personal introduction.
I’m one of DENSO’s mid-career hires. I previously worked at a major electronic components firm where I gained experience in various things including quality assurance and production engineering. I was eventually assigned to autonomous control technologies coordinated with MES (manufacturing execution systems). DENSO’s project aimed at linking together their current 130 factories really impressed me, and it was an honor to be assigned to one of the coveted data scientist positions at the company in October 2017.
However, DENSO did not yet have a data platform and there was no data to analyze so just three days after joining the company I was removed from my data scientist position and tasked with transitioning platform production in-house. Therefore, I’m not actually a data scientist at DENSO; rather, I’m an engineer creating a data collection platform.
Here we see the main requirements for an IoT-related factory platform.
The platform must allow for scaling out and in via equipment connection increases and decreases, saving of data from all types of sources to a database, and provide loose coupling via middleware to enable co-creation with a wide range of partners.
We were aware of the Kubernetes system, which is widely known throughout the industry, and it was necessary for us to connect DENSO’s 130 factories. So we have been using this system as a platform to achieve unhindered scaling out over short time frames.
As an example, let’s examine a scaling out from 4 to 5 pods. (Examines some documents.) We have units of time on the Y axis and numbers of messages processed per unit of time on the X axis. Four pods are being used, and by entering a simple command we can increase the number of pods by one, thus easily increasing the number of messages processed. In this way, Kubernetes enables scaling out to compensate when current performance is not sufficient.
Method for Converting from Various Factory Data Formats
Sawada: Umm, so I’m guessing that not many of you are familiar with PLCs.
(Audience laughs.)
PLCs, or programmable logic controllers, are devices used in manufacturing operations to control equipment. At DENSO, we use PLCs produced by various companies including Omron, Rockwell Automation, Mitsubishi, Keyence, Siemens and others, and these PLCs have differing communication-related specifications.
(Examines some documents.) According to one example, 16 different bits are assigned to the address, with each bit having different meaning. The first bit, for example, indicates that the equipment is online, and the next bit shows that the equipment is in automated operation mode. Address bit assignment varies by PLC manufacturer, so bit definitions are different at each DENSO production center. This wide diversity in data structuring makes it very difficult to carry out ETL (extract, transform, load) operations and save data to the database, which is why we use Kafka.
(Examines some documents.) These are hex dumps of the memory whose decoding rules are stored in the RDB (relational database). The decoding rule is referenced during the streaming processes in Kafka streaming which enables the hex dumps to be inserted into the database.
Loose Coupling Between Middleware via In-house Applications
Sawada: Things are organized as shown here.
The factory is on the left side, and GCP (Google Cloud Platform) on the right. Within GCP, middleware such as Mosquitto and Kafka are in pods within the GKE (Google Kubernetes Engine), and through these data from the factory is input to BigQuery. These pieces of middleware are loosely coupled by having in-house applications in between them, so even if changes are made to the middleware, swapping out the in-house applications accordingly enables end to end data delivery to BigQuery.
We initially used Java for creation of in-house applications, but later decided that Golang was better for container applications.
The Java docker image size had grown to 195 megabytes, whereas a Golang image size as low as 15 megabytes was possible. Therefore, we switched from Java to Golang. I believe that this sort of change could not be done without in-house development.
All that was left was to create our own Golang framework which enabled a patchwork-style approach for introducing different modules. This made greater module reusability and short development times possible.
To conclude, here’s a summary of what I talked about today.
We used Kubernetes to enable scaling out for 130 factories in a short time; applied rules to the RDB for converting from various factory data formats, and then carry out streaming ETL; utilized loose coupling between middleware through in-house applications; and created our own framework to achieve a system for building various applications in short times for provision to worksites. Thank you for your time.