I’m Andres, I’m a software tester for the automotive industry, and today I’m sitting in the driver’s seat of a car, while holding a laptop, executing testing, storing logs and filling in an Excel table. I could have used the passenger seat, but that’s the place for an additional 12-volt power supply, a couple of big CAN (Controlled Area Network) interfaces, a pair of custom-made circuit boards, and a bunch of cables of all colours. It looks a little messy, but that's just how it is. It is a car, not an office desk, and this is how it is when you deal with a test bench in a car.
AI generated image of a typical setup when testing software in a car.
A test bench is an environment used to validate a product or system. A testbench normally includes several hardware components like cables, power supplies and interfaces with the intent of simulating and simplifying a real-life system and making the testing process easier and more effective.
Let's use the example of a real car to understand this better
Scheme of a Typical Vehicle Network
Car software explained
To understand the car software and the testing required, we need to break down all the component parts.
- ECUs (Electronic Control Units) exchange information through a CAN bus [Ref 1].
- A CAN bus network is a topology where different devices are connected to a single shared cable (bus).
- In order for these devices to communicate, they use a communication protocol. The CAN protocol is the most common communication protocol used in the Automotive Industry, but there are others, including LIN (Local Interconnect Network), FlexRay, MOST (Media Oriented Systems Transport), and Automotive Ethernet. In practice, it is common to find different protocols working together.
- Each ECU can receive specific information that may or may not be present in the CAN bus. For instance, the Brakes ECU can receive information from an ABS (Anti-lock braking system) sensor located in the wheel.
The previous image is just a simple example. A current Mercedes S-class possesses around 80 ECUs. As a result, each ECU sends information to the CAN bus, which the remaining ECUs will then read to operate. However, not all ECUs will read all the information present by the CAN bus.
The Engine ECU sends the engine temperature value ten times every second to the bus. The Cluster ECU will read this value to update its temperature gauge. However, the engine temperature is not relevant to the Airbags ECU. Therefore, the Airbags ECU will not read this signal. To do this, each signal present in the CAN bus has identifiers that can be recognised by the different ECUs, so they can decide if that information is relevant for them or not. The structure or algorithm that specifies this is given by the CAN protocol.
Testing Car software
A car is essentially a network of different devices, each with its own purpose, and they are connected by a CAN bus. We start by testing each ECU individually. The theory behind this comes from the Automotive SPICE (Software Process Improvement and Capability Determination) [2], a process model used to develop software for vehicles. Without going into too much detail here, this is a ‘V’ type maturity model adapted for the automotive industry, where the apex of the V corresponds to individual ECU development testing. So, before testing all together, you need to test individual units. How is this done?
Testing ECUs and using the test bench
As mentioned, testing ECUs individually is quite a normal practice. I personally tested infotainment, ADAS (Advanced Driver Assistance System) and suspension systems. The ECUs responsible for each of these features are different. I mean, really different! The ECU in charge of the engine can be from Siemens, the one in charge of the braking system can be from BOSCH, and the one for the Airbags can be from TEMIC. Also, each ECU can use different programming languages to implement its internal code, but all of them will use the CAN protocol to communicate with each other.
It is common for your work to involve testing and identifying bugs in a specific ECU. Your company could develop both ECU hardware and software, or just the software, while using the ECU from another vendor. Or, both the ECU and software belong to different companies, and your work is to provide software testing services. In any case, to test just one ECU, you need a test bench that simulates the real car, so that your ECU “thinks” that it is installed in a real car. A simplified example of a test bench used in automotive can be seen in the next picture.
Scheme of a Typical Testbench used in Automotive Testing
Basically, you use an interface that communicates with the ECU in the CAN language, and on your Laptop, you run a simulation of the rest of the ECUs present in the bus. It is possible that the ECU requires some other external inputs, such as a 12-volt power supply, to simulate the car’s battery.
Thanks to this test bench, you can simulate inputs to the ECU and observe its reaction to them. The inputs are signals present in the CAN bus that are simulated using an app installed on your laptop.
Mocking car software
As a tester, the app is your best friend. It is where you define your test cases, and it allows you to simulate or mock different conditions. For instance, you can simulate that the car is running at 100 km/h, the ECU you are testing thinks you are driving that speed, but actually, you are sitting in your office. A proper app simulates the rest of the ECUs and car components involved in your system, making the software installed in the ECU you are testing think it is in a real car. In a basic app, you can create blocks that represent the ECUs present in the car. In those blocks, you will write scripts that can manage the information present in the bus as a real ECU could do.
Let’s take an example to understand how an app will work. Imagine a simple Climate Control System exemplified below:
Scheme of a temperature control system on a vehicle
In this setup, there are two inputs. The temperature inside the car is measured by the temperature sensor, and the desired user temperature. The user sets the desired temperature with a dial or a touchscreen. Based on that, the Climate ECU must decide when to use the heater or the cooler. Some examples of use are:
If Sensor Temperature > User Temperature, swap from Heater to Cooler
If Sensor Temperature < User Temperature, swap from Cooler to Heater
If Sensor Temperature = User Temperature, use neither
We also know that the inputs to the ECU are temperature signals with defined integer values, and the outputs to the heater and cooler are binary signals that can be 0 or 1. Without the app, all the hardware necessary for the test is required (Sensor, Touchscreen, ECU, Heater, and Cooler). But with the app, life is much easier.
The app can:
- Remove the need for most hardware. Only the ECU and laptop are required.
- Simulate input signals. The app provides a user interface where we can input different values for both Sensor and User Temperature to simulate conditions. We can also simulate extreme conditions (max cold or hot) and even values outside the standard range.
- Observe the outputs. To test the software, we observe the ECU's outputs to verify if they meet our expectations within the specification.
- Simulate the feedback of the Heater and the Cooler to the temperature sensor.
- Automation of test cases. It is possible to automate actions like setting input values and checking outputs.
As you can see, the use of an app brings many benefits. Among them, there are not only budget savings for hardware but also the flexibility to test. Standard apps in the industry include vector Tools like CANoe[3] that allow us to simulate complete CAN buses, create automated test cases, and record logs. There are also other companies that provide similar tools, like Peak[4] or the more simplistic free-to-use software Cangaroo[5], popular in the do-it-yourself community.
A personal story of getting out of my bubble
All the processes mentioned are part of my daily routine. Over the last seven years, I have been working in the German automotive sector, where I have been identifying and reporting bugs in the software that controls various systems in a car. For personal reasons, I will be moving to my homeland (Spain) soon. So, logically, I need a new job.
As the development of automotive software is not as strong in Spain as in Germany, I have been seeking other opportunities as a QA (quality assurance) tester. Maybe testing websites, apps or APIs. As previously mentioned, other industries have their own ways, and I’ve started to learn about them.
It was through this process and the Ministry of Testing that I learned about tools like Selenium and Postman, as well as the importance of GitHub projects, testing portfolios, and the MoT job board. And here is where I felt like I was living in a friendly and comfortable bubble. I realised that the automotive sector is just a small part of the entire software industry when I started investigating other ways of software testing. A few visits to the Ministry of Testing Club made me realise that there is a whole world of software testing that I had no idea about.
It was at that point that I created a post in the Club called “Are we living in Bubbles?”[6]. The post generated good engagement, as it is a true feeling that I am sure some people can relate to when trying to change professional paths. I have, however, a lot to acknowledge about this feeling, as it made me feel pressure to learn new things and open my mind to new tools and ways of doing software testing. This is something I will continue to do, as technology advances rapidly, so it is best to be prepared.
To sum up
If you've reached this point, thank you for reading. I hope that my article has given you a basic understanding of how software testing works in the automotive industry. Also, to understand the importance of being humble enough to admit that there are more things to learn about testing outside of your daily routine.
Finally, I must say that in the middle of this, I was offered a new position in Spain (my homeland). I will be testing the software that controls the routing and ticketing system for public transport. I know this will provide me with access to a wide range of new tools used in testing and to some other ways of working. I will keep you informed about my next testing adventures in the Club.
What do YOU think?
Got comments or thoughts? Share them in the comments box below. If you like, use the questions below as starting points for reflection and discussion.
Questions to discuss
- Have you seen something similar to this?
- Have you done a process similar to this, or used mocks and stubs?
- What are your thoughts and takeaways?
References:
- CAN bus - Wikipedia
- Automotive SPICE - Wikipedia
- CANoe: Development and Test Tools for All SIL and HIL Projects - Vector
- PCAN-Basic: API for Connecting to CAN and CAN FD Buses - PEAK-System
- Getting started guide - CANable
- Are we living in Bubbles? - Club forum
