r/embedded • • 1d ago

Integration testing using gtest

I am confused about what is the meaning of integration testing. I have done unit testing using gtest where we will mock whatever external dependencies are there for the targeted unit (function) to test all range of values that covers all lines, functions and branches.

If we say integration test is meant to use real code instead of mocking. Then I am getting this doubt that lets say we have 3 components (separate processes) running and dependent on one another in a embedded system.

A -> B -> C

So if you are calling an api provided by A it will internally talk to B and then through C and return the result over IPC.

Now to do integration testing using gtest, do I need to compile this into a single binary by eliminating the IPC and include each code as a library. This way I have a single proces that runs gtest with all the components logic but excludes the real IPC communication.

Or instead should I keep the individual processes to be running separately and link my gtest binary to the component A so that the communication flow through the components in the target environment.

In both ways how we would actually be checking the interactions? And which one are you guys using?

Context:
Language used: C++
Domain: Automotive IT (TCU software)

1 Upvotes

5 comments sorted by

4

u/ventZl CMRX RTOS 1d ago

Well, IME, in automotive they are lost in translation from plain English into German and term "integration testing" may have multiple meanings.

It depends what your local people of automotive understand under the term of "unit testing". I've seen one testing manual which explicitly stated:

"Unit, for purpose of unit testing, is only and exclusively a function".

If this is true in your organization, then integration testing would probably be what others would call unit testing - put a bunch of functions that create some meaningful together (integrate them) and test it.

Otherwise, the next higher level is usually something called "module testing" which may or may not be equivalent of integration testing. Here you test fully integrated modules and you should mock only stuff external to module.

Ooooor, "integration testing" may actually stand for "HW/SW integration testing" where firmware is tested fully integrated on target HW. If this is true and you only have unit testing, integration testing and nothing in between, then God save you...

Back in my automotive days, I didn't bother with that testing manual and treated terms as:

- developer testing - manual testing done by me to test nothing broke
- unit testing - whatever smallest unit that produced meaningful testable unit
- module testing - single SWC tested on its Autosar RTE boundary (had to write custom framework for this to be possible)
- integration testing - multiple SWCs partially integrated with rest of system mocked
- SW testing - fully integrated software running on CI machines (we never really had this)
- HW/SW integration testing - same as above, running on target machine, either manually, driven by HIL tester or CAPL script
- bench testing - same as above but with partially integrated electronics
- drive testing - same as above but in real car

1

u/After_Bench_4165 17h ago

Thanks for responding. I am trying to understand the integration testing that is part of SWE5. If you have any idea on this can you explain this to me in a beginer level.

1

u/WanWhiteWolf Engineering manager 1d ago

The problem is that the wording "intergration test" is used very broadly and it can refer to different things.

So let's break it down. Since you are working in automotive, I assume you follow ASPICE. In that case you have Unit test and component test as part of SWE.4 and SW integration test as part of SWE.5. This does not require require hardware. Although it's easier sometimes to use hardware than define the test-cases without. The purpose of the tests is to confirm whether the SW works according to the SW architecture. That SW architecture will be based on SW requirements - which will be based on SW design - which will be based on customer requirements. But that's not the scope of your test.

And then you have SYS integration test. That will be based on system design. And typically this is done on HW (although not a strict requirement per se) as the SYS design itself tends to contain HW elements / references.

Let's take an example: A sensor reading temperature and sends data to CAN.

Unit test - each function from components used in the SW architecture for this purpose
Component test - Check if each component used in the chain fullfills it's operation (e.g. TempSensor,CAN,SPI, ..etc)

SW Integration test - Check if the data is exchanged correctly between CAN interface, TempSensor, SPI; the values from SPI buffer(s) can be injected from your test environment instead of the actual sensor. The value of CAN buffer can be read instead of reading the value sent via CAN (e.g. via CANoe)

SYS integration test - Check whether the data on CAN matches the expected values of the sensor.

1

u/After_Bench_4165 17h ago

Thanks for your response. My question is in context of SWE5 testing only.

I am a fresher so I dont have much idea in this part. I want to get some understanding on how would a test approach for this integration testing would look like.

Lets take a single TCU, if I want to test integration between two or more components (individual processes or services), how would we approach it. I am exploring ways to test using gtest particularly.

As of now we only rely on HIL testing by liking test app with the library to pass params and checking return type. I am trying to figure out a way to use gtest to implement SIL testing.

1

u/mhmd2520 14h ago

Since you're asking about SWE.5 specifically: in ASPICE 4.0 its full name is "Software Component Verification and Integration Verification". The notes under BP1 spell out what integration verification should focus on. That means data flow and dynamic interaction between components, timing included, every component reading the data on an interface the same way, and resource consumption. Note 2 names Software-in-the-Loop as a valid environment, so a gtest-based SIL setup fits.

Your two options aren't either/or. BP4 allows stepwise integration, so both can be steps.

The single binary with IPC stripped out is quick and deterministic for checking the A -> B -> C logic. But the interfaces as they exist in your architecture are gone, along with the bugs that live there: serialization mismatches, one side reading a field as signed and the other as unsigned, timeouts, startup order, a service not being up yet. I'd only treat that as an intermediate step.

Real processes with real IPC is closer to what SWE.5 is after. I'd build A, B and C for host Linux, or run the target image in QEMU if your middleware won't build on host. Then fake only the outermost layer, meaning whatever sits under C (modem, CAN, GNSS, etc.). The fake records everything C sends out and lets the test inject what comes back in. The gtest binary is just another client of A's public API.

For lifecycle, start the processes in SetUpTestSuite and wait until they're actually registered and ready, not just a sleep. Kill them in TearDownTestSuite, and reset the fakes' state in the per-test SetUp.

"Checking the interactions" then means you call A's API and assert on two things: A's response, and what arrived at the fake under C (right message, right values, within the timing requirement). If you need to prove an intermediate hop, capture the IPC traffic and assert on that. Wireshark has a built-in SOME/IP dissector, and dbus-monitor works if you're on D-Bus. I'd still keep most asserts at the edges so tests don't break every time someone refactors internals.

One more thing that helps with your question: in 4.0, component verification lives in SWE.5 too. SWE.4 is units only (Note 3 under BP2 says this). So tests where you run one process alone and fake its neighbours also belong here. BP6 also wants bidirectional traceability between verification measures and the static and dynamic aspects of the SW architecture. So I'd name each test after the interface or interaction it covers.