Tests in embedded

Having hardware in the loop testing takes more work than pure software testing, but can certainly be done.

You can set up a laptop or something like Raspberry Pi to program and communicate with the target. Then run external test code on that host. Eventually you might end up with a whole rack full of these..

Simplest case is testing that drivers work and access peripherals that can somehow identify them and possibly even report their state.

In my view the next level is having something that simulates something in the 'world' around the target. Or even real sensors and actuators.

If you're doing micropower stuff, you can setup a supply current measurement to the test host so you can see if the code stays within energy budget.

Then you can start adding external measurements and stimuli controlled by the host. Sometimes I've used a modified target board acting as IO simulator, too. LabJacks are also great for this. Sometimes I use GPIB/LXI controlled test gear, too, but those can get expensive if you want to store the whole test setup for future use.

I have not seen too much good frameworks for this, but I've also not been looking for those for a few years. Typical setup is building the code using the usual CI pipelines (I'm using Bitbucket and Gitlab), running software tests there and then SSHing to the HIL test host to run HW-connected tests there. Tests are usually in Python.

The design companies I've been working with have their own in-house test systems, which they're using for production testers - I've stolen a few good ideas from them on how to organize the test code structure. Some of the tests can then be run in the production testers, too!

It helps if you simulate everything you can in the test system. That lets you test the main code on a normal computer with no special hardware. Of course that's not always feasible, and it doesn't help much with testing the hardware you are simulating.

In general, yes, test automation is a huge win and is basically mandatory these days.

Right, whatever can be tested SW only is usually much easier that way. Device drivers, energy budget and peripheral interactions and silicon revisions.. I've been thinking if MAME could be used for some of these!

And repeatable builds. Once I set up Docker-based builds for some things there's no going back. Now I can compile a bit-perfect copy and start any troubleshooting or new versions from that.

Il 16/05/2020 19:08, Mikko OH2HVJ ha scritto: [...]

Docker-based builds... interesting, could you explain?

We used to do the same thing by archiving entire machine snapshots (ghost images), and later using virtual machines. Docker just makes that more storage-efficient.

CH

If the compiler is available for Linux, you can create a Docker image, which is basically a stored image of disk. Then, you attach the source directory as a directory inside that 'image' and run your compile there. Since it uses Linux layered filesystem, any changes that are done inside the image only affect the top layer. After compile you start again from the original build image, which is exactly the same.

Here's an example command I'm compiling one STM32 project from a Makefile in src/ directory:

docker run -it --mount type=bind,source="$(pwd)"/src,target=/work/src \ stronglytyped/arm-none-eabi-gcc:latest /bin/bash -c "cd /work/src; make \ clean-all"

That runs in interactive (-i) mode image "stronglytyped/arm-none-eabi-gcc:latest", which contains ARM GCC environment. The src/ directory is attached as /work/src inside that image. Then, bash is used to run "cd /work/src; make clean-all" inside that image. The makefile does the build, tests and outputs the binaries and programming files for me under /work/src/dist", which I can see as src/src/dist outside the docker.

I'm actually running Mac and using Docker to do builds. And the exactly same Docker image+source code is run on the CI/CD/test server when I push my changes, so the binary compiled by CI/CD is exactly what I had while developing on my Mac.

CI/CD server does the same compile and in addition runs HIL tests, code signing, delivers the package to staging area etc.

This is how almost all modern (cloud) software is done, there's a lot to apply in embedded world. Of course, the integration/HIL testing and software delivery are usually very different!

(To do production builds, use version controlled build image, not the :latest)

Absolutely. Here's an example from a small project I finished recently:

Right, what the above example is missing and I hardly ever see is:

- record/playback for capturing and replaying tests

- visualization/comparison tools for checking and verifying results That's more on the TDD side.

I'm thinking about building a framework for the above. Any thoughts?

Examples covered in this course (a long one, good stuff near the end):