For benchmarking C implementations, the there are a few benchmarks, but they all have their problems. Many benchmarks have memory requirements that are far too high or need functionality not necessarily available. Some are quite one-sided in what they measure (e.g. Whetstone, Dhrystone, Coremark).
So, I deciced to write a new benchmark, stdcbench. I wanted it to be suitable for small systems (4KB of RAM, about 32 KB of Flash). There is a trade-off here, since all the data and code will fit easily into caches on bigger systems, but IMO it is worth it.
The current version consists of 2 modules, which on typical systems should contribute about equally to the score.
c90base: It benchmarks a commonly-implemented subset of what the standard requires for freestanding implementations of C90. It consists of three submodules:
1) Huffman/RLE decompression (adapted from real-world code) 2) Integer matrix multiplication (synthetic) 3) Insertion sort (adapted from real-world code)c90lib: Benchmarks the standard library. I consists of two submodules:
1) Computation of lnlc-width (adapted from real-world code). 2) Peephole optimizer (simplified from real-world code).C99 features (e.g. bool, restrict) are used where available, but not necessary.
So far, stdcbench seems to achieve the goals: benchmark a wide range of important standard c functionality, without giving too much emphasis to any particular aspect.
Scores are reported for each module and as total.
Example output from a i7-7500U-based system (benchmark compiled with GCC
7.2.0 using -O2 -march=native):stdcbench 0.2 stdcbench c90base score: 7827 stdcbench c90lib score: 6548 stdcbench final score: 14375
Example output from a STM8AF5288 at 16 Mhz (benchmark compiled with SDCC
3.6.9 using -mstm8 --opt-code-speed --max-allocs-per-node 10000):stdcbench 0.2 stdcbench c90base score: 6 stdcbench c90lib score: 6 stdcbench final score: 12
Future plans for the benchmark:
1) Come up with module(s) for floating-point performance. What matters for embedded systems? How should correctness be verified for floating-point? 2) Find out why the c90lib module hangs on C8051F120 (possible compiler bug). 3) State run/reporting rules. 4) Benchmark a few interesting systemsI am looking forward to comments from you.
Philipp