As I'm working an a Java processor I'm interested to benchmark embedded Java systems. However, the usual SPECJvm98 is far too large for many embedded system. After searching around on the net to find embeddded benchmarks in Java I've decided to write a new benchmark suit.
It contains several micro-benchmarks to evaluate CPI for single bytecodes or short sequences of bytecodes, one synthetic benchmark (the Sieve of Eratosthenes) and two application benchmarks.
To provide a realistic workload for embedded systems a real-time application was adapted to create the first application benchmark (Kfl). The application is from one node of a distributed motor control system. A simulation of the environment (sensors and actors) and the communication system (commands from the master station) forms part of the benchmark for simulating the real-world workload.
The second application benchmark is an adaption of a tiny TCP/IP stack (Ejip) for embedded Java. The benchmark contains two UDP server/clients exchanging messages via a loopback device.
There is a great variation in processing power on different embedded systems. To handle these variation all benchmarks are `self adjusting'. Each benchmarks consists of an aspect that is benchmarked in a loop and an `overhead' loop that contains any overhead from the benchmark that should be subtracted from the result (this feature is for the micro benchmarks). The loop count adapts until the benchmark runs for more than a second. The number of iterations per second is calculated which means that higher values indicate better performance.
The benchmark framework needs only two system functions: One to measure time in millisecond resolution and one to print the results. These functions are encapsulated in LowLevel.java and can be adapted to environments where the full Java library is not available. For example, the leJOS system has very limited output capabilities and and a special LowLevel.java exists in the lejos subdirectory. The benchmark is straight-forward to use. To compile and run all benchmarks on a standard JVM simple type:
javac jbe/DoAll.java java jbe.DoAll
I've put the results of JOP, leJOS, JStamp (aJ80), EJC and, for a reference only, different JVM versions (Suns JVM and gcj) running on a Pentium MMX 266 at my website:
If someone has access to other embedded Java solutions it would be nice to add benchmark results to the table. You can find the source of the embedded benchmark at the website above.
I would be especially interested in following platforms: the aJ100, TINI, Cjip, Lightfood in an FPGA, Moon and the PSC1000 (now called Ignite).
Martin
---------------------------------------------- JOP - a Java Processor core for FPGAs: