Hello All,
I have always wondered what is the best way to compare cross compilers.
We pay a lot for compilers, when a freeware could be just fine.
Can you all share your thoughts or ideas?
Are there some handy standard C-programs, which we can use to test assembly level code of compilers?
Or what are the best algorithms to test them.
regs ashu
Op Tue, 02 Dec 2008 11:36:41 +0100 schreef ashu :
Some compiler vendors release benchmark reports.
Using freeware doesn't mean that you don't pay a lot. The TCO's are different, and either could be larger.
IAR Systems uses the following code examples for their ARM compiler benchmarks - "covering most embedded application areas"
blowfish Blowfish algorithm bt_stack Embedded protocol stack car_navig Customer application decrypt Functions to descramble encrypted multimedia content. dyn_array Dynamic array allocation embos_test Real time operating system engine_control Customer application; turbo engine control float Part of customer application, battery charger. Floating point calculations for capacity, diffusion, service hours etc. floattest Synthesized test of float and integer arithmetics generator_controller Customer application; marine generator controller gsm_efr GSM Enhanced Full Rate (EFR) coder/decoder mars encryption algorithm math Misc. math routines microwave_sensor Customer application; microwave sensor for measurement of moisture in bins mix DVI ADPCM coder/decoder and Patricia trie (from retrieval) implementation. modeit Internet application with TCP/IP stack. reed_solomon_decoder Reed-Solomon decoder regexp Regular expression scanner. Uses search string with special characters to match patterns of text. rijndael encryption algorithm serpent encryption algorithm sha secure hash algorithm spirometer Customer application, Spirometer. Device that determines how well the lungs receive, hold, and utilize air, to monitor a lung disease. susan Image processing to detect the position of edges/corners for the guidance of unmanned vehicles. temp_display Customer application; temperature logger
Of course, different MCU have different application areas, and some examples might be closer to your application than others.
-- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/
We have just had the usual spirited discussion about some of the pros and cons of free compilers vs commercial ones ("GNUPro Development, v850").
You may want to read that first.
I have found that the biggest difference is the libraries used. If you are doing "basic" memory-constrained microcontroller programming, it is probably best to avoid linking some of the general purpose external libraries that often come with the free compilers.
I am thinking mainly of the gcc+newlib combination here. A careless use of e.g, "assert.h" or malloc can link in printf and all of stdio. Some "distributions" of gcc have other much more compact libraries such as avr-gcc. And for some larger projects where you have lots of RAM newlib may be fine (and even required) for your project.
It is very important when starting out to generate a linker map (and assembly listings) so you can see where all the memory went!
For my applications on single chip microcontrollers I have adapted a very small public domain "printf" for any numeric output or debugging requirements. With this change, and also avoiding malloc and friends, I get extremely compact code on e.g. ARM. Much better than that from a
*casual* attempt with IAR and Keil. I would imagine (hope) that the commercial vendors would put more effort into library memory footprint and performance - but these are not always that important as explained above.Your own programs? The only real side-by-side test I did was with a particular speed-critical routine from my own software.
-- John Devereux
Aaargh - not another vehicle for a pros and cons of GCC religious argument! Please see the ARM IDE thread, and the hijacked FreeRTOS/SafeRTOS for medical apps thread - plus plenty of others.
-- Regards, Richard.
There are no good standardised ways to compare embedded compilers (and only limited possibilities for non-embedded compilers). What you need from an embedded compiler varies wildly from user to user.
You have to make yourself a list of what's important for *you*. Then look around for available tools. Give estimated ratings for the tools in the relevant categories. Then you can determine a short-list, collect price quotations, trail versions, etc., and try them out with your own developments. Update your ratings based on your findings, and pick the winner.
Simply running example code through the compiler and looking at the assembly will only give you a rough idea of the generated code size and speed - that's only one aspect of compiler choice, and not often a major aspect.
Relevant categories to consider (not necessarily in this order) are:
Availability - can you get the tools where you live? Are there distributors, or must you deal with foreign suppliers?
Support - do the vendors give good support? And what terms or prices?
User community - is there a good user community around the tool, which can provide help, advice and examples?
Locking - are the tools locked to a workstation, dongle, license server, etc., or can they be freely used on any PC? Are there going to be problems getting replacement dongles or locks? What about archived tool versions?
Target support - do the tools support targets you are using now, and others that you want to use?
Host platforms - do the tools support your choice of host platforms?
Other tools - does the compiler come with an IDE, debugger, profiler, flash programmer, etc.? Are these any good?
Integration with other tools - do the tools support your choice of debugger hardware? Do they work with your choice of editor, compiler, source code management system, etc.?
Price - consider the price per node, different prices for different options, upgrade or maintenance costs, support costs, etc.
Training and learning - will it take a lot of work to start getting the best out of the tools? Is there good documentation, tutorials and examples?
Quality, maturity and reliability - are you confident that the tools are good enough quality?
Optimisations - does the compiler generate small and fast code?
Libraries - are there good libraries included with the tools? Is the source code available, and can you modify it?
Licensing and EULA - are you happy with the licensing and contract requirements? Some vendors require additional signed contracts - are you happy with that? Some companies view open source licensing as a positive feature, others see it as a negative thing - you need to know where you stand, and what your rights and responsibilities are. Tools like gcc may be free, but they are not "freeware".
That should probably be enough to get you started :-)
If you have a specific target processor in mind, I'm sure you can start a new "my compiler is better than your compiler" flame war.
Also, with newlib, you have to be aware that there are a LOT of configuration options for what you want and what you don't want, which drastically affect the size of the binaries. For example, don't want floating point? Don't configure it, makes printf smaller. Or use iprintf which only had integer support.
The default builds tend to have everything included, so beware.
Fair enough, I never played too much with these. FWIW you can get in just as much trouble with commercial compilers too. For example I seem to recall my IAR ones were just as bad if you used printf carelessly, but you could link in an integer-only version which was much smaller.
-- John Devereux
When I want a really skinny sprintf for use with WEB server demos, etc. I use this:
-- Regards, Richard. + http://www.FreeRTOS.org Designed for microcontrollers, more than 6000 downloads per month. + http://www.SafeRTOS.com Certified by TÜV as meeting the requirements for safety related systems.
My one is based around a basic formatter where you can pass it an arbitrary function that performs output of one character. So you can then define your own "printf-like" shell functions and use them for printing to e.g. graphics displays, even with proportional fonts etc.
-- John Devereux
Here's one sprintf() written long time ago:
As to the OP's question - compiler comparison is subjective and shall be done based on certain (sorted) criteria. For example once I ditched certain compiler because important portions of it's syntax were wildly unique, even if it's code was little better than some other's.
What? We're supposed to hijack this thread with GCC ranting?!?
OK!
As I say to my wife when I'm about to disassemble some vital and expensive part of the house or a car: "Why should I pay a professional to f*** this up when I'm perfectly capable of f***ing it up myself, for free?".
The extension to free tools should be obvious...
-- Tim Wescott Wescott Design Services http://www.wescottdesign.com Do you need to implement control loops in software? "Applied Control Theory for Embedded Systems" gives you just what it says. See details at http://www.wescottdesign.com/actfes/actfes.html
Here is one more which contains a configurable printf (via dietfeatures.h):
/* feel free to comment some of these out to reduce code size */
#define WANT_FLOATING_POINT_IN_PRINTF #define WANT_NULL_PRINTF #define WANT_ERROR_PRINTF #define WANT_LONGLONG_PRINTF #define WANT_LONGLONG_SCANF
Hans
:) There is actually a lot of truth in this for me. Somehow it is not so frustrating to discover a bug or work-around some suboptimal feature in free tools (or my own libraries). But when the vendor of a £1500 compiler eventually told me "oh yes, we never got function pointers working in that memory model"...
-- John Devereux
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.