True Performance Comparison?

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
I was wondering about performance comparisons. Wouldn't using a
programming language other than assembly be an unfare comparison?
The reason I say this is for one that not all compilers optimize too
well. So the final code may be really incorrect.

In assembly however you can do the minimum that each processors
instructions take to do the task. Say add for instance. You can
create the insructions necessary to add two numbers and you know
you're using the minimum amount of instructions required for that
processor to accomplish that task. You do that for each one. Then
you're comparing the same functions like add, subtract, etc.

However with C or any other language you're depending on variances
in how the compiler gets the resulting code which may or may not be
optimized for the task. You see that when people fuss about someone
using a particular compiler with one micro and another with a
different micro and trying to compare them. Someone could just
complain that it was an ineffeciency of the compiler and not the
controller.

Thought I would open it up to discussion and find a way to compare
apples to apples.

We used to compare performance in just our coding by setting a bit
before our code started and clearing it when our code finished. That
way we could connect it to an o-scope and see the difference.
However before hand we just ran the before and after switch without
any code in between to account for code execution of the switching
routine. That way code execution in between could be measured. It
wasn't for comparing performance of microcontrollers but just to
code efficiency.  Of course at that time compilers we pretty crude.

Any thoughts, agreements, disagreements, or just discussion.  Love to
hear various thoughts on what people are experiencing.

Michael
http://www.geocities.com/microcontrollerworld /
http://groups.yahoo.com/group/MicrocontrollerWorld

Re: True Performance Comparison?

Quoted text here. Click to load it

Only if you're planning on writing your programs in assembly
language.

Quoted text here. Click to load it

The only thing that matters in the end is the performance of
the hardware/toolchain _combination_.

Quoted text here. Click to load it

The performance of hand-tuned assembly only matters if you plan
on writing your application in hand-tuned assembly.

--
Grant Edwards                   grante             Yow!  If elected, Zippy
                                  at               pledges to each and every
We've slightly trimmed the long signature. Click to see the full one.
Re: True Performance Comparison?
Quoted text here. Click to load it

With larger projects, the view of software changes from a bits-and-bytes
endeavor to an engineering endeavor.  Creating code in 'C' is cheaper, pound
for pound, then doing it in assembly language.

The goal is to finish the project.  The goal is not to display prowess under
the processor architecture.

Thus, I would argue that comparing 'C' is fair.  It fairly reflects what
will be done in practice.  It is true that bad compilers will penalize the
processor designers for inefficiences they did not create, but who said life
is fair?

I think the lesson to be learned is clear.  If you're designing a new
processor, pay very close attention to the tool chain that will or will not
be available to support it ...

Dave




Re: True Performance Comparison?
On 20 Oct 2004 08:15:30 -0700, Michael Monteith

Quoted text here. Click to load it

No.  Instead, using assembly would be unfair.  How are you going to
compare the following ARM instruction

   ldreqb  r0, [r1, -r2, lsr #27]

with equivalent for, say, AVR?  For '51?  For PIC?

For the record, this instruction loads byte (with zero extension) into the
register r0 from the memory at base address in r1 minus offset in r2
shifted logically right 27 bits.  And it only takes effect if Z flag is
set.  With proper sequencing of commands (eliminating register
dependencies) this command executes in exactly one cycle.


Quoted text here. Click to load it

"Use the proper compiler" is usually the answer.  And if the code is
"really" incorrect, what is the point of comparing it?


Quoted text here. Click to load it

http://groups.google.co.uk/groups?q=performance+apples&group=comp.arch.embedded .*


   Vadim

Re: True Performance Comparison?
Quoted text here. Click to load it

Well, that would indicate that the ARM is capable of doing something in 1
instruction that the AVR, 8051, and PIC can't do in any less than a much
larger number of instructions, therefore I'd conclude the ARM is faster than
all 3. ;-)

-->Neil



Re: True Performance Comparison?
Quoted text here. Click to load it

For that kind of application you may be better off with a much simpler
and less powerful processor and a CPLD or FPGA to do the shift register
work.

--
Ben Jackson
We've slightly trimmed the long signature. Click to see the full one.
Re: True Performance Comparison?
Quoted text here. Click to load it

Three words for you: $,$,$ :) Yes, it's been suggested. Blackboxing
the problem like this always turns out to be more money though, even
in an ASIC.

Re: True Performance Comparison?
michael_r_ snipped-for-privacy@yahoo.com says...
Quoted text here. Click to load it

IMHO,  that kind of performance comparison is primarily of interesting
to the marketing department (and perhaps CS professors).  Engineers
trying to get a product ready for market are more interested in the tool
chain ease of use, power efficiency, memory size and MCU and support
chip costs.   The cycle-by- cycle efficiency of one instruction set over
another  isn't all that important when compared with the multitude of
choices of tools,  memory and package sizes, power efficiency, and chip
cost.  

Then you have to cope with the fact that instruction set efficiency
measures depend a lot on the KIND of code you are writing.  Some
processors have horrible times with high-level languages, but do very
nicely at bit-twiddling.  (Else why would anyone buy a PIC chip???).

Mark Borgerson


Re: True Performance Comparison?

Quoted text here. Click to load it

To me, a high performance compiler is the one that lets me
write the most functionality per time unit while being able
to decipher it 5 years later with the usual documentation
of the code.
It lets me change some functionality 10 years later in the
shortest time, by reading the available code documentation.

The amount of ASM statements is not reaaly interesting.
The amount of flash is, hmm, tiny compared to 10 years later.
The clockspeed of the cpu, is hmm, well it could take less
current.

Rene
--
Ing.Buero R.Tschaggelar - http://www.ibrtses.com
& commercial newsgroups - http://www.talkto.net

Re: True Performance Comparison?

Quoted text here. Click to load it


Hi Michael,

I know we have discussed this before - just for the record below is a copy
of my previous communication on the results published on the FreeRTOS WEB
site (see http://www.FreeRTOS.org/PC )

"I don't think an absolute and undisputable measure of a microcontrollers
performance is practical when writing programs in C.  What the results show
is a 'comparison' of several 'systems' being used in a 'normal' manner.

'Comparison' because I don't attempt to give an absolute measure - only
provide results so different systems can be compared to each other.

'System' in that both the compiler and hardware are included.

'Normal' in that no attempt is made to optimise to the particular hardware.
Only C code is used.  If for example the hardware included a hardware
multiplier then the test does not specifically write assembler code to
ensure the multiplication is done in the fastest possible way."


Quoted text here. Click to load it

This is a topic that people often have a strong opinion on.  As with
everything it is always open to interpretation and dependent on what you
want to do.  Optimising for a particular microcontroller is contrary to
wanting to write portable code for instance.

Regards,
Richard.

http://www.FreeRTOS.org



Re: True Performance Comparison?
michael_r_ snipped-for-privacy@yahoo.com (Michael Monteith) wrote in

Quoted text here. Click to load it

It depends on the type of performance comparison you are interested in.  If
you want to compare processors for a specific application, then it would
make sense to use the development tools that would be used for said
application to develop the benchmarks, and compare the results.

If you are strictly interested in processor performance, then, hand coding
the most optimal assembly language using a very good assembly programmer
could make sense.

The latter seems like a useless comparison for real world applications,
unless they are also to be coded in assembly by a highly skilled assembly
language programmer in the respective assembly languages.

--
Richard

Re: True Performance Comparison?



Quoted text here. Click to load it

That is why you never compare Apples to Oranges.
Are you comparing Languages? CPUs, Languages?
How good in the assembly?  It is very possible to write bloated slow
Assembly code.
That is why bench marks should be taken with a grain of salt.




Re: True Performance Comparison?
Quoted text here. Click to load it

It depends upon which of the two questions you want to ask:

1) Is processor x faster than y?
2) Is the response time on x faster than y with application z?

If you're looking for raw processor performance testing (#1) to figure out
if x is faster than y, then yes, assembly language is the only way to do it.

If you're looking to answer whether or not a computing experience is faster,
then it's all going to be down to a combination of how well the program is
written, what language it's in, and what compiler it's running on.

Now plenty of people have taken an example of #2 above and conclude that it
means #1, including and especially Apple (remember the "G5 is the fastest
computer on the planet" ads? That's blatant false advertising). Just because
one app runs faster than another on a given platform doesn't mean that the
platform is faster.

As someone already pointed out, the only thing that really matters is speed
perception to the end user. To directly answer your question above, yes, any
other programming language would absolutely be unfair if #1 is the goal.

-->Neil



Re: True Performance Comparison?
Quoted text here. Click to load it

I'll put it bluntly.  Performance as such is a non-entity.  It simply
doesn't exist.  Might as well try to compare beauty.

Quoted text here. Click to load it

Depends.  On what performance it is you want to compare.  In
what circumstances.

Quoted text here. Click to load it

That pair of statements is plain nonsense.  Optimizing more or less
well is not a question of generating correct code.  If the code isn't
correct, it would be terminally stupid to even let it participate in
any performance comparison.

--
Hans-Bernhard Broeker ( snipped-for-privacy@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Re: True Performance Comparison?

Quoted text here. Click to load it

Damn.  There go my plans for the world's most optimizing C compiler:

---------------------------------8<---------------------------------
#!/bin/bash
cp /dev/null `basename $1 .c`.o
---------------------------------8<---------------------------------

--
Grant Edwards                   grante             Yow!  I am having FUN... I
                                  at               wonder if it's NET FUN or
We've slightly trimmed the long signature. Click to see the full one.
Re: True Performance Comparison?
Quoted text here. Click to load it

 LOL, no kidding.  They wouldn't adevertise and fight over who
optimizes the best.  Guess I've seen WAY too many instances of
compilers generating extra code only to make things easier on
themselves.  Ever looked at what compilers really generate and ask
yourself "why in the world did they do that?"  I guess in these days
people don't care as long as the code works.  I guess I came from a
time when you had limited processing power and you squeezed every bit
out of it and never assumed anything about a compiler.  It certainly
plays in when you're wanting to do performance comparisons.

Michael

Site Timeline