ARM IDE

The compiler business is small and it is generally quite possible to contact the developers directly. Support staff deals with the day to day stuff but developers are generally very much involved in dealing with real issues. Unlike open source it is generally easy to determine who actually is responsible for specific design and implementation decisions.

"big-name test suites" offer organized methodical language testing to a specific standard. There are no similar test FOSS test suites available that I know of. GCC's regression tests the last time I looked were a small scatted collection of random code fragments.

One of the biggest problems facing compiler testing is the ability to deal with testing multiple code generation sequences that come from syntactically identical sources. Code generation for example that deals with variable placement context for example. Outside commercial tools very little serious work has been done to address this problem.

Regards

-- Walter Banks Byte Craft Limited

In message , Walter Banks writes

Quite so I can directly get in touch with all the developers and project managers for any of the compilers I sell. Then again I have NDAs with them as a distributor. So far I have not had any legitimate question from a user that we have not been able to answer.

More to the point with the complete version control they have they can tell me EXACTLY who did which source file when and the full history. Also they have all the supporting documentation. I doubt you can do this with any version of GCC

Quite. The Big Name test suites are developed to a standard as high as safety critical software. Full histories and documentation with formal procedures.

Quite. The problem is that whilst the big name suites have some provenance, as the authors are also on the C and C++ standards bodies and helped shape the standard they are writing a test suite fro the GCC test suite does not have that similar standing.

Of course it does. Neither Plum Hall nor Perennial are perfect. During the last five years, I've found two or three code generation bugs in a commercial compiler claiming to be verified by those two, with embarrassingly simple pieces of code (like a backwards for loop). Of course I hope my failing simple code snippets were now added to the compiler maker's test suite, to make sure this doesn't happen again.

Ironically, my score on gcc bugs is just one in 10 years. No, I don't want to draw any statistically significant conclusion from that, but I wouldn't say a compiler is significantly better than another just because the first's test suite has a "Plum Hall" label on it.

Define Safety Critical.

We're doing automotive stuff, and view almost everything as safety-critical. No people are instantly killed if the MPEG decoder in your car stereo crashes, the RDS interpreter tunes to a wrong frequency, or the display goes black. But it may irritate the driver (especially if the bug is accompanied with loud noise), distract him from traffic, and

Having a validated compiler just increases the probability that the problem was not introduced in the ".c"-to-".o" conversion, but does not eliminate it.

Of course a compiler will use things like kernel32.dll, which might behave differently on different versions of XP. One advance beta version of the linker we use behaved differently on XP and 2000.

Stefan

Well, you *think* you've answered their questions satisfactorily. Maybe they just gave up trying to explain the problem.

Multi-level support has its disadvantages when you want to go into deep details. Like, you want to know whether __mulli3 has been designed with CPU erratum #1009823 and application note #512 in mind, and whether you can safely call it from inline assembly (completely made-up example). If all you have is a first-level support contact, you'll have a hard time trying to explain them what you want.

Here, some phone companies implement the last mile using VoIP and DSL. Try calling them to find out whether they use G.711 or G.729. With one, fax will work, with the other, it will not. Now explain that to a support agent, who is trained to tell people to check that their devices are correctly plugged in.

As far as I know, most open-source projects (including gcc) have public, world-readable version control systems.

But if I have a problem I don't care who *did* X, I want to know who can

Stefan

No.

Why?

An end-user does not normally have these NDAs or this sort of influence with the compiler companies - getting developer contact can be hard or impossible if you are a small company with only one or a few licenses. Going through a distributor (such as yourself) is one way to get contact with the experts - assuming that all steps in the chain are happy to help.

Of course, that's a generalisation and there are plenty of exceptions. I'm sure Bytecraft will give excellent support directly from the developers, if we can judge by Walter Banks' contributions to this public newsgroup. I have also had excellent technical support from ImageCraft - not exactly a big player, but famous for their support.

On the other hand, I have had technical issues with a big name compiler in which I found a bug in their library. I even sent them a fix for it. I was basically told that they don't care about the problem, and that they were not going to fix it.

Technical support quality is definitely a widely varying priority for different suppliers - part of the job of a distributor is to know and recommend suppliers on such criteria.

I don't need to ask - I can simply look at the publicly available source code repository:

Yes. That is how it works. However the problem is that some questions (and I have seen these questions sent simultaneously to several distis and the compiler company) are answered on page 2 of the manual.

In many cases the person asking the question has very limited experience and the language and the target and is asking the wrong question completely. This is why compiler companies use first line support and distis to weed out the idiots, simpletons and those who seem unable to read the manual

The other point is I can discuss the problem with the end user and the developer in more depth having an NDA in place. It helps I am a qualified and experienced SW Engineer.

The problem is that once a user has the developers email address they thing the have a personal friend and help line for life. The first port of call (before the manual) is the developer.

Where I get any support from inside a company I always remove any direct email addresses to the developers before passing on the information.

Getting direct access tot he developers doesn't happen in any other industry.

BTW can the users of your companies products talk DIRECTLY to you about the software functionality of your companies products?

Where required they do.

Quite so.

Interesting. Love to know who that was....

Distributors recommend specific tools on many criteria. (btw there is no universal "best" compiler for anything.... well that's not strictly true. For some of the arcane MCU there are some times "the best" but in other cases there are many factors to weigh up. None of the factors are size of margin before you ask!

For EVERY version of GCC?

You know what all the code does and why it is there?

How do you know?

Granted, developer support is a *lot* more helpful than phone company support, but I have already got back bug reports for formal reasons without anyone having looked at the problem. One that I remember is that a filesystem library was unmarshalling stuff using 'char', not 'unsigned char'. Everyone sees that this is wrong by looking at the code; I sent file name and line number. The answer I got back was "please send a self-contained test case including an image of the failing file system". Had I been talking to a developer familiar with the code, not a support guy, that would probably have been fixed that in a second.

(Various events led to the situation that we don't use that library; hence I have not dug much further.)

Stefan

... snip ...

How do you compensate for variations in parameter passing, calling conventions, etc. in various libraries? Do you mean you have the common library source, and compile it under each system?

We don't compile the library or the compiler. We don't start with the source.

We test the library supplied with the compiler. We test on windows XP, this is the host system for these compilers. They only run on windows.

What "phone company"?

This is a problem I mentioned before there are three types of Char. the sign of plain char is implementation defined. The standard library uses plain char quite a bit

Absolute bollax. When talking to the internal support team or the developer the first thing they ask for is "please send a self-contained test case including an image of the failing file system". They want to see an example because a lot of users do not understand the compiler, the language or what is actually happening.

Statistically I have found it worse amongst those who say "If I had the source I could fix it" which is why I am very unconvinced when people say having the course of a compiler is useful

The other thing is the disti and the compiler support team may have seen this problem many times before.... and know the solution. It is not always (or indeed often) a compiler bug.

As for talking to "the developer" the support teams are often the same as the developer and the rotate tasks and have a spell on the support desk. In other places the support team is far better at handling support problems than the developers. It is a different sort of game.

The support team are usually qualified and experienced engineers with quite often far more experience than the people who call in. Especially when it comes to the language and compilers.

As for "it would have been fixed in a second" what is not true. You can just change something in a compiler. You need to look very carefully at what else it effects and do full regression tests (for the build) and full tests (Plum hall, maths, language extensions, corner cases etc etc but only AFTER you have looked to see if you need to add new tests. And done all the documentation

Some compilers where you have the source and can "fix things in a second" are a complete bag of nails and the source is a complete nightmare.

Like many you see the problem and the solution far too simplistically.

You have just proven my point. First-level support assumes the user is too stupid to use the product, and users with a clue have a hard time to convince them that they actually know what they're talking about.

In this particular case, the problem was in CD multisession handling, thus "sending an image" would have meant to burn a CD and shipping it over the ocean, along with our hardware because the reference hardware has no CD drive. Expensive in time and money.

The problematic code was about this: char buffer[4]; fetch_buffer_somewhere(buffer); address = buffer[0] + (buffer[1]

In message , Stefan Reuther writes

The first line support I know are as capable and experienced as the developers. There is usually little difference between the two technically. The difference tends to be the developers are less tolerant than the support guys. The support guys will also be able to answer most questions faster because they have been asked them more than once.

No more than talking to the developers. What makes you so special and different to anyone else who calls?

I think you have just proved my point. If you can't describe the problem and encapsulate it without having to send the hardware and a complete CD a compiler developer is less likely to want to talk to you than a support person.

Assume makes an ass out of you and me...

Far be it for me to disagree with you

Alright. I misinterpreted your "with a single standard library. We test that."

Well, as the saying goes: At court and on the open sea, we're all in God's hand.

You are liable if you are *found* negligent in a court of law. Now in an ideal world people would only be found negligent if they actually did neglect significant parts of their duties. But ours is, of course, not an ideal world.

On a similar note, it's impossible to validate any chip with more than about 60 pins, simply on the grounds that even a test of all possible combinations of static binary inputs to its pins (not to mention analog inputs, voltage thresholds, timing, sequence, ...) would never finish during the earth's life expectancy.

You do realise he is talking about the source to the *library* here, not the compiler? An experienced developer of a high level is going to be just as capable of understanding, debugging and correcting a toolkit's library as the library developers are - assuming there is decent information available about things like calling and naming conventions in the library. They are not going to be as skilled at specific tasks such as writing the library code in a way that it compiles properly for all its targets, and perhaps they are not going to write code as optimally or portably. And they are certainly not going to be able to run all the tests and qualifications.

But in situations like this one (and cases I have seen myself), the user is perfectly capable of finding and correcting a bug in the library source code if he has access to it. The fact is that some (by no means all) compiler support lines cannot deal appropriately with this situation. They either assume that the user is the problem, or they follow rigid and inappropriate routines such as asking for complete examples of the problem.

I agree that having the source to the *compiler* is not helpful to most users when it comes to finding or fixing bugs (it can be useful for other things, such as for changing development platform, archiving tools, or as a safety net in case the compiler developers stop development on the tools). But no one is talking about compiler source code here.

No. Plum Hall and Perennial are language conformance tests. The kind of feature you are describing is implementation specific and not covered.

Plum Hall and Perennial both are well respected as independent test suites that validate language implementation against the standards. Both companies are part if WG-14 and offer an independent view of how the standard should be interpreted.

There is a lot of confusion about the meaning of successfully running these test suites. Compiler companies run these as one of several standard and private tests to validate and release compilers.

Regards

-- Walter Banks Byte Craft Limited

I understand all about these support issues, and the reasons why it is often undesirable to allow users direct contact with developers. I have also been on the other side of the equation. Yes, we *do* have end users who talk directly to myself and other software developers - not many, but some experienced and technically competent end users have direct contact. It can be of great benefit to both parties. I've also had occasional direct contact with developers of hardware parts we buy - again, it benefits both sides.

The important thing here is that we are talking about expert users - not average users, and certainly not the "idiots, simpletons, and those who seem unable to read the manual".

For almost all open source projects, many low-cost tools, and a fair number of high-cost tools, a major source of support is from public web forums, newsgroups, or mailing lists. A good public support forum is frequented by the tools developers, as well as expert users who enjoy using the tool, and are happy to help other users. Clearly there are times when conventional technical support, either directly to the suppliers or via a distributor, is better suited - for example, you can't sign an NDA with a public forum.

Incidentally, the support I have had from ImageCraft has been mostly on their mailing list, with many of the questions answered directly by the developer. Many other people's questions have been answered by me, and other "high level" users.

I'd prefer not to name names (especially as I can't remember which of two possible suppliers it was...).

I have always been a fan of choosing the best tool for the job, and that choice depends on many factors - technical support is only one of them.

The quality of the logs and comments probably varies somewhat, as it does in any project from any developers. But the history goes back a fair way - look at .

Of course, that particular source code repository is for the main FSF gcc tree. There are a great many other gcc development trees for different targets, experimental branches, or alternative development branches. Some of these will be directly open to the public, others will be internal to their developers (the GPL license requires you to make the source code available along with binary releases, but makes no requirements on the source history). If you are getting your gcc binaries ready-made (such as from CodeSourcery) rather than checking out and building your own copy, you might want to ask your particular supplier about their source code tracking and logging. Of course, CodeSourcery commit most of their changes back to the main FSF tree, so you can look them up there.

So you have to take "reasonable steps" to validate. As the industry norm is to use certain test suites as part of it that is the standard.

If you can find another test suite of similar coverage (Perennial is some 70,000 tests for a C compiler, written by known people who also develop the standard) quite apart from the build and regression test suites etc along with showing that care has been taken in the development of the compiler (ie the process, specifications and documentation etc) that that is OK.