Re: Version Control in Embedded Systems

Hi, Koushik:

Koushik Banerjee escribió en comp.software.config-mgmt:

[... ] My statement was a relative one(though I had not mentioned that,

And a very relative one, you can be sure.

Philips, you mean...

DVD player? Philips for sure, then.

Well, you can bet you're pretty lucky for sure. A DVD player is a player after all, so there's an easy way to put the new software into the hardware... though the player. Even your procedure is a very expensive one, since most DVD player will end up at the local Philips official representant and this action will be billed by the hour!!! Most other embebbed software is not so lucky: where do you expect to place the upgrading CD onto a microwave oven, or a TV set, for instance? They will be returned to the mother mill, maybe in a different continent, and that will be rrrrrreally expensive, don't you think so?

Well, I don't know of a single Dell which would take 5 minutes to warm up the OS, less talking about barebones Unix (Both mininum i386 Solaris or Linux boot up well under 20 seconds, and WinNT/2000 within a minute).

--
SALUD,
Jesús
***
jesus_navarro@undominio.net
***

Hi again Koushik:

Koushik Banerjee escribió en comp.software.config-mgmt:

[and a lot of uneeded text follows]

First of all, I would ask you NOT to answer after the text. Just leave what is relevant for your answer and do it just where it do has any mean!

What Steve was telling you was that he had found his "best possible way" by doing the way he was doing it: going to the point it was about the most direct and simply way he could think about!

As it has already been answered to you, that isn't really true: there are two famous "idioms" regarding programing that never will really be repeated too enough: Do it as simply as possible (but not simpler); and do it the most expectable way. Ten lines for a clear and expectable 'while-then' loop will be less bug-prone than a "clever" 'for' oneliner for sure! Someone named it "programming-on-jeans" versus the "too clever" "programming-on-smoking" concept.

Again, that's what you're teached in school. When you affront a 4-6K lines project (hardware or embebbed projects tend not to be too long) the "formal" OO approach not only isn't too efficient (if you tend to be too "formal" you cannot take advantages from the oddities of the hardware you're managing) but it tends to be too buggy too (all your formalities, being such a "little" project is overload -the percentage of API-related code lines vs. "productive" code lines is too high, that can fit well within your "the more code lines, the more bugs" theory).

--
SALUD,
Jesús
***
jesus_navarro@undominio.net
***

Koushik:

yes, many embedded systems can be designed to be easily field upgradable - you just need a fixed bootstrap routing and enough hardware that works to communicate. FPGA's may be easily fixed in the same manner, if one loads them from a piece that can be field upgradable

- i.e. - load the thing from flash via a micro, and your VCS can be implemented in all of your software. ASIC and mask ROM people come from a different world, because the mask charges are outrageous, and you can't fix burned silicon in the field, but for volume production, it's the only way to go.

Andrew

Koushik Banerjee wrote:

variables,

controlled

you

the

you,

since

Frankly, the only thing that is going to give *me* sleepless nights, it not having complete control of what runs on the systems I design, and the faults that can result. What looks simple and clean in an HLL may not be, in terms of the code that actually runs, on the MCU.

Now, I will grant that there are benefits to compile-time checking over run- time "defensive programming"--particularly for "access methods", which can resolve to simple inline fetches; but understanding the system is far more important than the programming language used to implement it.

or

the

No, it depends on understanding the meaning (and in embedded work, often the implementation) of the lines of code one writes. In many cases, it also depends on the availability of tools.

A good verification team would eliminate the "couple of expensive tries to get it right." Our ASIC group has put out three ASICs that were first pass successes.

Of course, we're talking about pretty big designs for non-consumer products - so development time is on the order of many quarters (4 or

5), not months as Philips would be doing. But that still doesn't excuse a good verification team.

Marc

You're a dinosaur- make all your future products web appliance- access the auto-junk via the cellcom-to-CAN- what the hell is this good for if you can't update products on the fly.:-)

[...]

Do you mean you cease to be an engineer while coding? Hmm.

I'm sure the people who worry about time-to-market love the rapid, simple approach, especially if it works the first time.

Where I come from, products are enhanced for years through upgrades, while the underlying hardware is mostly static.

BH

nk

is

ood

eone

oding

able

It is tempting to take an easily found bug as a sign of programming=20 incompetence. But the programmer's ability to do system or integration te= sting=20 is often limited. The only thing the prorammer can always be charged with= is=20 unit testing. I would think all of us on this newsgroup have seen bugs cr= op up=20 because of integration or system issues that weren't the fault of the pro= grammer.

Dwain

=2E

, n

--=20 Dwain Wilder Bear Meadow Folk Instruments

I arise in the morning torn between a desire to improve (or save) the wor= ld and=20 a desire to enjoy (or savor) the world. This makes it hard to plan the da= y. =97E. B. White

systems do

To

applica-

only

unspeakably

If Eric is a dinosaur, so am I. While I'd *love* to make all my products field-upgradeable, I'm often constrained by cost. When designing, in particular, high-volume products, I have to balance my budget to the nearest penny. Fairly often I have to make a choice as to where to put the upgrade system: - in the product (hardware costs incurred with every item) - in the lab and/or distributor's workshop (hardware costs limited to those that actually need them)

This decision is also based on knowledge of the likelihood/frequency of upgrades. If the code is good quality, and the spec is stable, there is no good reason to incur recurring hardware costs for the sake of insurance.

OTOH, I always try to ensure a code upgrade path that doesn't involve soldering... but even that can be a tough call.

Steve

Not quite sure what your point is. Is Eric living in the past or is he insightful?

My own opinion is that field upgrades are a "pain". It's not so much as to the technical implementation that's the problem but more related to the possible uses cases and company infrastructure that's the killer.

If you're upgrading remotely, by modem say, one has to cater for all possible scenarios of lost communications. You can restrict these use cases by having only service centre upgrades, but in most cases for low COGs products, it's cheaper to just replace the CPU board or even provide the customer with a replacement unit.

The 3 main reasons why a embedded product would be field upgradeable (in decending order of importance) are:

1. Marketing feature -- it certainly looks good on a brochure and anyway your competitor has it. The customer perceives this as lower maintenance cost as they've already conceded that the software is likely to have bugs.
2. High risk development, so the software is likely to have bugs or that the product has so much requirement creep that the software's stability is questionable. Or that the development schedule was so unrealistic, that some of the customer's requirements have to be done in a 2nd revision of the software.
3. Future feature enhancement -- on its own is unlikely as it is usually combined with Point 2. Typically these "features" were part of the original requirements, but were omiited from the initial software release due to schedule pressure (am I right!).

Here's a test which separates the men from the boys -- tell your software team that they're about to develop a product of which at least 50K units per year will be made AND the software will be going to mask ROM and see how they react ;-)

If this type of product is new to the software team, then they will need to up the ante on their Software Process, as this is a clear example of "right first time" development.

Ken.

+====================================+ I hate junk email. Please direct any genuine email to: kenlee at hotpop.com

John Larkin wrote in news: snipped-for-privacy@4ax.com:

Open Source has the virtue of not being done in private. Real authors leave their name on their work, for better or for worse. They participate in the support groups for their work, and their email is public.

Imagine if the creator of "Clippy" (the Office paperclip character) were so exposed! ;)

--
Kenneth Porter
http://www.sewingwitch.com/ken/

Or worse yet, MS Bob!

--
Paul Hovnanian     mailto:Paul@Hovnanian.com
note to spammers:  a Washington State resident
------------------------------------------------------------------
If your only tool is a hammer then every problem looks like a thumb.

[snip]

programmer.

Yes. That's all the more reason to keep the software development as closely coordinated with the hardware as possible in embedded systems. Version control (and every other development discipline) needs to be applied across the whole project, not just each piece separately.

--
Paul Hovnanian     mailto:Paul@Hovnanian.com
note to spammers:  a Washington State resident
------------------------------------------------------------------
Smoking is one of the leading causes of statistics.
                -- Fletcher Knebel

Now try working in an environment where there are multiple hardware versions in the field that need software updates from time to time. And a customer base that can't always be trusted to match the h/w part number with the s/w part number. This version control has to be automatic.

--
Paul Hovnanian     mailto:Paul@Hovnanian.com
note to spammers:  a Washington State resident
------------------------------------------------------------------
Marching to a different kettle of fish.

Extreme Programming is done, in one version, by pairs of programmers. Again, better code gets done whan it's done in public. It's interesting that a 2:1 increase in manpower is justified by the better quality.

Is lynching legal in Washington State?

John

whose wife thinks Clippy is "adorable."

programmer.

i had to repair a crappy test fixture a couple years ago. The engineering tech that built it had sent it to the production floor with over half the tests mis-programmed. He would be looking for the right voltage, but the wrong polarity, and other obvious mistakes.

The test techs had a list of tests that were bad, and the fixture missed a lot of defects. I replaced the worn out wiring harness and fixed the software. I ran about 100 boards to verify that it worked perfectly. That is, till someone in SMD assembly swapped some .01 µF caps with resistors that fed one of the analog MUX inputs.

It was the first thing tested, and the boards passed, but when they were installed in the equipment, they slowly charged, and caused a severe DC offset problem. The test supervisor came to my bench waving a board, claiming it had never been tested.

I set it up and ran the test for him, and it passed. He told me what it was doing, so a quick look at the schematic, and another board revealed the assembly errors. While he was ranting about defective software, I copied the test to the end of the test. By repeating the test I verified the initial test, and that there were no swapped parts. I had it ready and running before he stopped long enough to catch his breath. After he saw it was fixed, he just walked away shaking his head.

--


Its August 5, 2003, so I'm 51 today!
Michael A. Terrell
Central Florida

Even worse. A customer has different version of a device, with feature dependent software on different boards, and the idiot swaps cards around, and ships a unit in for service. Not only do you need the software logs, you need to verify the serial number of ever circuit board BEFORE you start repairs or upgrades. Or, they have you upgrade one unit, and try to copy the software to other units, built at different times, with different rev numbers on the boards, and screw up a million dollars worth of equipment, and you catch hell for their stupidity.

--


Its August 5, 2003, so I'm 51 today!
Michael A. Terrell
Central Florida

In comp.arch.embedded John Larkin wrote: [...]

I don't think you're getting that thing about the manpower increase right. Just because "Extreme Programming" has two people working together, at the same screen, all the time, doesn't imply one of those two would have been unemployed otherwise. The key to XP is not how many people work on the code, but how they *do* that work.

It's more like this: XP uses half as many development machines compared to other methods, because two team members always share one box, at all times. I.e. it's not a 2:1 increase in manpower; if at all, it's a 2:1 *decrease* in machine power.

The only real drawback is that you can't use the typical cubicle office space pattern easily --- XP teams need more space per cubicle, and they'll be talking all the time, so you need better sound isolation between cubicles. In other words: you need actual offices, not cubicles.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

someone

coding

maintainable

It is tempting to take an easily found bug as a sign of programming incompetence. But the programmer's ability to do system or integration testing is often limited. The only thing the prorammer can always be charged with is unit testing. I would think all of us on this newsgroup have seen bugs crop up because of integration or system issues that weren't the fault of the programmer.

Dwain

But who handles the integration ? I suppose it is someone from the development group. And to end it there is a integration testing, where we are supposed to test what would be typical integration issues. I understand that there would be still some which could miss this, but the idea is to have them at minimum. I cannot comment much on the hardware perspective, but in software, a domain expert would know what cases need to be tested out of the system test cases to catch bug in this phase.

Koushik.