Re: Version Control in Embedded Systems - Page 4

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: Version Control in Embedded Systems



[...]

Quoted text here. Click to load it

Do you mean you cease to be an engineer while coding? Hmm.

Quoted text here. Click to load it

I'm sure the people who worry about time-to-market love the rapid,
simple approach, especially if it works the first time.

Where I come from, products are enhanced for years through upgrades,
while the underlying hardware is mostly static.


Quoted text here. Click to load it

BH


Re: Version Control in Embedded Systems
Quoted text here. Click to load it

Now try working in an environment where there are multiple hardware
versions in the field that need software updates from time to time. And
a customer base that can't always be trusted to match the h/w part
number with the s/w part number. This version control has to be
automatic.

--
Paul Hovnanian     mailto: snipped-for-privacy@Hovnanian.com
note to spammers:  a Washington State resident
We've slightly trimmed the long signature. Click to see the full one.
Re: Version Control in Embedded Systems
Quoted text here. Click to load it

   Even worse.  A customer has different version of a device, with
feature dependent software on different boards, and the idiot swaps
cards around, and ships a unit in for service.  Not only do you need the
software logs, you need to verify the serial number of ever circuit
board BEFORE you start repairs or upgrades.  Or, they have you upgrade
one unit, and try to copy the software to other units, built at
different times, with different rev numbers on the boards, and screw up
a million dollars worth of equipment, and you catch hell for their
stupidity.

--


Its August 5, 2003, so I'm 51 today!
Michael A. Terrell
We've slightly trimmed the long signature. Click to see the full one.
Re: Version Control in Embedded Systems
snipped-for-privacy@eds.com says...

Quoted text here. Click to load it

I think this is completely wrong, as regards consumer products.
Color displays sell better than monochrome.
Customers want the color of the box to match their other
components, which is why audio/video equipment is either
all silver or all black.

Selling consumer electronics is all about emotion, which is
why advertisements are written and illustrated in particular
ways.

MOST consumers don't know that there might be the identical
decoder in two DVD players, one selling for $100 and one for
$200.  They will pay more for specific brands, particularly
if they owned that brand before, or because someone they know
has it.  Other than that, price usually drives the decision.

And frankly, a good display, keypad, and case can cost several
times what the core electronics does.  Chip vendors are in a
continuous race to the bottom, but that doesn't change the
price of metal vs. plastic for the box.

--Gene






Re: Version Control in Embedded Systems
Quoted text here. Click to load it

C|N>K!

From what planet did you import this notion?

I take it you've never written firmware for mask-ROM, microcontrollers,
which are soldered directly onto the product PCB, to control cost?  Or
code synthesized into a compacted ROM, within an ASIC?

Remind me not to hire you, for any projects with a half-million-dollar
NRE or initial parts purchase. In fact, I think I'll put you on the as-
sembly line,  remanufacturing a few tens of thousands of product PCB's,
to replace MCU's for a code change.  Be careful not to peel any traces!

Once out of prototyping, the cost of changing embedded software is oft-
en every bit as high as changing hardware--and sometimes worse. (After
all, changing the odd SMT component, or adding a little flat-wiring is
a bargain, compared to replacing an MCU).

--
egr



Re: Version Control in Embedded Systems

: >
: > <cut>
: >
: > So fixing a software bug when released into the market/into production
: > even for embedded systems is easy <cut>
:
: C|N>K!
:
: From what planet did you import this notion?
:
: I take it you've never written firmware for mask-ROM, microcontrollers,
: which are soldered directly onto the product PCB, to control cost?  Or
: code synthesized into a compacted ROM, within an ASIC?
:
: Remind me not to hire you, for any projects with a half-million-dollar
: NRE or initial parts purchase. In fact, I think I'll put you on the as-
: sembly line,  remanufacturing a few tens of thousands of product PCB's,
: to replace MCU's for a code change.  Be careful not to peel any traces!
:
: Once out of prototyping, the cost of changing embedded software is oft-
: en every bit as high as changing hardware--and sometimes worse. (After
: all, changing the odd SMT component, or adding a little flat-wiring is
: a bargain, compared to replacing an MCU).
:
: --
: egr
:
:
My  statement was a relative  one(though  I  had   not mentioned that,  but
can be judged  from  the  statement which  follows).  So  what  I  meant
that  was  (read relatively)  easier  to   fix a  bug  in the  software for
embedded system than   it  is  to do  it with  hardware.
You  are  right  that   I have not written any firmware  myself. My main
job  is  configuration management, but I  have written code also.  I  have
worked   for  the european  leader in CE,   and  that  is  how my  idea  on
embedded  system comes  from.  It  might be possible that they have  better
process and  technology working for them to  get this   done easily(they are
in the  market  for more than 100yrs). At least  I am aware that  if  they
needed  to  fix  a application  bug in the  DVD  player,  all that was
needed  was to  create a CD with  the flash, which  would then   be
distributed  to  the dealers who would  then place  it in the tray of  the
DVD player,   which would self-program  itself,   (this process took less
than 5minutes). If that sound  difficult, I know   of  no  easier way even
in software industry itself.  A high   range  DELL server takes more  than
5minutes to  boot with NT/2K OS,  leave aside  Unix  style   OS.



Re: Version Control in Embedded Systems
Hi, Koushik:

Koushik Banerjee escribiˇ en comp.software.config-mgmt:

[...
] My  statement was a relative  one(though  I  had   not mentioned that,
Quoted text here. Click to load it

And a very relative one, you can be sure.

Quoted text here. Click to load it

Philips, you mean...

Quoted text here. Click to load it

DVD player?  Philips for sure, then.

Quoted text here. Click to load it

Well, you can bet you're pretty lucky for sure.  A DVD player is a player
after all, so there's an easy way to put the new software into the
hardware... though the player.  Even your procedure is a very expensive
one, since most DVD player will end up at the local Philips official
representant and this action will be billed by the hour!!!  Most other
embebbed software is not so lucky: where do you expect to place the
upgrading CD onto a microwave oven, or a TV set, for instance?  They will
be returned to the mother mill, maybe in a different continent, and that
will be rrrrrreally expensive, don't you think so?

Quoted text here. Click to load it

Well, I don't know of a single Dell which would take 5 minutes to warm up
the OS, less talking about barebones Unix (Both mininum i386 Solaris or
Linux boot up well under 20 seconds, and WinNT/2000 within a minute).

--
SALUD,
Jes˙s
***
We've slightly trimmed the long signature. Click to see the full one.
Re: Version Control in Embedded Systems
Jesus,

: Hi, Koushik:
:
:
: > needed  was to  create a CD with  the flash, which  would then   be
: > distributed  to  the dealers who would  then place  it in the tray of
the
:
: Well, you can bet you're pretty lucky for sure.  A DVD player is a player
: after all, so there's an easy way to put the new software into the
: hardware... though the player.  Even your procedure is a very expensive
: one, since most DVD player will end up at the local Philips official
: representant and this action will be billed by the hour!!!  Most other
: embebbed software is not so lucky: where do you expect to place the
: upgrading CD onto a microwave oven, or a TV set, for instance?  They will
: be returned to the mother mill, maybe in a different continent, and that
: will be rrrrrreally expensive, don't you think so?
:
Most embedded systems today I understand have their method of field updates.
I mentioned CD containing the flash software was one way of doing it. There
also was a mechanism using connectors, which could be plugged into the
EEPROM, by which you could burn the flash memory with the new software, This
feature works for all systems which work on flash memory, that is the TV
system has this feature, your mobile phone has this feature (you can look
for some dots inside your mobile phone, which are used to upgrade software.
CD is a cheaper way of doing it, considering no expenses, but for the
connector stuff, you need the hardware to be present with the dealer. That
is the difference.
Yes,   I understand it is not possible for PCBs or few other systems like
those, but I do not agree that it is not possible for all systems. And true
costs can be a factor but considering the cost of throwing away the whole
product versus the cost to make the product functional again -  I do not
feel one needs to think a lot on that.

Koushik



Re: Version Control in Embedded Systems
Quoted text here. Click to load it
true

I repeat that field-upgrading is *not* always appropriate for embedded
products. (See my earlier post.)

In many cases a better use of available cash is to get the code right in the
first place. A bug that renders a product non-functional should never be
shipped - period. Field-upgradability should *never* be an excuse for sloppy
work.

Steve
http://www.sfdesign.co.uk
http://www.fivetrees.com



Re: Version Control in Embedded Systems



Quoted text here. Click to load it

You're a dinosaur- make all your future products web appliance- access
the auto-junk via the cellcom-to-CAN- what the hell is this good for if
you can't update products on the fly.:-)


Re: Version Control in Embedded Systems
Quoted text here. Click to load it
systems do
To
applica-
Quoted text here. Click to load it
only
unspeakably

If Eric is a dinosaur, so am I. While I'd *love* to make all my products
field-upgradeable, I'm often constrained by cost. When designing, in
particular, high-volume products, I have to balance my budget to the nearest
penny. Fairly often I have to make a choice as to where to put the upgrade
system:
  - in the product (hardware costs incurred with every item)
  - in the lab and/or distributor's workshop (hardware costs limited to
those that actually need them)

This decision is also based on knowledge of the likelihood/frequency of
upgrades. If the code is good quality, and the spec is stable, there is no
good reason to incur recurring hardware costs for the sake of insurance.

OTOH, I always try to ensure a code upgrade path that doesn't involve
soldering... but even that can be a tough call.

Steve
http://www.sfdesign.co.uk
http://www.fivetrees.com



Re: Version Control in Embedded Systems
Quoted text here. Click to load it

There are many systems that must not have an update capability.
Yesterday (Thursday 7th August 2003) I was asked to specify an OTP part
because for security reasons it must NOT be field upgradeable. This is
for a state of the art system (for next seasons F1 cars) And yes it will
be on a CAN network.

There are many systems such as control systems for safety related
equipment, security equipment etc that do not want any way of upgrading
the code because the code shipped MUST be as certified.

Changes are not permitted. In fact many have to use a specified version
of the compiler.

As for "web accessible" some of the development areas I visit are not
permitted web access (for safety and security) let alone their projects.
Besides many of the projects don't have room for web access. I assume
you mean Internet access. A small but important distinction.

Not a dinosaur just working in  more professionals area than you are.


/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills  Staffs  England    /\/\/\/\/\
/\/\/ snipped-for-privacy@phaedsys.org       www.phaedsys.org \/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

Re: Version Control in Embedded Systems

Quoted text here. Click to load it

Not quite sure what your point is. Is Eric living in the past or is he
insightful?

My own opinion is that field upgrades are a "pain". It's not so much
as to the technical implementation that's the problem but more related
to the possible uses cases and company infrastructure that's the
killer.

If you're upgrading remotely, by modem say, one has to cater for all
possible scenarios of lost communications. You can restrict these use
cases by having only service centre upgrades, but in most cases for
low COGs products, it's cheaper to just replace the CPU board or even
provide the customer with a replacement unit.

The 3 main reasons why a embedded product would be field upgradeable
(in decending order of importance) are:

1. Marketing feature -- it certainly looks good on a brochure and
anyway your competitor has it. The customer perceives this as lower
maintenance cost as they've already conceded that the software is
likely to have bugs.
2. High risk development, so the software is likely to have bugs or
that the product has so much requirement creep that the software's
stability is questionable. Or that the development schedule was so
unrealistic, that some of the customer's requirements have to be done
in a 2nd revision of the software.
3. Future feature enhancement -- on its own is unlikely as it is
usually combined with Point 2. Typically these "features" were part of
the original requirements, but were omiited from the initial software
release due to schedule pressure (am I right!).

Here's a test which separates the men from the boys -- tell your
software team that they're about to develop a product of which at
least 50K units per year will be made AND the software will be going
to mask ROM and see how they react ;-)

If this type of product is new to the software team, then they will
need to up the ante on their Software Process, as this is a clear
example of "right first time" development.

Ken.

+====================================+
I hate junk email. Please direct any
genuine email to: kenlee at hotpop.com

Re: Version Control in Embedded Systems
Eric:

don't be so hard on the guy - I've done the kind of things you describe
(high volume ASICs, mask ROMs, NIST traceable ATE), but if I were to
assign this task to someone, I would make sure that they know the
pitfalls -yes mask ROM and ASIC require thorough debug before they are
manufactured, but I would not throw this guy into a situation where he
is in charge of making this kind of a decision if he does not know about
it.  Most of this comes from EXPERIENCE (if I had a buck for every
mistake I've ever made....), and  because someone does not have enough
experience, don't label him/her like that.   This is mentoring, required
in our field - and if you were to put someone in charge of something and
it failed - it's YOUR fault, not theirs - never throw someone in too
deep.  If we forget to mentor people, we will be SOL when we are done
doing things.

Andrew

Eric Roesinger wrote:

Quoted text here. Click to load it


Re: Version Control in Embedded Systems
           snipped-for-privacy@highSNIPlandTHIStechPLEASEnology.com "John Larkin" writes:

Quoted text here. Click to load it

A prime example of the need to "fix the customer before fixing the
system". The above siuation is wholly avoidable with the right
approach. Remember System Design beigins by getting the spec right
for yourself and the customer.

I have related in this and other groups my use of prototypes (on PC
and in cardboard cut-outs) as aids in "spec-bashing sessions". It is
quite extraordinary when you get a roomful of execs and operators
pressing the imaginary buttons and making comments about what they
expect to see happen as a result. The sessions need to be highly
structured and a good moderator is absolutely vital. The result is
usually a pretty good preliminary requirements specification that
one can get ones teeth into and not bite much fresh air. Usually
we do it all again a few weeks later but this time reviewing the
spec very thoroughly and really stretching it hard. The second
meeting everyone has to prepare for properly.
 
As you develop the system you are going to continue to find the
odd problem in the requirements specification. It is vital to raise
a problem report on this with the client and ensure he responds to
it promptly. Be on your guard to ensure that feature drift does not
creep into the requirements without your noticing it (I know some
clients try to do this). If your teams are on the ball you should be
able to spot this specification drift easily enough. When you do quote
the cost of adding the feature to the client and give him the option
of changing the requiremenst and paying for it or to limit his needs
to the original.

However good you are at getting the spec right first time there are
many instances where a change must be implemented in a myriad of
documents. You need to control the changes in a very orderly manner.
Such changes may come from anywhere; testing; users of previous
systems; the client maintenance etc. Encourage the filling in of
problem reports (hint: only one problem per report as this eases
closure later). Have a sound process for dealing with problem reports
and ensure they are always under review (even the simply answered ones).
Always record all problem reports submitted and track them until each
problem is resolved (by a new version issue).

My simple system development process is partly described in the
flowchart on the HIDECS page of my website and in papers I have
presented to a couple of conferences. It is in fact the central
change management core of the process. There is another diagram
that deals with the range of documents required at each stage of
development. You are invited to look and download the papers
if you want to find out more.

--
********************************************************************
We've slightly trimmed the long signature. Click to see the full one.
Re: Version Control in Embedded Systems
On Wed, 16 Jul 2003 23:09:04 +0000 (UTC), snipped-for-privacy@amleth.demon.co.uk

Quoted text here. Click to load it


All good stuff. Where I work we use the term "train"  rather than
"fix". "Fix" has the implication that you want to kill someone ---
which on the other hand with System Specs, may be the case ;-)

Quoted text here. Click to load it

What's also useful from these sessions, are the use cases & scenarios
as they could be used in System Validation and provide a guide to
Software Verification.
 
Quoted text here. Click to load it

Basically with new features (particularly when introduced in later
design) you have to play hardball with the customer and state changes
in terms of dollars ($$). Recording changes & problems provide good
metrics that can give one insight as to the cost of change, for future
projects.

Quoted text here. Click to load it

It's often the case with projects that too much documentation is
produced or not the right documents, particularly if the product is to
be submitted to some regulatory body. Why too much? Typically for
products that go to regulatory bodies, engineers will tend to produce
a lot of "just-in-case" documents, may be because they haven't
received guidance as to what documents to produce. If the product is
to go for CE Mark or FDA (510K) submission, then the engineer should
be made aware of what documents are mandatory at the onset.

Quoted text here. Click to load it

I would recommend an online tool to track problems & issues and make
it available to all team members (not just engineers). We use PVCS
Tracker and the use of it is outlined in our work instruction so that
everyone knows how it should be used.

Quoted text here. Click to load it


+====================================+
I hate junk email. Please direct any
genuine email to: kenlee at hotpop.com

Re: Version Control in Embedded Systems
On Wed, 16 Jul 2003 23:09:04 +0000 (UTC), snipped-for-privacy@amleth.demon.co.uk

Quoted text here. Click to load it

Right: locking the conference room door.

John




Re: Version Control in Embedded Systems

: > snipped-for-privacy@amleth.demon.co.uk ("Paul E. Bennett") wrote in
:
:
: Yes, One should design for bug free hardware as well as software. I
: tell my programmers that I want them to make life hard for our test
: engineer. Let him spend hours until he scrounges up a bug instead of
: just a couple of minutes.
:

If the tester happens to find a bug just after he starts testing, I think
either the specs were not cleanly defined(or rather the test cases dont
match the spec) Or the programmer is really a BAD one. In such cases it is
better desirable to do away with that programmer and get someone else good
to re-write the code (NOT revamp or fix the code), because it is easier
writing something altogether new yourself, than to fix some shit of someone
else. And anyways if the developer left behind some obvious bugs, his coding
style would also be very bad and less readable to others, less maintainable
and blah-blahs..

: Yes, we should use the most cost effective trade-offs available. By
: some that will be rigorous code inspection, by others it will be a
: solid test-harness. Others will find a combination of the two.
:
: There is one issue that I didn't see addressed.
:
: If a bug is found don't deny it. Sounds trite but I've seen it happen.
: The engineer that I'm happiest with assumes the truth of a bug report,
: tries to reconstruct it in his development environment, and then asks
: the question "Is this isolated or is this found somewhere else in my
: work (read Hardware and/or Software)?" IMHO, quick patches are
: worthless, even if you have a zillion installations. You'll just have
: to make another one. If your lucky the tester will catch your new bug
: before it gets out the door.
:
: This question could and should also be applied to marketing decisions
: (read GUI design and feature sets)
:
: Another point that ties in to Version Control (that's how we started
: the thread :-} ).
: There must be communication between the Development and Test Groups.
: At the very least the Test Group (and actually the developer himself)
: should ask "Did you touch shared modules?" The answer to this question
: will define how extensive the regression test for introducing bugs to
: other features will be.
:
: Yaakov



Re: Version Control in Embedded Systems
Quoted text here. Click to load it

I've only ever worked with one such developer - unfortunately it wasn't
just his code that was problematic, it was his entire attitude to work.
If he got challenging work he regarded it as being an attempt to stretch
him past breaking point, if he got routine work he described it as an
attempt to 'de-skill' him...

pete
--
snipped-for-privacy@fenelon.com "there's no room for enigmas in built-up areas" HMHB


Re: Version Control in Embedded Systems

: > I think it was "Peopleware" that first described those developers who
are a
: > net drain on a project, because the code they produce is bug ridden and
: > takes more effort to correct than it would have for someone else to do
in
: > the first place. (Some developers are an order of magnitude more
productive
: > than others).
:
: I've only ever worked with one such developer - unfortunately it wasn't
: just his code that was problematic, it was his entire attitude to work.
: If he got challenging work he regarded it as being an attempt to stretch
: him past breaking point, if he got routine work he described it as an
: attempt to 'de-skill' him...
:
: pete
: --
: snipped-for-privacy@fenelon.com "there's no room for enigmas in built-up areas" HMHB
:
I  guess then he was not looking  for working altogether.  People who  have
excuses for every action of others  (negatively) are better to be  left off
to themselves. Though   I  have worked with quite a few of them, one in
particular was pretty annoying more because of his attitude towards
programming/learning  and  taking suggestions from others.  His aptitude
towards computers  was  pretty low,  which  cannot be  taught. He  always
felt low on  himself if  others tried to correct him  though he was just out
of college with a different background.  But what  I learnt/understood  was
it was always   better to leave such souls to  themselves, they  and you are
better off that way.

Koushik



Site Timeline