What Software Engineering Process is best suited for Embedded Projects - Page 3

Do you have a question? Post it now! No Registration Necessary

Translate This Thread From English to

Threaded View
Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

Because the number of possibilities to test is exponential with each
feature.
example - 2^416%,  2^1665%526, ...

If you have a small number of features, it maybe possible to completely
test.
However, it doesn't take a large number of features before complete testing
is impossible.




Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

We long ago recognised this so we now agree a set of major and minor
features with our clients.  We test all major and minor features
individually and all combinations of major features.  This limits the tests
to a manageable subset of the impossible total and picks up all but the
most obscure bugs which the client agrees to pay to be fixed.

Ian


Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it

If features are mostly independent, you may be able to reduce the
number of testcases (or increase the guaranteed coverage) by testing
many pairs simultaneously.  For example, 19 on/off features.  2^^19
combinations, 171 pairs of features.  But you can cover all 171 pairs
with 10 testcases, like so:

 1a 2b 3a 4b 5b 6a 7a 8a 9a 10a 11a 12a 13b 14a 15b 16b 17a 18a 19a
 1b 2a 3b 4a 5a 6b 7b 8b 9b 10b 11b 12b 13a 14b 15a 16a 17b 18b 19b
 1a 2a 3a 4b 5a 6a 7b 8a 9b 10a 11a 12a 13a 14b 15a 16b 17b 18a 19b
 1b 2b 3b 4a 5b 6b 7a 8b 9a 10b 11b 12b 13b 14a 15b 16a 17a 18a 19a
 1a 2a 3a 4a 5b 6a 7a 8b 9a 10b 11a 12a 13a 14b 15a 16a 17a 18b 19a
 1a 2b 3b 4b 5a 6b 7b 8a 9b 10a 11b 12b 13b 14a 15b 16b 17a 18b 19b
 1b 2b 3a 4a 5b 6b 7a 8a 9a 10b 11a 12b 13a 14a 15a 16b 17b 18b 19b
 1b 2a 3b 4b 5a 6a 7b 8b 9a 10a 11b 12a 13b 14b 15b 16a 17b 18a 19a
 1a 2a 3b 4b 5a 6b 7a 8a 9b 10b 11a 12a 13b 14a 15a 16a 17a 18a 19a
 1a 2b 3a 4a 5b 6a 7b 8b 9b 10a 11b 12b 13a 14b 15b 16b 17b 18b 19a

The number of testcases grows with the log of the number of
independent features.  There are tools for generating such sets of
testcases.  (I just wrote one,
http://burtleburtle.net/bob/math/jenny.html .  I have a hammer.  I've
been looking for nails.)

Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

Can you define independent in this context?

Ian


Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it

Yes: If a testcase is expected to test many separate pairs of
features, then all of the features in it must be exercised.

Two ways for some features not to be exercised are errors and
comments.  If an error is raised, it may be raised before some
features are hit, so any pairs involving unhit features aren't tested.
 For example syntax errors in SQL will prevent any data access from
being tested.  A comment, something like the <!--> or <pre> commands
in HTML, will cause most inner commands to be ignored, so any pairs
involving inner commands won't be tested.

The tools AETG and jenny allow lists of restrictions, that is,
features that should not appear together in any testcase.  That allows
a set of testcases to be generated that will avoid those feature
combinations that are known to raise errors or otherwise cause
trouble.  A couple dozen restrictions aren't bad, but zillions would
be awkward to handle.  Thus, "mostly" independent.  Any restrictions
need to be tested separately.  The tools can cover triples as well as
pairs, but pairs seem to be what people actually shoot for.  ALLPAIRS
is another such tool, but it doesn't handle restrictions, and it only
covers pairs.

Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

Well I wouldn't _assert_ that it's impossible (hence the
question mark) to fully test _all_ real time systems but,
certainly, with many larger systems it's next to impossible.

Others have mentioned the huge number of possible
pathways etc but if you add to those the asynchronous
nature of external real world events both to the currently
executing instruction and to each other I fail to see how
those can all, sensibly, be tested

Some years ago I had a small project using a PIC with
one external interrupt. The PIC had an internal bug which
caused it to miss the external interrupt if that interrupt
occurred during T2 of a particular instruction. I don't know
how that sort of issue could be tested for?

Mike Harding


Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it

In any project you need the following requriments:

1. You know the System you are designing
2. Knowledge of CPU used inside out
3. Tools that you can test in real time
4. Before you design the system make sure it can be tested
   and it will work and the system output will work
5. No point in using UML if the devlopers do not know it
6. Same tools should be used within the project
7. If purchasing 3 party products, make sure they work and you
   are not the Beta test for product purchased
8. Keep it simple at the start of the project.
9. Not a standard, to many RTOS and To many tools that do not work
10. Managers who can not communicate with people
11. Feed back to devloper and customers

Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

I did not say we have no quality system just the we have no 'formal' quality
system.  We have a quality manual, we write quality plans, we have software
engineering guidelines and in fact guidelines for all aspects of
development including electronics and mechanics.  However, none of it is
mandatory and because of the wide range of project types we indertake it
would be quite wrong to impose a rigid system on every type of project.
This means the the method of ensuring quality is decided and agreed with
the client at the proposal stage.

Quoted text here. Click to load it

On the contrary, as I mentioned in my very first post on this topic, the
most important factor is the people and the culture in which they operate.
This is a far better means of ensuring quality than a rigid system which
simply frustrates people.  So in some cases clients are quite willing to
accept 'trust us, we have done this sort of thing before, our reputation
rests on our success, it'll be alright' credentials.  After all you are
only as good as your last job.  if you screw up, word soon gets around.  As
far as winning new contracts with this approach is concerned, we would only
take this approach if the timeframe was extremely short in which case any
competitor with a rigid QA system would simply be unable to meet the
requirement.

Ian


Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it

I can not belive this:

Quoted text here. Click to load it

Who keeps comeing back ? Can you give me more back ground of what you
do because if you work in the
  car industry
  projects are under 6 months
  Add on's or new features to a project
  Manager have no skills
  No buget for Process
  Company is going bankrupt
  
  
  

Then i can understand where you are coming form.

Re: What Software Engineering Process is best suited for Embedded Projects
           snipped-for-privacy@waytohere.com "Gerald Maher" writes:

Quoted text here. Click to load it

The model does not really matter. So long as you have a clear idea
of what you require as deliverables at each stage of development
and have recorded the process you use (simple ISO9001 requirement
often stated as "write what you do and do what you write").

Secondly, ensure that you have a secure repository for the project
documentation and that you make someone responsible for keeping it
in order. A CVS or RCS system will help electronically if you wish
to go that route.

Thirdly, have a system by which problems get reported and then proven
to be fixed before it ships (no more than one problem per report or
it becomes a nightmare very rapidly). Change management is very
important and you need to impose that no-one makes a change to
production code unless they have a written work instruction to do so.
Combined with this, of course, is regular technical review, run
properly and given a level of status that management and marketing
cannot overrule the outcome of the review reccommendations.

Finally, subscribe to Jack Ganssle's newsletter "The Embedded Muse"
which contains a lot of useful stuff. Even Jack's books would be
useful I think. If you visit Jack's web-site I think you will find
a lot of the back-issues of the newsletter. No. 85 was quite a good
one. Of course, keep asking questions in here and if you need some
help with formulating your procedures I have some templates that
can give you a good start.

--
********************************************************************
We've slightly trimmed the long signature. Click to see the full one.
Re: What Software Engineering Process is best suited for Embedded Projects
snipped-for-privacy@amleth.demon.co.uk ("Paul E. Bennett") writes:

Quoted text here. Click to load it

But make sure the documentation gets updated properly.  Making the
updates a chore to do ensures that it won't get done.  If the person in
charge of the documentation doesn't have much day to day interaction
with the people doing the actual work, things will get out of sync.
If there's an approvals process for updating the docs, then the
updates will either be late or not get done.

It is not uncommon for upper management to feel that everything is
documented properly and accurately only to find out later that nothing
matches.

Also realize that the end result of all this may be that you can
quickly and accurately figure out where things went wrong, but it
might not help you prevent things from going wrong in the first place.
Amazing and spectacular failures are usually created with a well
thought out process and are nearly always well documented.

Quoted text here. Click to load it

After integration occurs the person that fixed the bug should again
sign off on the ready-to-ship code.  If the person reporting the bug
is in-house, they should sign off on it too.  It's much too common an
error to have the modules work perfectly but then fail mysteriously
when integrated together.

There should be a real traffic cop that has control over what gets
into a release and what doesn't.  Feature creep and endless last
minute bug fixes can ruin a well planned project; yet this is a
regular occurance.  This traffic cop should have veto power even over
more senior people or the marketting department.  At one company I was
at this position rotated with each release, so that the same person
wasn't always the most hated person in R&D.

--
Darin Johnson
    "Look here.  There's a crop circle in my ficus!"  -- The Tick

Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

The documentation update is supposed to be revisited every time you
make a change to the product. It is just one more item on the checklist
at the end of stage review.
 
Quoted text here. Click to load it

Part of the end of stage review is to make sure it all matches. Otherwise
the review has failed. Documents must be updated before product modification
as you need the documentation to ensure inspect/test against. It is not
permitted to just check the update. It is a re-test of the part, assembly,
unit, whole system.
 
Quoted text here. Click to load it

Hence my building in the need to be able to manage the changes that will
arise out of the problem reports (my third point).
 
Quoted text here. Click to load it

In my environemnt theclient will sign off on it too. However, as that is
at the high integrity end of the market (some life-critical) it is rather
understandable that they feel they should.
 
Quoted text here. Click to load it

Interesting description of the "Configuration Manager". Yes, such a post
holder does need to be a little bombastic but he does need to be polite
enough about it as he is also part of the team.

Of course, here we get into the realms of team make-up and I always
favour the "Surgical Team" approach (multi-disciplinary). A small
number (no less than three and no more than 7) is best for the really
high integrity stuff.

--
********************************************************************
We've slightly trimmed the long signature. Click to see the full one.
Re: What Software Engineering Process is best suited for Embedded Projects
snipped-for-privacy@amleth.demon.co.uk ("Paul E. Bennett") writes:

Quoted text here. Click to load it

Things that are "supposed" to get done, don't.  It's a fact of life.
Pragmatism versus idealism.  I've never had a "stage review" and am
not even sure what it really is.

I've often been in situations where documentation is supposed to be
updated but it isn't.  And I've seen this cause tremendous problems.
Thus my advice - if a process involves onerous amounts of work, then
chances are it won't get done properly, so make the process easy to
follow.

Example: I add a new command to the command line interface, something
used only by developers.  I now add half a paragraph to a document
that wrote, and take it to the official documentation person (not an
engineer).  She says I need to fill out a documentation change
request.  First line on the request wants a documentation number, so I
ask how to get a documentation number.  She says to request one.  From
who ask, and find out it's from her.  Can she just fill in a number
when I hand her the form?  Etc.  At some point I petition my boss to
just remove the convenience feature rather than have to update the
documentation.  (On the other hand I actually _tried_, and I had one
of the only software related documents that was official)

I'd like to work for an idealistic company, one with well thought out
processes that actually work.  But I've never been at such a place.
Sometimes I wonder if they even exist anywhere outside of USENET.

Quoted text here. Click to load it

We referred to these as "release managers".  Sometimes they were good,
sometimes mediocre.  They're high visibility jobs, so sometimes people
forget that getting the job done is more important than looking good
(the bad managers are the ones that refer to "my release").

--
Darin Johnson
    Caution! Under no circumstances confuse the mesh with the
We've slightly trimmed the long signature. Click to see the full one.
Re: What Software Engineering Process is best suited for Embedded Projects
snipped-for-privacy@amleth.demon.co.uk ("Paul E. Bennett") writes:

Quoted text here. Click to load it

This is a classic case of the tail wagging the dog and unfortunately an all
to common result of introducing a 'quality system'.  Thay should be
controlled by you not the other way round.  So the system should be you
send her a memo which says please add this paragraph to this document,
taking out all the necessary numbers in the process.  Report back to me
when this is done (and anyway I would like in done in the next two days).

Ian


Re: What Software Engineering Process is best suited for Embedded Projects

Quoted text here. Click to load it

In companies by whom I have been employed, that "supposed" was read as
"always". There was no way, with their procedures in place, that a product
(hardware or software) would get modified without such changes in
documentation already having been completed. My whole career has been in
companies where QA and High Integrity Systems were the end result of
a large team effort. In one of those companies I managed to simplify
the procedures to a point where we were able to run the whole document
change management on four forms and a register. The company had about
200 employees/contractors and by the process, so simplified, won
TickIT certification on its software developments.
 
Quoted text here. Click to load it

I agree, the update process should never be made onerous. The KISS
prinicples apply as much to the processes we use as much as the products
we make.
 
Quoted text here. Click to load it

[%X]

Many companies that run decent development processes have gone to the
expense of having mult-part forms printed (already numbered). This is
not always necessary and your young lady could have simply added the
number then passed back a copy of the form for your record. There is
no need to make such tasks complicated or onerous.

Quoted text here. Click to load it

They do exist in the real world and I can confirm they are not just
a figment of usenet imagination. I have introduced a few companies
to the simple procedures that I developed back in 1982. I even
brought two software companies together as they individually had the
two halves of my process enshrined in software. One bought the rights
to the other's software and now sell it under their own banner. If
you want it all electronically then the tools are there (assuming you
have enough money to pay for each active seat your development
requires).

--
********************************************************************
We've slightly trimmed the long signature. Click to see the full one.
Re: What Software Engineering Process is best suited for Embedded Projects
I'll give you two examples from past experience, both from the telecom
industry. One was being part of a team of around 15-20 people in
Motorola. It was further subdivided into a "platform" team,
application team, validation team, and network management team. 4-5
people worked on hardware, 4-5 people worked in validation, 2 on
network management stuff, and up to 8 on telephony apps.
It followed the V-model, where, after the requirements were written,
then the validation team worked on a test plan to test each
requirement, while the developers continued on with architecture and
coding. THIS WAS THE MOST STABLE PRODUCT I'VE EVER SEEN.

Now fast forward to my days in Ericsson at Menlo Park, CA. It seemed
to follow NO process, and at best, it was best approximated to Extreme
Programming. There were several small builds, with each subsequent
build creating more bugs. This project took two years to stabilize.
(Is it stable now? who knows). Quality sucked, the validation team had
no requirements to work with, so all they did was load the system with
calls. The product was unstable so much that companies didn't buy that
telephony system, and chose Cisco's or Avaya's instead. This led to
the inevitable closure of that branch and it was moved to Ericsson's
headquarters in Europe. I'm sure Ericsson is a disciplined company.
Many big companies are disciplined, except for when they buy startups
whose half-cooked source code was written by code cowboys and hackers.

I agree with the earlier reply that the waterfall model is not real
and just seen in textbooks. I worked with the V-model, and I trust the
resultant quality from such a process discipline.

-Mike



snipped-for-privacy@waytohere.com (Gerald Maher) wrote in message
Quoted text here. Click to load it

Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it


Hi Mike ,
your term "code cowboys" and "hackers" is great

But i find it hard to belive that Ericsson ( the Bigest telcoms in
swedded ) follow no Proces, Is that the same in Sweeden  ?

That is like saying Siemens of Germany have no Process,
GE and Motorola of USA has no process
Nokia of Finland has no process

Re: What Software Engineering Process is best suited for Embedded Projects
snipped-for-privacy@waytohere.com (Gerald Maher) writes:

Quoted text here. Click to load it

All of those companies are HUGE.  They have no universal process that
applies to all groups in all countries.

Consider that these companies have research divisions separate from
product creation divisions, departments acquired through buying out
smaller companies and startups, internal political rivalries, etc.

--
Darin Johnson
    "Floyd here now!"

Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it


Yes i know in every sheep field there are always one or 2 black sheeps

Re: What Software Engineering Process is best suited for Embedded Projects
Quoted text here. Click to load it
... snip ...
Quoted text here. Click to load it

No, there are usually one or two sheep, one side of which we know
to be black :-).

--
Chuck F ( snipped-for-privacy@yahoo.com) ( snipped-for-privacy@worldnet.att.net)
   Available for consulting/temporary embedded and systems.
We've slightly trimmed the long signature. Click to see the full one.

Site Timeline