Best Practices to Manage Complexity in Hardward/Software Design?

And to get rich off of selling failure. Few of us have that option.

John

Know the design priorities. This means understanding the most critical requirements.

Understand that all designs are trade-offs between conflicting requirements and all but one solution are sub-optimal given the solution space. No matter what design technique you use, including all diagramming and emergent design processes, this is true. You are not likely to find the optimal solution.

Depending on the end purpose, and if it truly is highly complex, multiple design teams are a good idea. They'll have landed upon differing sub-optimal solutions. Comparing and reconciling the competing designs can lead to a better sub-optimal solution.

Make sure you fully understand the problem. Break down the problem into sub problems. This is easiest for me to do in a hierarchical fashion, but is not the only way. There are many good, non-software engineering books that talk about problems solving and present diagraming techniques. If you are serious about design, you should read a few of these and even play with other diagramming techniques (though don't necessarily try to impose these non-SE diagrams on the corporation!). The design does not have to be hierachical, and the implementation probably should not be.

Prototype the difficult problems.

Support the design with tests (test before code!).

Walk through designs. Absolutely!

Know that requirements change.

Do you think that software is as hard to change as mechanical systems?

or is it possible that, even though software is easier to change than a mechnical system, it still requires some work and many managers/customers assume easier change = free change?

Well, it would be in a lot of businesses. The context of freezing, in my mind, is short time frame vs. medium or longer time frame. You adopt the concept of freezing in order to keep things moving in the short term. You adapt as you must. If you must adapt in the short term then so be it. However, that is "requirements pull" vs. "technology push". Requirements pull probably always trumps technology push.

I well understand your comment: We built a new wastewater plant (2 batch reactors) for our City. It was a

100% replacement. It was designed to have adequate capacity margin. It was deemed "topped out" by the Dept. of Ecology (DOE) when it came on line - with virtually no new hookups. Why? Because the engineers designed it under the assumption that inflow and infiltration (I&I) would be drastically reduced as part of a parallel project. The City is over 100 years old and some of the pipes are quite old and there is undoubtedly stormwater intentionally (but undocumented / unknown) piped into the sanitary sewer from long ago. Nobody in their right mind would predict a large % decrease in I&I by virtue of a single fixit project in a City of this type. But they did.

So, we built a 3rd reactor recently. As we improve I&I, the DOE allows more capacity but the rate of improvement is slow so we had to do this.

Did the requirements change? No; not during construction because the "requirements" were those assumed by the engineers. And yes; for the better but not as fast as expected and planned for. And no; because the engineers requirements didn't line up with the performance used by the DOH. So, the requirements were faulty and a change in a moderate time frame was necessary. Actually, other changes were done on the first two batch reactors to make them more productive - but, again, it was post-construction because that's when the idea struck. Actually, I think the latter change was a matter of requirements pull and realization that there was better technology available.

You freeze in order to keep out the inadvisable changes that would come if a variety of personalities had free rein. So rather than diminishing the value of the idea outright, it's best to consider that there might be good purpose and application of the idea.

Here's a context for it: You have a bunch of folks designing something and some of them are really bright but aren't very "common sense". They keep changing things. The project stretches because of the spinoff affects of the changes. Little is gained by the stretch - in fact, it costs money. So, you create some hurdles for the changes to cross; you set some (perhaps arbitrary) time points where there will be a "freeze" - which is a hurdle. First changes are easy to make. Then they are made a little harder to make. Then they are made pretty hard to make. Much can have to do with how "integrated" the thing is that's being changed or how far into production it is or how many already exist in the field that have to be maintained, etc. etc.

Fred

I depends. It's pretty easy to change the diameter of a hole in AutoCAD. So, if that's the context of a change to a mechanical system then it's just as easy as changing a line of code. In fact, it's equivalent to changing the value of a constant and that's how it's done. Now, if the next step is to reprogram an NC machine, then that's something additional. But isn't that similar to linking the new compiled code...? Maybe.

Fred

I'm not familiar with the spiral/agile model, but you'll have to do some pretty fancy talking to convince me to agree that something other than a strict development cycle flow is going to produce a better system in the long run.

The problem I've seen is that, in the process of stumbling over one's feet to get _something_ out that works, that the managers can touchie-feelie-see, one ends up skimping or dumbing down the high level design. Then, when that happens, the remainder of the project's development is crippled, with the end result either that it took MORE time than it would have taken had a proper high level design been performed, or that the performance/maintainability/extendability of the system is greatly comprimised.

You can't have your cake and eat it too. Either take a hit sticking to your requirements, or dumb-down your system.

--
%  Randy Yates                  % "I met someone who looks alot like you,
%% Fuquay-Varina, NC            %             she does the things you do, 
%%% 919-577-9882                %                     but she is an IBM."
%%%%            %        'Yours Truly, 2095', *Time*, ELO   
http://home.earthlink.net/~yatescr

I&I is always a problem. The sanitary lines under approximately 1500 slab-floor houses, including mine, were surreptitiously cracked just before the slabs were poured to provide drainage to avoid their getting damp from ground water. When it rains, it pours in. Since each community pays for its portion of operating expenses _and_sunk_capital_expenses_ in proportion of its flow, there is strong incentive for each participant to reduce its flow to the minimum. Expenditures for I&I reduction are sound investment. We could swap stories all week, but I think it's getting pretty remote for now.

Jerry

--
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

Oh, it's still a strict development cycle... if anything, it's more strict than the big design up front. It just panders a little better to customers and, in my experience, handles changing requirements much better.

Not at all. You don't dumb down the design, you just instantiate those parts of it that are clearest first.

It depends on what you mean by "proper high level design" --- don't get me wrong, there is still abstract, forward-leaning thought involved. It's just that most designs I've seen try to get too stuck into details too early.

Actually, you can. That's part of the point of the spiral / agile methods: allow some flux in requirements (it will always be there), but don't make dumb decisions about it.

Ciao,

Peter K.

Sure, but you have to know that before you write design docs, too.

They know about it in their own head-space. If you ask an engineer to build you a bridge, for example, you probably know where you want it to go and what kind of traffic you want to drive across it, but that's not all a bridge does. It's up to the engineer to anticipate additional loads from wind, temperature changes, tides and currents, etc. What the bridge does is deal with all of these things, but it's not fair to ask a customer to describe the parameters to which he expects his bridge to conform, because he doesn't even know what's reasonable.

There's some level of CYA you have to do when you're working on a contract basis, but even here I think that the onus is primarily on the software engineering professional to be personally sure that he is delivering what the customer needs. There is a lot of customer interaction required to create that surety, of course -- more than is required just to safely paint the customer into a workable corner.

Yes, there's a lot of that that needs to happen before coding, of course.

-- Matt

Bad experiences in defence systems means some defence customers are

*much* happier to make large physical changes than small software ones. The software change itself can be simple, but testing it for unforseen consequences can be a nightmare. Especially if it is safety critical (e.g. flight control systems).

I think most managers are just in love with the notion they can fix things economically after the product has shipped. Viva flash! :-)

Regards, Steve

The key problem with anything that produces an early prototype is managers tend to mistake the prototype for something near to market ready, however emphatically they are told it isn't. Without the pressure that brings I doubt people like Randy would be criticising more adaptive methodologies.

My experience with completely up front designs is they always produce something obsolete by its shipping date. I think anything more adaptive than that has to be of huge benefit, whatever drawbacks it has.

Regards, Steve

In my experience, those who are tasked with writing detailed software specifications (e.g. sales and marketing) are often not qualified to do so. It seems that the ability to write a very precise and detailed specification is a very similar skill set to the ability to write the software itself! So unless the spec writer is a programmer himself, it is usually severely lacking in detail and precision, at least in my experience. Your story (snipped) illustrates that concept nicely.

FYI, that photo is debunked here:

At least in my company, after you change the diameter of the hold in AutoCAD, you have to update required documentation, notify the vendor, possibly pay a change fee, figure out how to deal with the existing units with the "wrong" whole size (see through, throw away, return for correction), verify the fix has been implemented, etc.. The software is more like change the code, test it, and post it on the web site. It still takes work, but it's all in house, requires less time and management, and usually less cost. But for a manufacturing company that out-sourced its software, it could be the opposite!

Stung by the spell checker!

I accepted as a requirements ...

Jerry

-- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

It

Sort of. If you want to sell what I write, don't tell me what it should do. Describe to your prospective customer what it will do, and show me.

Jerry

--
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

I agree that this is an issue. It's a matter of "training" your stakeholders, though, and I've had more success with incremental involvement of stakeholders (in parallel with the incremental implementation), than trying to get their whole-hearted attention for the times when a big design needs it.

The phrase "attention span of a gnat" springs to mind.

There are certainly benefits and costs to all approaches. As previous posters have said, you need to choose the approach that satisfies as many of the interested parties as possible. The more you know about each approach, the better you can pick the right(est) one.

Ciao,

Peter K.

Ah, at last somebody who is capable of putting into words what I think!

The reason why I got concerned about these things (apart from having learned x86 assembly under that ridiculous MS-DOS memory model with

64k segments and the memory beyond 1M all but unreachable) was a code for scientific computations I used during my MSc project.

The MSC project, where I tested a method to do a certain type of data analysis, evolved into a code-development project for a company whose data I had used for testing. These people wanted to use my techniques in their R&D facility. They wanted to see if they could use the technique in their in-house data processing.

The brief for the post-MSc project was reasonable: Take the ecclectic program system, consisting of fortran, C and matlab code snippets, and convert into a C demo program that could be used as a stand-alone application by the staff.

Everybody understood that it was out of the question to re-implement the core scientific routine that was available in fortran.

I did everyting else: Wrote the main wrap-around, the data-I/O, implemented the tricks I had come up with in the MSc project. All that remained was to make a GUI for some interactive handling of the data, and to marry the fortran code to the rest of the program. The GUI had to wait for now, it could be handled by other means. The important thing was to include the core fortran program, so the main fucntionality was available.

The fortran program originated as some "example code" for a textbook on numerical modelling, that appeared around 1978-79. The five-page overview of the particular modeling technique, as found in that book, was the only "documentation" available for the code. The book gave an overview of the theory. It was of no use whatsoever, what the code was concerned.

The program itself was straight-forward. There was a main function that imported some parameters, organized some loops, and passed the parameters to the core subfunctions. The parameter IO in the main program was by file. Not that it mattered, I could write a wrap-around function in C and call the fortran core subfunctions from there.

Except it was not possible. Every status or error message generated by the core subfunctions were written to file, not passed back to the main function. Every computed parameter was written to file, not passed back. The concept of "returning the results in arrays for the main program to decide what to do" had not been respected. There was no way of implementing a smilar code from scratch; doing that is a project at the PhD level. (I know people who got their PhD degrees by writing codes with exactly the same functionally, if not in the same way.) There was no way of re-structuring the existing code, since no relevant documentation existed.

So the project blew sky high, if compared to the original brief. I was less than happy about that, but "luckily", the people involved decided it could continue as an R&D project, and so I got onto the path towards my PhD.

Since then, I have been very aware about documentation, error handling, data structures, data flows etc. What I call "design".

Rune

"Fred Marshall" wrote....

However, when you change the diameter of a hole in AutoCAD you then have to go thru the physical process of actually changing the hole diameter it the end part. With software, once the line of code is changed, you recompile and the change is done (minus any paperwork and distribution).

- Scott Frye