ideas on document set preparation process

I would again stress the benefits of LaTeX here. I can think of several key points:

LaTeX sources are revision control friendly - as plain text files, you can easily see differences in them.

You can include a document section within other documents. For example, if you have a list of error codes that should appear in many documents and must be consistent, then you just have a single error code list that is included by all documents that use it - with only one place to update, it will always be consistent.

Because everything is text based, it is all amenable to scripting and makefiles. The list of error codes (for example) could be held in a simple tab-delimited text file, and a Python script could process that file to generate a C header file and a LaTeX include file for documentation - with everything being built through a makefile. You can also arrange for the documentation build process to check out specific versions of the files from the revision control system before building the documents.

And of course you have the quality of LaTeX formatting, solid pdf files with hyperlinks, indexing, file attachments, etc., generation of diagrams with graphviz/dot, inclusion of syntax-highlighted source code, and so on.

My suggestion is to talk to a company that specializes in such systems. An example is Agile, a system that I was quite happy with but that's just an example.

I do not know LaTeX but editing software is not really the answer. You need something way bigger, something that ties in schematic sets, layouts, mechanical CAD, BOMs, design reviews, module specs, in your case TSOs, and so on. For example, it should include sign-off control for ECOs and design reviews (not all signed off -> no release).

I can't comment on the actual techniques to employ.

So I'll comment on something else: For documents going to a customer, if there's more than a half dozen engineers or so, you should have a full- time technical writer. If you had a team of software and electrical guys would you all pitch in to design mechanisms, or would you hire an experienced ME? Ditto for technical writing.

If management can't see the light on this, get everyone to start logging how many hours they spend on keeping documents consistent, preparing documents for external consumption, etc. After you've collected a quarter's worth of data, if think you can show that enough hours have been burnt to justify a full-time position, take your data to management and point out that you can keep doing it, inefficiently, and it can keep dragging you down. Or, they can hire someone who's actually good at it.

Hi David,

David Brown wrote: []

and you benefit from concurrent editing. The copy-modify-merge paradigm is so powerful on LaTeX documents that you could simply /forget/ somebody else may be proof reading the document and correct typos or syntax (yes, we are non-english writers in an english world!)

I remember working on an article co-authored by other 4 people and the day before submission we were applying changes like hell, each in his individual corner without the need to even talk about the change. It is sufficient to coordinate on substantial changes that would affect the message, otherwise everything could have been done in parallel and flawlessly.

And LaTeX *forces* you to think in terms of links because it's way easier than copy/pasting. On the contrary Word instills habits there are hard to remove and often those are the worst to fight against.

The aim is to think structural, encapsulating information in objects (e.x. files, but could be also DBs) and linking them.

I thought about 'continuous integration' for documentation as well, with Jenkins updating the repo and building automagically all the documentation set whenever there's a change. If one day we will achieve that I'll quit, it means they won't need me any further ;-).

We are trying to improve our toolset around LaTeX behind the scenes, we have established a 'core' team of individuals with a roadmap to make an environment based on LaTeX to write documents efficiently and with higher quality. Unfortunately this effort is not recognized at the enterprise level and we are doing it on our own expenses.

But beyond building the necessary environment for LaTeX (scripts, classes, templates, ...), we are trying to demonstrate how our current way of doing documentation is not only unsustainable but also a weak point of our products that does not reflect their quality.

We need to raise awareness before coming with solutions, and in the meantime we need to get ready when the chance arise to show what's available 'out there'.

Most likely, actually, the solution is not *just* LaTeX, we could imagine that our 'data modules' are stored in a database that people fill with different tools (maybe a web interface) and then the LaTeX document queries all those data modules in. IIRC the LaTeXDB allows you to do so.

Al

Hi Joerg,

Joerg wrote: []

That's a valid point. Does somebody know about any company out there doing this type of analysis/consultancy?

Your point is valid. As of today we've been working with shared directories (only recently a real VCS has been put in place) where everyone could modify nearly *any* document at his wish. All too often the schematic was not the last one, the datasheet was not up to date, the power budget table was not correctly referring to the last mods...and so on.

This is more an issue of 'configuration' rather than document writing and we are trying to cope with it with a home brewed db that allows to sign the docs. Unfortunately the db contains only those documents that are considered delivered, so no place for the /en cours/ doc. True is that we have SVN for that.

Al

Hi Tim,

Tim Wescott wrote: []

It's a good argument and often that's the sentiment of the team; why the hell should I spend hours with the header of my document?

The logging suggestion might be a good idea, it really depends on how much people affected are going to commit themselves to prove something to the management. Anyhow having some data may be a game changer, suddenly you show the potential benefit and it would be difficult to undermine.

I've never worked with technical writers, but our document set is not negligeable (in the end is hundreds of documents) and I'm wondering what kind of processes do they follow to maintain coherence and consistency throughout.

If anyone *is* a technical writer or *knows* one that may help in the context discribed in this thread, please do contact me! :-)

Al

ble.

The only comment I'd add is that you probably need a circuit version contro l system.

The projects that I was involved with had numbered versions for each chunk of electronics, with numbers representing major revisions - usually corres ponding to a new printed circuit layout,and alphabetic tags for minor modif ications - changed components values, and cut-and-link modifications to an existing printed circuit layout.

The document structure included slots for justification/explanations of eac h revision and modification,and every document had to be tagged with it's r evision number and - mostly - the modification level.

The distinction between this and revision control system is that several ve rsions of a circuit can be active at the same time - development will be in venting the next version, while trying to get the most recent version to wo rk, while production will be shipping the previous version, and service eng ineers may have to deal with even older versions.

I agree with this. I've worked on teams where the full-time engineering staff was about 3 people and it was still helpful to have a tech writer.

That was how I worked with the technical writer on my most recent "big" project. Mostly we were doing a user manual. My job was to provide accurate content (text, screen shots, sketches) and his job was to fight Word and occasionally to make a nice drawing out of my sketches.

For some simple user-facing stuff (like software update notices and instructions), I liked to write plain text documents myself. I could check these into the version control system, and I could ship the document along with the software update. The user got a separate copy of it, but they could also read it right on the target system if they wanted - text viewers were available but Word, OpenOffice, etc were not.

It doesn't have to be much... even a piece of paper that people write down their hours on once a day, and then give that paper to the "logging guy" once a week, is enough.

Our documentation set was not nearly that big, and we didn't have a formal documentation spec to follow, but I found that the tech writer was not shy about referring to how the manuals were done for other products, some of which did have formal documentation specs. For some things, there were company-standard document templates that could be used.

A little bit of it is normal version-control stuff: there *must* be a place, that only the tech writer can access, that has all the work-in- progress stuff he or she needs. They may keep old versions, or copies of documents from other projects, for reference - this is OK, but nobody else should get a potentially old document.

Some tech writers still prefer to operate by using red (blue, green, pink, ...) pens on paper to make revisions. You may need to provide your tech writer with a big desk/table and a big filing cabinet.

Matt Roberds

Colleagues have recently authored several technical books for established publishers, using the Asciidoc format. This is like an extended Markdown in that it's intended to be easily readable in its source form, and be just plain text (so is easily managed in a revision control system). The difference is that it's fully powerful - it's semantically equivalent to DocBook.

Combine that with a private Github repository, or a self-hosted GitlabHQ instance, and you really have a dream system that easily does everything you've asked for, and should require little maintenance.

GitlabHQ and AsciiDoctor are both open source, with good active and growing communities and commercial support options.

Clifford Heath.

How worrying. I thought hard traceability was a requirement for all aerospace components. I have been on the edge of the business from time to time - one of the applications of kit we used to make was to check that turbine blades were made of the right defect free raw material.

This sounds penny wise pound foolish. A good technical writer come documentation champion can make all the difference since it is their job the keep the manuals and engineering diagrams in step with the evolving product. I have also found contract technical writers useful in another way - they ask obvious questions that users *will* want to know and are so obvious to designers as not to be documented at all.

It is also better use of their time - they will be familiar with the right toolset(s) and be doing it all day and not just occasionally as an adjunct to their own technical specialism. Screen captures for software manuals is truly mind numbing work that is best contracted out.

Reviewing can be done by almost anyone that can read and spot mistakes in diagrams. Ideally people not directly in the authoring team - you tend to read what you think you have written rather than what is there. Reviews also should just seek to identify errors, ambiguities and defects (and not to try and fix them in the meeting).

Approval sign off should be limited to a small number of trusted experts who are quick readers and diligent about checking stuff.

This may well annoy your technical experts in the early stages of drafting a document unless you have a nice seamless version control system that automates bookin and bookout of working documents.

You also need something to flag up that there is a later revision of this linked to document.

An idea of how big your semi-large sets of documents actually are would help judge things a bit better. My previous but two organisation went down the route of cubic yards of controlled documents and maintaining the damn things in a self consistent state became a full time job!

I was never that enamoured of them because some of the stuff documented stated the blindingly obvious and did not describe how it was actually done but how someone in a suit higher up the chain of command with no experience on the shop floor thought it should be done.

I'd avoid the database idea. Databases are nice for many things, but I don't think they are a natural fit for data modules. In particular, you don't (normally) have version control, history, or logging - it would be like moving back to your shared directory system but with a less convenient interface.

Hi Bill,

Bill Sloman wrote: []

I'd love to get the hardware guys on this as well. Unfortunately revision control is often viewed as some sort of additional burden instead of a relief.

I've personally saw cases where entire pieces of a schematic were suddenly gone between one officially released version and another and since the guy who followed the change is not there anymore there's no absolute idea on the reason why that happened!

In our drawings too we have place for that, but is not the mindset of those who design the boards. On top of it I'm not sure the tools easily integrate with VCS. We have Cadence licenses which are 15 y.o. and according to our guys there's no VCS feature in the tool.

Anyone with experience on revisioning schematics/layouts? What about mechanical drawings?

Unfortunately if you need a separate document to include all that you may still lack of coherency between the document and the schematic entry. On top of that it would be nice to have some sort of 'diffing' between two versions of the schematics, but I'm not sure where we are today. And what about 'merging'?

I completely agree. We have what we call BreadBoard or Elegant BreadBoard which is a functional representation of the final unit and often doesn't need to meet all the requirements for a specific product (like reliability or radiation tolerance). It is modified and fine tuned heavily, is where we train our brains and test our ideas but we still need to keep track of all changes and revisions since is a deliverable.

Al

"In our field you cannot simply live without documentation and it is the only trace available to make a full and exact copy of the product should the customer wish."

I.e., "Documentation is important"?

If you had said "Safety is important", wouldn't you expect to have a "Safety Officer" on staff to ensure that valuable issue is adequately and ACCOUNTABLY addressed? I.e., that there is a blank on the sign-off for the "Safety Officer" to signify the design/implementation/manufacture satisfies the (stated) safety requirements for your project/organization?

Yet, (below) you indicate that your organization doesn't even pay "lip service" to this wrt "Documentation".

I.e., step 1: create this position step 2: put the position *squarely* in the release path -- nothing goes out the door without its blessing step 3: hire someone competent for the task step 4: be ever aware of step 2 so this person's requirements/suggestions are given serious consideration (else you risk not shipping ANY product!)

Probably because you have th ewrong people preparing those documents.

Why are changes made? Do you have a firm definition of your target? Or, are you making it up as you go?

Documentation Officer/Czar

You seem to be focusing on too many of the little details instead of getting our team (incl management) to openly acknowledge the importance of Documentation in your product/life cycle. Until that idea pervades their consciousness, you'll never get the result you seek. At best, you'll put mechanisms in place -- that breed contempt for the person who "stands in the way" by *enforcing* these rules.

I.e., just like you can't "legislate quality" into a product. The people involved have to *believe* in this (quality) in order to achieve it. Same applies to the Documentation role (or Safety, etc.)

Fix the culture, first. Then, folks will come up with solutions BECAUSE THEY *WANT* TO ADDRESS THE PROBLEM (because they believe in it)

Wait. I've not said that we do not track what we are doing (or writing). The lack of standard I was referring to are for published material and relies on the separation of content from presentation. On top of it content is handled in a Common Source Data Base which keeps track of type of infomration.

All our mods are tracked down and defects are documented, after a certain stage in the product development even if we find the mistake internally we are obliged to inform the customer about it (it does not usually happens on prototypes unless we want to keep track of the defect).

The technical staff is made out of ~40 engineers, all of them are involved in writing documents. No single one is spared, at any level. And while I agree that technical writers may do the job better than what we are doing, I believe that we would need a platoon of them.

One of our latest delivery, for a Manufacturing Readiness Review, included 105, between documents and drawings (mechanical and electrical), for a system that has *two* sets of *two* boards, in cold redundancy. Consider that we have ~10 reviews to go through, each potentially updating the entire documentation set. Even if we employ technical writers, we should have a mechanism to make the whole documentation set coherent, not simply relying on their capability.

We currently have something like 4/5 projects going on...do the math and you'll find that most of the time is spent on documentation and not development.

On a later budget for our FPGA development we put in ~1000 hours of development (rtl and verif), for a total of 3200 hours! Guess where the rest of the hours go?

I agree with the fact that writing documents is a job by itself and engineers are often not to keen to spend time on it (even if I personally do not understand why), yet is an essential part of our product that we cannot disregard.

We have a review meetings and checklists which help concentrating on critical stuff. The quality engineer chairs those meetings and minutes everything that has been said. An bug is opened if we need to take corrective actions and/or clarify questions. A second meeting is scheduled after the last deadline in the list of actions and all bugs status is reviewed and agreed upon.

This is what normally happens on schematics, layouts, drawings, code, but not on documentation. Usually a doc is submitted for review and no formal process is followed, people who need to sign are asked to read but often do not have appropriate time to do it so mistakes are left in.

I do not understand this at all. A VCS should never be on your way. We are using SVN and most of people use TortoiseSVN that even an idiot can use. There are some 'advanced' features that might be left to experts, but committing changes and updating the repository are two mouse-clicks away.

This is something we should configure. I used to have mail alerts which could be easily filtered in your mailbox. I also believe that there are tools that warns you if an updated version exists.

see above.

Most of our projects have strong review boards which would never accept gibberish. From time to time people sneak in some junk with the hope that it goes throug; sometimes it is stopped before going out, sometimes it goes out, sometimes it passes a review, but very rarely it passes

Al

I've started reading about S1000D

LaTeX alone does not solve the need to have a single source of data, it does facilitate it though. Yet you could easily decide to re-implement or copy the same type of info anywhere in your set of LaTeX documents.

With a db you take care that same type of information is not reproduced. The document can contain only *one* single reference for that type of information and a second one cannot be included. Mechanisms like that avoid unnecessary work and ensure consistency.

If one result should be available to several document, it must be tagged and the db should allow the extension for such an element. Only then it can be structurally included in several documents.

Indeed, while I'm writing, I realize that with this approach the LaTeX document is only used to extract data from the db and formatting (but not to introduce extra information that is not present in the db).

Maybe other tools might be more apt to the job instead of LaTeX (Docbook?).

Al

Yes.

We do have Quality Officers that review our docs and they are very good at it, but 98% of their remarks are mistakes we could avoid employing a more reasonable process.

This would be the mandate of our quality engineer who proof reads and checks for inconsistencies. This approach is far from being efficient with the actual way of writing docs. No one is trained on what content goes where and learning is done through mistakes...a very lengthy and costly process.

Is never a static target. To my knowledge *only* the Shuttle On Board Software had the privilege of a 'quasi-static specification'.

Still we can say that between formal 'change requests' from our customers we live with a static target. Yet changes do happen, often due to mistakes which we track on qualification samples with Non-Conformity Reports (or NCRs). An NCR triggers an investigation and a mitigation/correction activity that typically results in changes.

What rarely happens though is the update of the documentation set (analysis, description, ...). Most of the time this is due to the amount of formatting problems our /cher amie/ Word is challenging us with. So, instead of facing formatting problems each time we modify the doc, we tend to postpone this inevitable and painful moment until it cannot be pushed forward any longer (typically one week before delivery!).

As mentioned elsewhere in this thread, the amount of documentation we need to provide cannot be handled by one person. It should be handled either by an organized group of writers or by an organized group of engineers.

The Czar should only make sure the process is followed, and this is typically what the Quality Officer ensures.

You are right. The issue should be aknowledged at higher level and awareness shall be made before any step is taken. Still people often tend to be scared of problems without solutions, therefore I'm trying to build the case and provide a plan to address it.

Providing a solution without showing the problem is yet another mistake often made, which generally brings nothing but a simple 'why do we have to change?'

I tend to disagree here. Relying on people willingness to understand on the value of documentation is too risky and provides a very irregular quality pattern (some are very dedicated, some are careless). Even if they see their name signed on the front page of a document, they are still protected by a chain of approvals which undermine the involvement. The quality should be built in the process, people should follow the process and Quality Officers should ensure they do.

AFAIK in civil aviation there are teams of external officers who monitor quality standards, but there are also other officers who monitor the first ones to see if they are really doing a good monitoring job.

I see three types of people around, those who lead, those who follow and those who don't give a damn. In the end it is sufficient to *fix* those who lead, the rest will either be on board or won't care anyway.

Al

Do they creep in because the *right* information is "too difficult for folks to get their hands on"? Or, because folks aren't committed enough to the process to *ensure* they have the "right information"?

Because you have folks writing who aren't "qualified" to write!

Would you hire a plumber to frame your house? A carpenter to plumb it?

Writing is more than correct spelling/vocabulary/grammar. Just as drafting a schematic is more than knowing which symbols to use for each component.

I've known lots of *brilliant* people that were lousy communicators. There's a special skill (like every other discipline) involved in being able to organize information in a concise, accessible manner.

Then how can you expect any kind of *quality* from your design? What guarantees have you that some fundamental assumption on which you've based an implementation won't later be invalidated by a spec change?

(groan) This sounds like a f@ckup just *aching* to happen!

Then pick a better tool and people who are appropriately skilled in using it! Your time should be spent preparing prose (and illustrations, etc.) *not* coaxing the document into the "right" format!

Or, adopt a medium where format is not constrained. E.g., HTML focuses on *content*, not form (per se). You don't worry about page breaks, line breaks, hyphenation, etc. -- they just "happen". And, happen

Fine. Get a department together. The point is, if documentation is important, then it should be a top-level sign-off item. So, the "Czar" can hold up the entire release process if folks haven't done their jobs properly. This requires a strong person in that position and a strong corporate commitment to this aspect of the product. If "market pressures" can force the Czar to compromise, then just do away with the position entirely and EXPLICITLY ACKNOWLEDGE that you don't REALLY care about docs! Then, when you lose a contract because of it, you can shrug and let folks wonder who the fool was to ignore that requirement!

-------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

well said! "Don't bring me a problem unless you've also (done my job FOR me and) brought me a solution!"

"We don't! It's not *my* name on the corporate letterhead..."

You can't do that. How can you "guarantee" that "Joe" has accurately described those items for which he was responsible? And, thoroughly? E.g., you can create standards for drawing schematics but can't force folks to pick *good* signal names ("A", "B", "C"? "STOP" vs. "/RUN"?)

Look at how much *grief* happens when you try to impose "coding styles" or "naming standards" on software teams. Someone is *always* grumbling (as if their "artistic license" has been revoked)

Great! Build *two* documentation departments! You seem to be trying to get that function "for free" out of staff whose primary responsibility lies in other areas.

I think you are ignoring the group (which may be one of the above three... or a fourth!) that effectively "becomes obstructionist".

Despite all the checks and balances in aviation, medicine, pharma, etc. there are still screw-ups. Is this because MORE process is required? Or, because the process is so impersonal that folks don't buy into it any more than necessary to "cover their asses"?

Hi Don,

Don Y wrote: []

It is practically impossible to know in how many places the same information is scattered. A snapshot of an hardware interface may be in the Interface Control Document, the design description, the Worst Case Analysis, the Failure Mode Effects Analysis, the Hardware/Software interface, as well as assembly instructions, Testing procedures, and God knows how many others.

Now imagine the customer comes with a change in the interface: "we realized that it would be better for us to have an LVDS interface instead of an RS422, it shouldn't be a big problem for you, they are almost identical". Yes, right, they are almost identical, but now that the team has changed because of internal reasons, it would be nearly impossible to know how many damned documents have to be updated.

Sure, changing the components on the schematic and doing a new layout is not even a one day job, but chasing the information to be fixed in a pile of documents you have not done is going to be a real PITA.

I completely agree with you. We lack of a corporate vision about doing documents properly. And the whole process cost us enormous amount of money, with a quality that is barely acceptable.

Couldn't agree more. And I think that in a two people shop you have to be able to write docs, but when you have 40 people runninng on 5 different projects I think the workload can easily justify the employment of an expert (or a team of) to take care about the docs.

You can *never* expect the specification to be rock solid from day 1 till shipping. Developments start early, if not prematurely, and systems are complex enough and geographically sparsed enough to imply an understandable level of evolution.

Customers try hard to avoid as much as possible to invalidate 'fundamental assumptions', but if that happens it means you need to scrap off everything that depends on that assumption and restart. This process is not for free, neither the customer nor the contractors.

Sure the specs should not change every other day, but up to the Preliminary Design Review it is possible and normal to accept changes. After that is a bit more critical.

OTOH a changing spec does not affect quality. Either you meet the spec or you don't, there's no quality associated with it. The quality concerns the assurance that every process, submitted to review and validated, is followed (that's why the perform reviews, inspections, tracking, ...).

Didn't follow.

I couldn't agree more. And if you prefer, I wouldn't give a damn about what tool should a technical writer use to get the docs in proper format. What I'm afraid of is that we employ a technical writer and we simply dump on his/her shoulders the whole load of problems. We should be able to build a process to allow correct exchange of information. Technical writers should be able to effectively be in touch with the subject-matter experts to retrieve all is needed to write the doc.

At a certain point I've thought about writing docs in a sort of wiki. The tool is plain stupid and information would be readily available everywhere. Then we could think about 'exporting' the content in the necessary format with the necessary layout.

Again I'm with you here. The will shall come from above and cannot be expected to be something that everyone engage with spontaneously. It must be demanded, with the apppropriate level of responsibility.

Uhm, I didn't get if that was ironic or not.

That's another issue in its whole. Unfortunately you cannot ask or enforce engagement. People are people and as an employer you have only two possibilities, keep them or change them. Is not an easy choice.

There are ways to get people 'on-board' and make them more involved, but it does not work always and indiscriminately, so you need to deal with your pool gene anyhow. Is always easy to win when you have the best staff within your team, not so much when they are on the low side of the bell curve. But you certainly need to be afloat with your mean value team, because that's where most of your population will be.

In the signature circuit there's the author (the one providing the information), the technical responsible (who's guaranteering technical correctness of the information provided), a quality officer (which guarantees the needed level of quality is met) and the final project manager who's signature is more of a legal (contractual) binding.

Good is an attribute that cannot be measured/verified. START is just as good as BEGIN, GO, RUN, etc. Yet what a quality officer may request is naming harmonization throughout the interfaces (hardware/fpga, hardware/software).

I'm not sure about that. A company aiming to produce maintainable code cannot avoid to have coding standards. Tools do exists to verify your code is compliant to your rules. We have already rules like MISRA to be compliant with. If the software team fills deprived of its 'artistic license' they better find a more appropriate job.

Engineers have strong egos and this is a great 'feature', but if this implies 'doing it *my way*' than I'd say not in here. We have procedures and we follow them, if somebody believe the procedure is wrong than it must fight for making it right. In Italy we use to blame the government [1] when we are unhappy, but I've always found that kind of complaint a sterile one.

the real pain is when that kind of person is amongst the leaders, your chance to win a debate is way more complex. Those who follow cannot be obstructionists by definition. The third category are random noise that should not affect the S/N ratio, otherwise you better think of a turn-over.

We've flown to the Moon propping open circuit breakers and overriding procedures and pioneering achievements often require to go beyond regulations, rules, calculations and follow instincts and intuition. But try to do that again on a regular basis and you are in for a disaster.

The Apollo program was heavily unreliable, yet people engaged and motivated made it happen. But moving 500 people every 2 minutes over

Shit happens, sure, but the process is a self learning one and we learn from mistakes through a complex and costly endeavor. Is not everywhere like this, cars accidents happen regularly but drivers, instructors, trainings, seldom are updated.

Al

Dealing with just this aspect of your response to Don's comments, it seems like you did not consider the need for Requirements Traceability from the project outset (the best place to consider how you handle such things within your process).

Fortunatly, there are some good Traceability tools out there. One company that produces such tools is LDRA

Alternatively, you could look at changing your design approach to something that is more "Component Oriented" in which the requirements traceability issues are more limited in scope and easily contained in component by component development.