Non-text format for source code

For great many reasons, it would be convenient to have source code in some rich format rather then plain ASC text. I could think of embedding illustrations, links and data into file, custom coloring, individual styles, comments, automatic macro expansion, presets and other features offered by modern word processors. The idea seems obvious; so there should be a tool for that already. That could be a pre-processor to import/export source text to/from some popular format such as *.doc or *.xml. Do you know of such tool ? Any comments?

Vladimir Vassilevsky DSP and Mixed Signal Consultant

formatting link

Reply to
Vladimir Vassilevsky
Loading thread data ...

Presumably you are familiar with doxygen et al?

--

John Devereux
Reply to
John Devereux

We use Doxygen for some projects - works well. Its just for documentation though, no source pre-processing, no viewing of your source as through a nice word processor. Hope that helps, Best Regards, Dave

Reply to
Dave Nadler

OK, I have not used it seriously. I am surprised there is no way to include the source code in the output?

There is the "literate programming" idea:

--

John Devereux
Reply to
John Devereux

There already are some examples of this.

Up to now non ascii representations for text based languages have a simple way that ascii is fed to most compilers. Comments and design notes are one thing but ways of having several related pieces of information become possible like unit tests as part of the source and uls to test files on the internet

This changes a lot when non text is part of meaningful syntax for a language. There are a couple examples of this ISO/IEC 61131 is a standard describing programming for ladder logic and block programming . ISO/IEC 61499 is an extension to block programming in 61131.

I wrote a parser for 61131 and it actually isn't that bad. Essentially the first step is extracting the date relationships between blocks into a data base and then compiling the data base

w..

Reply to
Walter Banks

e

es,

y

ext

HTML sources, eh? Why not, come to think of it...

Dimiter

Reply to
dp

Doxygen, not suprisingly, has options for this.

I shall forbear to mention POD in embedded company ...

Nick

--
"The Internet, a sort of ersatz counterfeit of real life" 
	-- Janet Street-Porter, BBC2, 19th March 1996
Reply to
Nick Leverton

There may well be; certainly declarations can be output. But this isn't like working with source code in a WSIWYG editor including diagrams, highlighting, font control... Which is what I understood Vladimir was asking about ? It would be great to have these features as long as it worked in conjunction with other IDE-like behavior (ie macro expansion, accumulation of diagnostic info, etc).

An Eclipse plug-in anyone ?

Right now I'd just settle for a stable IDE that didn't get confused and require me to delete mysterious intermediates that it doesn't "clean"...

Interesting ! Best Regards, Dave

Reply to
Dave Nadler

Doxygen can quite happily include the entire source code in the output if you want - both in a function-for-function arrangement along with the documentation, and in complete files. It all has syntax highlighting, clickable cross-references, searchable indexes, etc.

But it is an output generated from the source code - it is not editable.

I can't think /why/ one would want to edit these "rich format" parts within the source code, however. If you were able to change the font colour, weight or style within the source code, then you would lose all advantages of syntax highlighting.

If you've got documentation, diagrams, pictures, etc., that are part of your "source code", then keep them together with the source code. But there is no need to force them to be within the same files, or use the same editor. I've done projects where pictures were part of the "source" - they are stored as bmp files, and the Makefile-controlled build runs a converter to generate a "const uint8_t picture[] = ..." array from the file whenever it changes. There is /nothing/ to gain from trying to include that picture within another C source file. I also have diagrams in projects - if they are simple, they might be included in the source code comments, but mostly they are in a separate .dot file so that they don't distract from the source code. Doxygen puts it all together in the end.

It would be nice if Eclipse's build system worked better. Personally, I prefer my own Makefiles - they are much better than Eclipse at rebuilding /exactly/ what is needed, no more and no less. Eclipse is particularly bad at re-generating dependency files unnecessarily - for large projects, this can be time consuming.

Reply to
David Brown

For me, the issue is that I'd like to have a document that I directly edit and work with (not unlike using DreamWeaver, for example, in editing HTML) that supports the inclusion of good diagrams and perhaps, if I'm crazy enough, a short video tutorial if I'm in the mood to produce it.

I suppose a simple example (but not a definitive one) would be the insertion sort code. It's very simple to write, but a newcomer might be able to "see" it still better if I included a diagram showing the left part as "sorted", a right part as "unsorted", and the latest element picked off of the left side of the right part which is being inserted into the left part. Yes, this is a silly example. Anyone worth their salt already knows several sorts and most certainly this one. But that's beside the point. The point is that there exist cases where I _DO_ want to include some special help because the way the function operates is enough interesting that it deserves some extra effort in documenting it. Not just for others, but for me too when and if I return to it some other day.

I take some of the points made by the literate programming advocates, too. (There are some terrible examples held out as good examples, yes. But there are good examples held out as good examples, which really are good examples, too.) There are times when the best way to talk about the code design doesn't exactly follow the best way to lay out the code for a compiler. It would also be nice to have the ability to arrange what I say about things, together with the code that does these things, to be different than the way a compiler must see them and to have that handled so that all I (or anyone else) sees is the source as I intended the source to appear for purposes of best communication, not for compiler consumption. That would be ANOTHER desire of mine, as well. At least, sometimes and in some places.

It seems to me that in this day of so much work on standards, which include not only HTML5 but also XML and many others, that we could have compilers which aren't stuck parsing simple ASCII text. It was fantastic (and the only way to go) when all I had was an ASR-35 clacking away and a video display (if you dared to even consider the idea) could set you back US$40,000 or more. (I wonder how many remember the old Plato terminals?) But today? With workstations absolutely everywhere? And disk space almost as cheap as dirt (though not nearly as likely to serve its purpose quite as long?) Not just one video display; nay, not even two; but often three per workstation??

Jon

Reply to
Jon Kirwan

The main problem with the output from Doxygen is that it's wrong.

The comments are wrong, therefore any output based on the comments is wrong.

OK, I'm being Hyperbolic, but I'd have to say that at least 80% of the time, the Doxygen-generated docs for embedded stuff that I've come across was worse than useless.

There are just too many cases where the function description, parameter list, and return values are just plain wrong. Somebody always ends up refactoring the code, changing the calling conventions, and so on without updating the comments.

Right now I'm using an industrial protocol stack library from a third party vendor. It comes with tens of meagabytes of Doxygen generated stuff. The first time I looked up a function in the Doxygen docs, the parameter list didn't even match the code. The next time, the parameter _names_ were correct, but the description of one of them was completely wrong. I deleted the docs at that point.

--
Grant Edwards               grant.b.edwards        Yow! I just heard the 
                                  at               SEVENTIES were over!!  And 
 Click to see the full signature
Reply to
Grant Edwards

This has all got very little to do with Doxygen. It's much more a process problem. What has happened is that generating the documentation is not part of the release or build process. Once you realise that building the documentation should happen as part of a issue build process, these faults go away.

We (MPE) have been using our own proprietary literate programmming system for the last 15 years or so. We are regularly told that our documentation is good. However, I have had to wave a big stick many times to ensure that house rules are obeyed.

The big issue with all forms of literate programming is to keep function documentation alongside the function itself in the source code. One of the downsides of Doxygen is the visual separation of function declarations and parameter documentation.

Stephen

--
Stephen Pelc, stephenXXX@mpeforth.com 
MicroProcessor Engineering Ltd - More Real, Less Time 
 Click to see the full signature
Reply to
Stephen Pelc

Of course. It's the age-old problem of the comments being wrong.

No, that wasn't the problem in this case. The problem was that the information in the comments in the source code which was being extracted were wrong. The documentation matched the comments in the code. The comments didn't match the code.

Unfortunately, they don't. Anything that is being produced from comments is wrong often enought that it can't be trusted and you always have to look at the actual source code.

The function name/type and parameter name/type should always be right, since that can be parsed from the actual source code. But, some documentation systems I've seen don't do that. They rely on comments for that as well.

In my experience, that doesn't seem to help much. Comments that are located _on_the_same_line_ as C code are wrong often enough that I generally ignore them.

The downside of any sort of documentation system is that it seems be viewed by many as an alternative to writing code that's easy to read and understand.

--
Grant Edwards               grant.b.edwards        Yow! ... this must be what 
                                  at               it's like to be a COLLEGE 
 Click to see the full signature
Reply to
Grant Edwards

People seem to think it is. They turn out that documentation and they offer it to the public. The ones I run into tend to be huge pages containing nothing but nouns. Seriously, the one I read in depth had a page footer, and it contained the word 'for', which is a preposition, and a version number which could be called an adjective. Apart from those -- nouns. It would have been nice to find some verbs to tell me about what would actually get done.

Mel.

Reply to
Mel Wilson

I've certainly seen that too. Documentation has to be maintained (updated, reviewed) as a serious part of development rather than an afterthought. If it isn't part of the regular build process the generated stuff will degrade to useless. If its done well and

*regularly used by developers who need it* this can be really helpful ! If its done to check a box and/or nobody uses it, useless...
Reply to
Dave Nadler
[...]

Well, I'm getting the same feeling occasionally; e. g., when posting to a Usenet newsgroup [aside: cross-posting to news:news.misc]. Still, even as there are standard ways of including all the sorts of nice stuff (such as SVG diagrams, or digital signatures) into Usenet messages, there're those who'd oppose even the inclusion of non-ASCII characters (or, if a non-English newsgroup is considered, the use of an encoding other than a prescribed one) or OpenPGP signatures along with the message, even if done according to the standards.

Somehow, I come to believe that the use of a message format more elaborate than "pure ASCII" (say: HTML5/XML) for email-like or news-like communication may have its benefits. (Perhaps as well as the use of such a format for the source code.) Unfortunately, there's a catch...

[...]

... The problem is that these new facilities, once they appear, are most surely going to be misused.

For an example, take a look at the contemporary Web [aside: cross-posting to news:comp.infosystems.www.misc, too]. Every other Web page is designed as if it's some kind of picture, not a text (which it is, first and foremost.) Digging a bit into CSS, what you're likely to find? Here comes an example:

body { width: 1024px; /* 'cause everybody's screen is at least that wide */ font-size: 14px; /* it's enough for everyone, no? */ } #leftbar { width: 384px; /* that's about 40 characters; good */ } #forum { width: 768px; /* that's the rest of the screen */ }

Naturally, as soon as the reader of the Web page opts for another font size (e. g., my preference would be no less than 20px), the design is screwed up instantly. (Not that this issue is not relevant to the source code as it is right now: we have those who adhere to the 80 characters per line limit, and those -- who use newer code editors and IDE's, and wider screens or smaller fonts, -- who don't; we have those who indent with 2 spaces per nesting level, and those who use 8; etc. And while a switch to a non-ASCII code representation may obviate some of these issues, it will certainly bring a bunch more.)

Given all the sorts of such poor designs I've seen on the Web, I'd hesitate to use a system which /requires/ the code to be worked on in a "WYSIWYG" fashion. Why, for the Web, my browser of choice is Lynx, specifically for its ability to ignore CSS, images, and all of such "creative freedom" that's so easy to misuse. I still believe that there /are/ valid uses of such technologies, both existing and to be developed, but for me, there remains a big fat warning regarding their possible (as in: most certain) misuse.

Hopefully, the technologies of tomorrow will come with some kind of "safety mechanisms" against such...

PS. Also to note is that I don't know of any version control system with good support for non-line-oriented data. Somehow, I feel that even computing the "difference" (as in: diff(1)) between two trees (as in: XML) is much more an issue than computing one between two lists (as in: ASCII text.)

--
FSF associate member #7257
Reply to
Ivan Shmakov

True, but extrapolate that too far and you get "comments are bad". Good code makes it really clear what the code DOES. Good documentation makes it really clear WHY; what's the purpose and organization (as contrasted to function) ? The worst comment I've ever seen (more than once !): // Add 1 to i

Good documentation allows quicker understanding of code structure and a good Doxygen tree is *really* helpful and can save hours of grep'ing and head-scratching.

But it takes discipline to do it !

Reply to
Dave Nadler

me kind

I think we do not need "tomorrow"s technologies to make most if not all of Vladimir's suggestion.

Me, I would just include html links in the source such that they will be treated as comments by the compiler/assembler/whatever and be done with it. Without a lot of effort the source will be both compilable and viewable with a browser, pictures and all. Now, I just put it all in a directory or make the suitable directory tree so it will still be quite easy for anyone to locate the relevant information but I can see how Vladimir's idea could save time for someone seeing something I have done for the first time. Even to me if I look at it some 10 years later (though I must say I don't have much if any of a problem reading 20 and even almost 30 years old sources of mine; writing the correct comments is the most important part when it comes to readability, after all).

Dimiter

------------------------------------------------------ Dimiter Popoff Transgalactic Instruments

formatting link

------------------------------------------------------

formatting link

Reply to
dp

Doxygen definitely can include (hyperlinked) source code; I use that all the time. It can also include diagrams, formulas..etc (never used it, but there are tags for that).

Don't forget the diff and a merge tool. It would pretty useless without those.

Reply to
Dombo

You can hardly blame Doxygen for that. Though I agree that programmer supplied comments often are misleading either due to refactoring or just mindless copy-and-pasting (sometimes it would have been better if the clipboard functionality were never invented).

Though I use Doxygen quite often to get familiar with an code base, I rarely read the programmer supplied comments. I mostly use Doxygen to figure out the structure of the code.

I have seen the same happen with manually maintained Word documents that describe the code in too much detail. Unless one is willing to invest significant effort into writing _and_ maintaining proper documentation with the needs of the user in mind (as opposed to just following the requirement "there shall be documentation"), the result is more often than not worse than useless.

I recently participated in a review meeting about a coding standard to be used in that organization. They wanted a rule that required every function to be fully documented. Having been-there-and-done-that and having seen tons of pointless and misleading comments, my recommendation was to rewrite that rule to: don't write comments unless they have added value.

Reply to
Dombo

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.