Parser, again!

Oh come > ...

Hey, that is the wisest observation I have read in a long time! :D :D . Very true - yet people constantly try to create more and more complete phrase books instead of simply using the language into which the phrase book has been written (admittedly most people don't know what to do with the language and have it simpler with a few readily available sentences to repeat though...).

Dimiter

------------------------------------------------------ Dimiter Popoff, TGI

formatting link

------------------------------------------------------

formatting link

Reply to
dp
Loading thread data ...

Very rational, which is quite irrelevant in the face of the fashionable religion du jour.

That's the rationale! If I've written it it must be easier/simpler/better than something I can't be bothered to understand.

Me a cynic? Shurely shome mishtake.

NIH. A common complaint.

You clearly have applied common sense, and haven't read the religious texts nor derived religious texts written by wannabe acolytes or "XP trainers".

Yup.

Yup. Earlier in my case!

Ah, but the religion dictates that the comments /always/ get out of sync with the code. The code is The One Executable Truth (fair enough), therefore comments (which will become wrong) are a heresy that should be eliminated in case they mislead.

Remarkable, but I've heard it. The next sound was that of me going ballistic.

Reply to
Tom Gardner

Very rational, although I wouldn't specify a proportion since the religious acolytes will want to prove their devotion to the cause (and thereby be promoted) and will turn it into a commandment.

There's a very interesting debate there. Is it better to have - a domain specific language, or - a domain specific library

I prefer the latter, because of tool support, ease of hiring more staff and usually because of higher quality.

Reply to
Tom Gardner

I agree, one should be careful not to give ideas too easy to twist :D .

I think - if I understand you correctly - the library based on a language is indeed a lot more efficient. Humans can learn languages, translation from one into another language is generally possible (though it might take some extension to the destination language). I can't think of a decent novel written by someone using a language at phrasebook level. Obviously I am generalizing around your statement, probably because I have some strong feelings about languages (not suprisingly so since I use mostly VPA, "Virtual Processor Assembler" which is my creation on top of 68k assembler, having evolved for over a decade now).

Dimiter

------------------------------------------------------ Dimiter Popoff, TGI

formatting link

------------------------------------------------------

formatting link

Reply to
dp

Finding the boundaries is no harder than with the ASCII version. And with UDP you have to build in a lot more error recovery.

Reply to
Robert Wessel

There are probably SCPI libraries you can download, rather than writing something yourself. This showed up as first hit in a web search for "SCPI library":

formatting link

What about SCPI?

You don't need anything as complicated as the types of parsers you find in the dragon book. If you really find you want a programmable extension language and if your microprocessor is relatively powerful, try Lua or maybe Guile. If it's a mid-sized microprocessor, Forth might be ok, though it's not for everyone. Any of these will bring you into a level of software hacking that you might not want to deal with given that you're (I'm guessing) mostly a hardware person.

Reply to
Paul Rubin

The reason is that PCs are typically used as Modbus masters, in which the timing is not so critical, as larger than minimum delays are used. The same applies for a point-to-point slave.

In multidrop slaves, the only critical event is when an other slave has sent a responded and the master send the next request to you. At this event, you must know where the request frame starts after the 3.5 character time gap.

Both TCP and UDP should have been specified. In fact many vendors implement Modbus/UDP. Modbus/TCP is nice in WANs, we have even run Modbus/TCP connection using GPRS across the continent.

Modbus/UDP is the natural choice for a small LAN.

There are hundreds of implement ions that simply prefixes a 1 or 2 byte count field in front of an existing message. These seem to work quite well, regardless this trivial structure. As long as the receiver flushed any characters received during a previous connect, a new connect should always start from the beginning of the frame and the frame synchronization should always be maintained.

Modbus/TCP has a fixed 6 byte header, containing the remaining byte count (2 bytes). Unfortunately, the protocol-ID is 0000h, a better choice might be some less likely value as 1234h.

For this reason, most simple TCP based protocols use the byte count to split the stream into frames.

The Modbus/Ascii is the simplest, just skip everything, until you get the colon.

In a TCP base protocol, if you loose synch, unless you have a robust preamble, the only practical way is to disconnect and then start over with a new connect.

The error recovery is practically the same if serial, raw Ethernet (MAC) or UDP frames are used. The master shoots out the request and if no (valid) response is received within a specified time, just repeat the request one or more times.

In a TCP/IP based protocol, you must also monitor the state of the connection (how? Keep-alive-time is usually faaar too long) and if the link failed you must try to re-establish a connection.

If the response does not arrive in time (communication failure/slave busy), what to do ? Resend the request on the same TCP connection, or should the old connection be (orderly?) shot down and a new connection created and the request sent over this connection.

Since the TCP transmitting side will try to deliver the message for a long time by buffering the requests, even if they have been obsoleted by the upper level protocol.

Think about the situation that a heavy machine fails and someone goes into the controlled heavy machine to study the problem. An other person without visual contact with the heavy equipment, notes "Hey, this Ethernet cable has been disconnected from the switch, I put it back", all the commands in the master TCP Tx queue are sent at once and the slave executes all these queued commands, starting the heavy machine, injuring the person within it.

Thus, you really have to be careful with the TCP Tx queue to inhibit backlog as well as detecting obsolete commands on the slave side.

With serial or UDP, the sent and lost requests are not being queued.

Reply to
upsidedown

Dear All,

Thank you very much for your valuable input. I lost myself a little on the XP/Agile debate of Tom, Les and Dimiter [:-)] but as for the rest this was a first class lesson!

To make things a little more clear, my gizmo behaves more like an programma ble instrument, an oscilloscope or a multimeter, than a PLC. It measures ph , conductivity and temp and was to activate relays (pumps). For those who l ike chemistry, this is a titration experiment and my device is an automatic titrator. The Pc and the device will talk to each other by RS232 on a poin t-to-point connection, the PC being the master, asking all the questions an d setting values. I need a readable language to make things more easy to de bug and test (with a terminal) and to give to the users the possibility of making their own programs.

Actually I have done this before, the syntax and the decoding routines, 3 o r 4 years ago, using brute force. I was looking for a more formal way to ap proach the problem.

What I could understand from your contributions is:

- For my application, using Forth or Lisp is a little like using a sledgeha mmer to crack a nut. I don't think will need that. A simple ascii protocol will do the job. Thanks for the suggestion anyway.

- Modbus is not very human-readable but is frame format was interesting fea tures that I can use. With this and the input of Dimiter, Mel, Robert, upsi d.. and Y, I'll be able to come up with a syntax. I finally understood that the commas and spaces and stuff are used for the benefit of humans, not ma chines. The only important things for the machine is the existence of SOM o r EOM or both. I think I'll go for the :xxxx>CRLF> format.

- The parsing routine is not difficult but there is no generally accepted a lgorithm (Paul). Thanks for the ideas Les and Y.

- Paul, last time I had this problem was with a research nmr machine I buil d. At that time I spend a lot of time searching the wed for free SCPI libra ries without success. This was 4 years ago. This time a just din't search! Thanks for the tip.

Regards

Mariano

Reply to
jmariano

One thing that should have been asked: what is the target processor? Why are you even writing the code in C? Yes, using something like Lisp (or these days Python) might be "overkill", but just in the sense that it's overkill to drive a 2000 pound car to pick up a quart of milk from a store 2 km away, especially if the "car" burns no fuel. If it's a small MCU then you kind of have to use C, but if it's something like a Raspberry Pi then you can use almost anything. Simplest practice with such a system might be to use XML or Json, depending. The parsing and encoding libraries are already there.

Reply to
Paul Rubin

A few years ago I designed for a guy a PH & conductivity meter, ensuring the conductivity meter did not interfere with the PH electrode was why this came my way. But I only did the analog front end which delivered the two analog values, that's where help was needed.

Since your application will likely involve long times from host command to device knowing it has succeeded (pumps staying active etc.) you might want to do something similar to the command-ack-outcome_notification sequence in my protocol which I posted earlier. [ the text is at

formatting link
] Makes life a lot easier; host issues a command, gets immediately an ACK reply which also carries information about how long (max) will it take for the command to finish; upon finishing, the device notifies the host how it worked out. This can take seconds if not minutes so in the meantime the host can keep on issuing other commands to the device, like read status, various measured values, make some light blink etc.

Dimiter

------------------------------------------------------ Dimiter Popoff, TGI

formatting link

------------------------------------------------------

formatting link

Reply to
dp

If I were you, I would stop searching at SCPI and continue with that choice. A simple implementation of SCPI is dirt simple to implement in a parser (that's the right word). You don't need to support the instrument models, multiple command, or hierarchy functionality. It's got human readable keyword/data statements that makes debugging easy. Search no more.

I have one complaint about SCPI. There is no defined inline success/failure mechanism for commands, probably because if its long history of being married to GPIB, which has error handling defined via some other means. In all of my SCPI implementations, I defined a success/failure status response that has served me (and my customers) very well. If the command succeeded, "OK\r\n" was returned. If it failed, "ERR\r\n", was returned ( is an error code). That response mode was switchable for compatibility with official SCPI.

You'll find your SCPI parser code will be reusable for future projects so design it with that in mind.

JJS

Reply to
John Speth

I'm using a Arduino Due board and the main reason for using C is that... I now C and Lisp (or Forth, or Python, or ...) I don't. I only know C, Fortra n, and a couple more and I'm to old to switch dialects unless there is a ve ry good reason for it. In this case, I think that it will take me more time to learn a new dialect than to implement the damn thing. But I might be w rong.....

Reply to
jmariano

).

Sorry, I was not clear: ph and temperature (for ph compensation) or conduct ivity, not both at the same time. I also have to design the ph and conducti vity front ends, but your case was much more tricky.

I have read your specification carefully and I will be using some of your i deas (hope you don't mind) but I'm not so sure of the ACK reply. I have to think a little about it. I think that with SCPI instruments, the ACK is a f lag on a status register that the PC reads when we feels necessary. How thi s two approaches compare?

Reply to
jmariano

My idea too. The OK/ERR mechanism seems a good idea. Thanks

Reply to
jmariano

Of course I don't mind, I would not have made it public if I did :-).

The main difference which the ack_with_time_to_outcome_report then the (perhaps much) delayed outcome report makes is that your device is less dependent on the PC latency - which is impredictable, can be from milliseconds to ages. You may or may not need that - you know what you want to do, if you do not need that sort of thing of course it makes no sense to add the complexity it will take. I defined - and implemented it - when I did it without giving it that much thought, cost me a few hours to draft it. By then I had enough experience to know how to do it - perhaps with some bearable overkill - so I would not run into problems later(someone else was supposed to do the PC side of things so I wanted to have control over what I was responsible for :-) ).

Dimiter

------------------------------------------------------ Dimiter Popoff, TGI

formatting link

------------------------------------------------------

formatting link

Reply to
dp

You don't really a parser, assuming the syntax is always going to be something like:

command [parameter] ...

Presumably the parameter will either numeric or alphanumeric; it makes it easier if the command dictates how many parameters there will be, and what type each will be.

And, which is sending the commands, the PC or microcontroller? Decoding will be harder work than sending the commands.

With certain languages, you could process this line-oriented kind of input just by using its Read statements, reading the command, and each parameter, as individual tokens, but is otherwise not difficult to do.

--
Bartc
Reply to
BartC

HINT:

Here is a snippet of standard C code which takes a null terminated string and returns a list of strings as delimited by the Sep characters;

whiteSpaceDelimitedList = s_parse( myBuf, " \t\n\r" );

For your problem, a white space delimited token stream is all you need to implement an extensible command line interface. Note that the Str input is destroyed in the process (nulls are inserted into the buffer -- a side effect of using strsep()), but you can work around that if necessary -- and usually it isnt.

The Lst_t* functions are left as an exercise for student.

Lst_t *s_parse( char *Str, char *Sep ){

char *p, *q ; Lst_t *x ;

if( !Str ) return( (Lst_t *) NULL ) ;

p = Str ; x = ls_crt( 1 ) ; while( (q = (char *) strsep( (char **) &p, Sep )) ){ if( !isNul( q ) && strlen( q ) ) ls_add( x, q ) ; } return x ; }

HTH, Cheers, Rob.

Reply to
Spam

*Blink* :)

No, it's more a matter of pointing at a thing that solves the problem that has reasonable cost.

I've done this both ways, with relative-COTS parsers or

100 liner roll-yer-own and it all comes out in the wash.

The trouble with COTS is there is lots of portability and generic-ity overhead. So for small enough problems, DIY frequently wins. Now your mandate is to break big problems into small ones with good interfaces between them.

COTS may also involve more risk and validation/verification overhead. So it's pure trade space.

Feh. :)

Can't help that.

I kinda tend to think that, too. Hell is other people's code :)

--
Les Cargill
Reply to
Les Cargill

I'll disagree, somewhat: "hell is having to *look at* other people's code".

E.g. I don't think a networking stack is hell; I just use it.

Good comments *reduce* the amount of code I have to look at because: - the problem I'm debugging doesn't match what their code is doing - they give me sufficient information for me to judge accurately the pros and cons of using their code, and/or to select between different approaches all without my having to make guesses based on speed reading their code and misunderstanding the subtleties and rationale behind it.

Good commenting isn't difficult, provided the code doesn't smell and provided time is allotted for comments.

Reply to
Tom Gardner

I'll go even further, good commenting saves programming time. Over the years it saves a lot of programming time and over many years it makes things possible which otherwise would simply be impractical if not impossible.

I can speak for myself only of course but while I am writing and commenting practically each line (here is a small utility as an example:

formatting link
) I often discover that I have to stop coding and (re)think something.

Saving time to type in a few words is a huge resource waste. If one does not have the comment words ready to type in he has even less code statements ready for typing.

Dimiter

------------------------------------------------------ Dimiter Popoff, TGI

formatting link

------------------------------------------------------

formatting link

Reply to
dp

ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.