I've seen a lot of posts over the last year or so that indicate a lack of understanding of the implications of the Nyquist theory, and just where the Nyquist rate fits into the design of sampled systems.
So I decided to write a short little article to make it all clear.
It's a little longer than 'short', and it took me way longer than I thought it would, but at least it's done and hopefully it's clear.
You can see it at
formatting link
If you're new to this stuff, I hope it helps. If you're an expert and you have the time, please feel free to read it and send me comments or post them here.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Posting from Google? See http://cfaj.freeshell.org/google/
"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
Very nice. Should be distributed to universities so the kids learn some real stuff.
Re "3.2 Nyquist and Signal Content": If Nyquist would have listened to some of today's content (digital radio etc.) he'd have said that it ain't worth sampling it :-)
I like the wording "line in the sand". Isn't that how Archimedes started studying his circles? They didn't need any white board with the smelly marker pens.
Pretty good. My only quibble is the various statements about what Nyquist said and didn't say.
formatting link
"Exactly how, when, or why Nyquist had his name attached to the sampling theorem remains obscure. The first known use of the term Nyquist sampling theorem is in a 1965 book[4]. It had been called the Shannon Sampling Theorem as early as 1954[5], but also just the sampling theorem by several other books in the early 1950s."
It was actually Shannon (among others) that did the sampling theorem; Nyquist made an observation. Your bibliography doesn't cite either of them. It's probably correct to use "Nyquist rate" but not "Nyquist theorem."
Nyquist published his paper about the minimum required sample rate in
1928. Shannon was a kid of 12 years back then. The paper wasn't about ADCs or sampling in today's sense but about how many pulses per second could be passed through a telegraph channel of a given bandwidth.
By ignoring anything that goes on between samples the sampling process throws away information about the original signal. This information loss must be taken into account during system design."
This seems like something of an oversimplification. If the orginal signal is naturally or otherwise bandwidth-limited to well below
2x the sample rate, there may not be any useful information available to throw away and the loss may not have to be taken into account during system design.
Thanks John, that's a good point. I'll probably leave the titles intact because the paper is a reaction to all the posts that have the words "Nyquist says" followed by something _wrong_ -- but I should put in a disclaimer, or something.
I'm going to go off and do some web searching; in the mean time do you have any URLs that point to the seminal papers?
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Posting from Google? See http://cfaj.freeshell.org/google/
"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
If you have the continuous-time signal, then you _know_ there's nothing of note above Fs/2. If you don't have the continuous-time signal, then you _can't_ know there's nothing of note above Fs/2, unless the sampled signal train comes with a Certificate of Limited Bandwidth.
Later on in the article I talk about signals that are, indeed, sufficiently bandlimited by their nature, and the fact that you probably don't want to do any explicit anti-alias filtering in such a case.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Posting from Google? See http://cfaj.freeshell.org/google/
"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
The wiki article has links to Nyquist's 1928 paper and to Shannon's
1949 paper. Few EEs that I've met have ever read either of them, and have absorbed "the Nyquist theorem" mostly by hearsay, which I guess is your point.
Don't you go knocking whiteboards. A big board with a nice fresh set of markers will multiply my IQ by about 1.3 or so. Sand doesn't work nearly as well.
I've read Shannon's paper, thorougly. At least, these: The Bell System Technical Journal, Vol. 27, pp. 379?423, 623?656, July, October, 1948. Note that this is NOT 1949. But it was, in fact, what made me understand Boltzmann much better than before and allowed me to better access the underlying meaning of macro concepts such as temperature and entropy (which have no micro-meaning.)
Hamming, Shannon, and Golay all worked together in the same place, if I recall, around that time. Marcel J. E. Golay, 1949, and a little (short) paper called, "Notes on Digital Coding" which came out in
1949. (I think he was pushed into it by Hamming and Shannon.)
In the first sentence of your papere, you say that if the signal is band limited to fo or less, then a sample frequency of 2fo or more is adequate to contain completely all the information necessary to recreate the signal. My understanding from water cooler conservation is that the signal bandwidth must be strictly less than fo for a sample frequency of 2fo and that the signal must be infinite in extent to allow perfect reconstruction.
IMHO. I could be wr>I've seen a lot of posts over the last year or so that indicate a lack
Golay didn't actually with Shannon at Bell labs. His first paper, the one I mentioned called "Notes on Digital Coding" was actually published in the Correspondence section of Proc. I.R.E., 37, 657 (1949) was written while he was at the Signal Corps Engineering Laboratories in Fort Monmoth, N.J.
Now, the additional info.
Golay's 1949 paper is supplemented by two more papers he wrote: "Binary Coding" I.R.E. Trans. Inform. Theory, PGIT-4, 23-28 (1954) -- written also from Fort Monmoth, NJ; and "Notes on the Penny-Weighing Problem, Lossless Symbol Coding with Nonprimes, etc.," I.R.E Trans. Inform. Theory, IT-4, 103-109 (1958) -- written from Philco Corporation when in Philadelphia, Pa.
Twelve of Shannon's papers (1948 through 1967) are conveniently collected in the anthology, "Key Papers in the Development of Information Theory," edited by David Slepian (IEEE Press, 1974.)
Shannon referenced Golay's 1949 paper in the book "The Mathematical Theory of Communication" (written with Warren Weaver, Univ. Illinois Press, 1949.) This book contains a slightly rewritten version of Shannon's first 1948 papers together with a popular-level paper by Weaver.
Shannon describes the Hamming-7 code in his 1948 papers in section 17, attributed to Hamming there, but since there is no reference to a specific paper by Hamming I suspect this reference must have been via personal communication with Hamming. (Golay also refers to the Hamming-7 code in Shannon's first paper.)
The first paper by Hamming is "Error Detecting and Error Correcting Codes" Bell System Tech. J., 29, 147-160 (1950.) Note this is actually _after_ Shannon's reference to Hamming's code. The anthology, "Algebraic Coding Theory: History and Development," edited by Ian F. Blake (Dowden, Hutchinson & Ross, 1973) includes this paper.
Blake says in his introduction to the first 9 papers in his anthology:
"The first nontrivial example of an error-correcting code appears, appropriately enough, in the classical paper of Shannon in 1948. This code would today be called the (7,4) Hamming code, containing 16 = 2^4 codewords of length 7, and its construction was credited to Hamming by Shannon. Golay gives a construction that generalizes this code over GF (p), p a prime number, of length (p^n -1)/(p - 1) for some positive integer n. Hamming also obtained the same generalization of his example of codes of length (2^n - 1) over GF(2) and investigates their structure and decoding in some depth. The codes of both Golay and Hamming are now designated as Hamming codes. The interest of Golay was in perfect codes, which have also been called lossless, or close-packed, codes. Since he mentions the binary repetition codes and gives explicit constructions for his remarkable (23,11) binary and (11,6) ternary codes, it is not stretching a point to say that in the first paper written specifically on error-correcting codes, a paper that occupied, in its entirety, only half a journal page, Golay found essentially all the linear perfect codes which are known today. ... The multiple error-correcting perfect codes of Golay, now called Golay codes, have inspired enough papers to fill a separate volume."
By the way, the Hamming-7 code can be used to generate the E7 root lattice, which corresponds to the E7 Lie algebra. And the Hamming-8 code similarly generates the E8 root lattice, corresponding to the E8 Lie algebra. Heterotic superstring theory has an E8 x E8 symmetry which is needed for anomaly cancellation. It is nifty that the very first error-correcting codes of Golay and Hamming play such a profound role in modern superstring theory.
An interesting supplemental work is from J. H. Conway and N. J. A. Sloane, "Sphere Packings, Lattices and Groups" from Springer-Verlag,
1988. But probably the best ever book on algebraic coding theory is: "The Theory of Error-correcting Codes" by F. J. MacWilliams and N. J. A. Sloane, North-Holland Publishing Co., 1977.
No expert, but figure 14 could possibly be improved by explicitly showing the rising edge of the pulse, maybe start the time axis at -0.5 or start the pulse at +0.5
I really like your style of writing. Accessible, and yet with no hint at the "for dummies" craze.
I hadn't really thought of the Signal-to-Aliased-Energy ratio as a metric for signal quality before. It will be a lot easier to discuss the quality of sampled systems with that as a tool. Very good presentation of the both the concept and the effect. I whish it was me who thought of that sequence of figures 6-9... oh well.
A couple of very subjective comments:
- There is something about the typography of the page that annoys me. I can't really see if there is an open line between paragraphs inside a
section; if there is, you may want to make it clearer
- The footnote indicators are WAY too small to be useful, or even seen.
- I can't see why you need the maths paragraph ("To understand aliasing...", eqs. (2),(3)) at all in section 2.1. I think that what you say there is covered by the plain-text parts of the section. Lay-men might find that maths paragraph scary, throwing them off your article, while technicians know it already.
- Similar comments apply for eq. (4). I can't see that it is strictly necessary, the explanation is in the text anyway.
- Eqs. (5)-(7) seem to be necessary, but are the only big equations in the article. Hmm... that makes me wonder...
As far as I can tell, you are 99.9% at the point where this is an article a non-engineer layman can read and understand. If you find a way to get rid of equations 1-7, maybe even eq. 8, you ought to be there.
Very good. I would change this phrase: "This will allow you to use a more open anti-aliasing filter." By "more open", I gather you mean a larger transition ratio, which lowers the Q of the resonators.
Note that the Bessel filter will ring at higher orders. I don't have my copy of Zverev handy, but I think the Bessel rings at 4th order and higher. The Gaussian filter doesn't ring at any order. The key is to look at the impulse response of the filter. If it ever goes negative, then filter will ring.
You might want to go into the inverse sinc response requirements in the recontruction filter.
Since understanding the formula _notation_ is not essential for understanding most of the rest of the text, I would suggest moving the formulas into separate boxes and moving these boxes out of the direct text flow e.g. into a box in the right margin of the paper.
Definitively !
The problem with quite a few text dealing with sampling is that they are written by mathematicians for the mathematicians. However, these days, most people using various DSP algorithms are programmers, not mathematicians, thus the terse mathematical notation can be hard to understand to them, especially without too much numerical analysis background.
Instead of using the terse mathematical notation, it might be more productive for most readers to publish e.g. a Fortran/Algol/C algorithm.
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.