Data aquistion problem

Hi Roel,

If this is a multi-channel AD conversion job I would talk to an engineer at Gage, Inc. Chances are, if they can't do it there may be nothing else around that does. Of course you could always build you own.

Regards, Joerg

The actual number behind "a few" could easily make the difference between easy and way out of league, there. Suffice it to say that if you can't afford enough Gigabytes of RAM to cache the entire "few" seconds' worth of data, you're looking at a massively parallel array of mass storage devices (e.g. ~20 modern harddrives in parallel might do it).

Sure. High-energy physics experiments handle rates even larger than that, even for essentially indefinite periods of time. But you may not want to see what such systems cost.

No way. No accessible port an a vanilla PC has anywhere near 500 MB/s of bandwith. 64bits at 66Mhz is the nominal width of a server-grade PCI-X/66 port, but you won't find that on anything that would still qualify as "PC-based", i.e. attachable to a reasonably typical PC.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Hello Hans-Bernhard,

It's not so bad. I recently bought a storage card for my camera and paid $40 for 256MB. Now they sell 512MB for the same price so memory costs should not be a huge issue here. Considering chips or RAM modules instead of these cards the difference between a few seconds or a few more should not be much above $100.

He'd have to buy a card with huge memory on board, store it there and then send it via the PC bus at regular bus speeds. I believe that is how Gage does it with their top of the line PC data acquisition cards. That would be the first company I'd contact in such a situation.

Regards, Joerg

[You snip my attribution, I snip yours...]

Flash RAMs are quite completely beside the point. We're talking about datarates well beyond anything those 40$ cards will handle.

You come back saying that again when you've found a motherboard that can handle more than 3 GB of memory that is remotely compatible with the price range of an ordinary PC.

I would slightly object to calling such a setup "PC-based" without further qualification. Such a beast (if you can find one that holds well above 1 Gigabyte) would essentially be a standalone solution that just interfaces a bit more closely with the PC than it strictly has to. It might prove quite tricky to supply 8 GB of sufficiently fast DRAM off a simple PC slot.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Hello Hans-Bernhard,

That's ok, you can. I do that to keep things brief so that readers who click from one post to the other don't have to scroll through everything a gazillion times. The only quotes I leave in are those where I believe they are needed to understand my post.

I only mentioned these to make the point that memory is cheap nowadays. Very cheap.

Ok, I am back. Not with a motherboard since most motherboards aren't made to handle this much data. But hard drives are. You need to store on the PC card first, then transfer. If you want to see what I mean look at this PCI card:

I am pretty sure you can gang two of them and, bingo, there is the bus width Roel needs and up to 4GB total memory on board. Of course the details such as ganging, synchronization etc. need to be thoroughly discussed with the card manufacturer. If you need to acquire analog data they also have cards with 2GB on board. Here is an example of a fairly regular analog card that can hold 2GB:

We have used such Gage products in ordinary PCs many times, usually in production or quality control applications.

You could call it standalone but the main thing is that it could work in Roel's application. Just as a side note, I did see a regular but "souped up" PC that had an incredible amount of memory installed, several gigabytes. It was used in a video processing setup. Nowadays even laptops such as a Dell Inspiron XPS can be had with 2GB of RAM if you want to.

Regards, Joerg

Hi Roel

What you probably need is dedicated hardware. A memory buffer with a word size of 64 bits on the input side and any word size on the output side to read the the buffer's contents at a leisure. This'll remove the need for extremely fast memory while retaining the required bandwidth.

Now for the memory size: let's assume that your experiment takes at most 5 seconds and that your data arrives with a frequency of 100 MHz (makes the calculation easy and yup, we'll allow for some slack here and furthermore, you may never know what the future may bring ;-) Thus, data arrives every 10 nsec. (My uneducated guess is that there numerous manufacturers who supply memory with access times much less than that.) One second of measurement yields 100 M of 64 bit words (= 800 Mbytes), therfore 5 seconds will give you 4 Gbytes of data.

Greetz

Waldemar

----- Original Message ----- From: "Roel" Newsgroups: comp.arch.embedded Sent: Friday, October 22, 2004 5:54 PM Subject: Data aquistion problem

"Roel" schreef in bericht news: snipped-for-privacy@posting.google.com...

Some DRAM structures may also allow writing multiple consecutive words without changing the high address bits, thus quite high sequential write rates can be achieved, but of course, you sometimes have to use the full RAS/CAS cycle, so some short time buffering is also needed. Also some time must also be allocated for refresh.

Since it is unlikely to find 4 GB memory modules, multiple modules will be needed anyway, so the memory word length can be easily extended to 256 to 1024 bits and write 4-16 samples at a time or in an overlapped way, thus allowing for quite slow memory cycle times.

Paul

"Store on the PC card" means you move the memory requirement to a different place: from the motherboard to dedicated hardware. Which brings us back to may gripe about what can still appropriately be called a "PC-based" solution, since during the actual activity, the host PC is really just sitting there doing nothing. I.e. the PC is not the base of such a solution, it just plays host to the party.

It takes quite a bit more than a bit of "soup-up" to make a regular PC applicable for this kind of work. For starters, you need completely different mainboards (server/"workstation" type), and most users will have to invest in a new operating system, too: Windows below the server-grade versions doesn't support more than about 4 GB of actual RAM.

2 GB isn't a major problem (well, if we set aside the power consumption draining the battery). 5 GB, however, definitely would be.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

The problem is not the RAM itself. A raw data rate of 533 MB/s is, indeed, manageable by the core components of any modern PC. The problem is that none of the available *ports* are anywhere as wide as that. The necessary bandwidth is available only on internal connections among chipset, RAM and CPU.

Let me remind you that PCs of current design (before PCI-Express) failed to support even the much lower 125 MB/s needed by Gigabit Ethernet via any of the existing ports, until chipsets were extended by a dedicated port for just that usage.

--
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Hello Hans-Bernhard,

True, it may not be the PC who does the main work. It does though once the data has been transferred to RAM and/or hard disk. But the main thing I'd see in this case is whether it would be enough for what Roel wants to do. If it works for him, who cares whether it is fully or only partially PC based.

Well, I wouldn't try this kind of stuff with some Windows home edition. But it isn't such a big deal to buy the server edition.

Agree, anything above 4GB is a problem in a 32 bit system. PAE gets around that by mapping and I believe that's what they did on the video processing PC that I mentioned which had far more than 4GB. But to truly run a fast and direct access to more than 4GB would require a transition to a 64 bit system.

Again, Roel might not have to do that. If, for example, these huge amounts of data are recorded to catch some event he could transfer overlapping 1GB segments to RAM and have his software comb through that data. Heck, we even did similar stuff in the DOS days. It is comparable to a Holter ECG where a patient's heart signals are monitored for days and then all that data is parsed to find events such as a suspicious fibrillation.

Regards, Joerg

If the dedicated hardware is not appealing and you want to use some PC systems, one alternative solution would be to use eight separate PCs with 8 bits digital input at 66 Mbytes/s or 64 PCs with 1 bit at 66 Mbits/s or 8,25 Mbytes/s. Of course you will need some synchronisation signals.

Using multiple PCs might be an option, if this is a one-off project and the system is required only for a limited time (a few months). Thus, the PCs could be purchased, used for the experiment and then "sold" to the other part of the organisation for normal desktop use.

Paul