DIY *nix computer w lots of ram and network: possible?

I don't think it is feasible for a newbie. Too many little things that can go wrong.

For a seasoned professional, it would be feasible, but I doubt most seasoned professionals would bother with it. You can buy something close enough off the shelf.

Why don't you work on something else. How about a DSP board which accepts analog stereo audio inputs and spits out mp3 data in UDP packets.

Just a thought.

--Mac

Well I would go with off the shelf solutions myself. Look into the Micro ATX format of motherboard. there are several that offer a host of processor choices and some have on board chips like the Geo line, etc. or look at the SBC (single Board Computer) format also available in the ATX format and sized to fit in a 3.5 in drive bay. The advantage to the SBC line is the fact they have lan, sound and video and more on board not to mention the support for the PC 104 modules. these things are small fast and can fit anywhere. Most support some kind of Disk on a chip technology so you can have your OS at board level. check out some of these links.

and not to mention our fav place EBay

Let me know how you make out, I've been interested in doing something similar.

R Morton

Google for Beowolf Cluster and see what you find. By the time they are publishing books on how to do things like this you can assume you have a chance. Read everything you can before you start.

I was buying the under-$100 Fry's motherboard/2+ ghz AMD cpu/fan, dropping in a pair of 512 meg ddram for under $100, $2 used pci video cards and cdroms, $6 scrap atx cases, $50 new atx power supplies. Get one cheap multiport keyboard/video/mouse switch box and one cheap network hub. Watch your temperatures, blow a lot of air through them.

They all boot off the cd's and run in the gig of memory. Or you can boot over the network off a single drive.

Find yourself a local linux user group, try to find a helpful person who isn't to rabid and who will answer a few hundred of your questions. Choose a linux that is known for being easy to setup.

Sound advice I'd say.

I'm soon going to put together a new router that will connect my two networks together.

It will be an AMD64 on a ASUS A8V deluxe (which has a gigabit network card built in) + another gigabit card.

Pretty over the top it will have 1 Gig Ram and say 200 Gig Harddrive.

I'll use Fedora core 4 or Suse professional 9.3 for the 64 bit operating system.

I'll host my webserver there too which will be reachable from either network - one on cable and the other on ADSL.

I'll have some gigabit switches to connect to other machines.

I think this is easily doable. Just put it together and load up the software almost. What you want sounds ok too. There are some micro ATX boards with 2 ethernet connections. Don't know if they are Gigabit though. This is better than my idea which doesn't scale well.

I'm confused here as to what you're looking for. First, it sounded like you want a LAN or a router, but by the end, I'm wondering, "Web TV? Playstations?? "

You can bet a bare-bones computer for next to nothing, with a MB that will take up to 4GB RAM, I'm pretty sure, plug in a network card, and load up just about anybody's distro of Linux - I guess there are a couple that are designed to be stripped down, and I guess QNX or so is optimized for embedded apps. But, for example, Slackware, which I use, could be installed in just a couple of megabytes of disk, completely stripped to the bones, although I'd want to put _some_ utility to it. ;-) Like, terminals?

I guess what I'm getting at is, what are you trying to accomplish?

Thanks, Rich

OK - what I'm really after is a large (say, >= 32G) ram disk: for the kind of processing I do, I'm almost always IO bound, CPU idle. I want to load my OS, programs and data into memory and stay there until I sync to disk.

Vendors that make large solid-state drives are either slow or really (>$10K) expensive. The closest commercial offering I've found to what I want is this:

But it's only for Windows, takes proprietary DIMMS, and isn't going over 4GB.

With 1GB DIMMs in the ~$70 range - I'm looking for a way to spend, say, $3K to get 32G of filesystem space with microsecond, not millisecond latency.

Ideally I'd like something like the RocketDrive, ie that plugs directly into my PCI bus and appeared as a drive. That seemed both very complicated and less scalable, so I was considering a cluster of IP nodes - on a dedicated switch (basically, a not-totally-poor-man's SAN).

Of course you're absolutely right - I could just string together cheap PC mobos that each are maxed out at 4GB of ram -- and if I'm serious about actually implementing this I probably will. But 8x(mobo + ps + cpu) starts to get big, loud, and (possibly) more expensive than it has to be. Those mobos will have so much extra stuff on them that's not needed for the application, that I was wondering what the feasibility was of constructing a node that has nothing but RAM and disk, that could lead to a cleaner, smaller, and hopefully cheaper solution.

Hopefully that's clearer - any thoughts?

thanks,

-usman

You'd be pretty unlikely to get microseond latency by the time you're going through your own PC's network stack, the switch, and your storage device...

Still, I like your idea. It would be pretty simple to build an "Ethernet hard drive" at the hardware level though... just some Ethernet controller IC and possibly even just an FPGA could do the actually memory access -- no processor needed. This approach forces you to put all the "intelligence" in a driver on the PC side, though -- and writing a file system driver for Windows is a HIGHLY non-trivial undertaking! A much easier solution would be to make your peripheral appear to Windows as a SCSI drive... writing a SCSI driver is not particularly difficult (Windows gives your driver the complete SCSI control block and says, "make it so!" -- i.e., do whatever you need to do to get the drive to "hear" the command), and then you just need to hook your SCSI device driver up to a system Ethernet driver somewhere. What happens, then, is that your RAM disk peripheral will literally end up being formatted as FAT32 or NTFS or whatever. This approach also makes it easier to be able to use the peripheral from other OSes (Linux, Mac OS, etc.)

The overall easiest solution would perhaps be to hack into some single board Linux machine (such as those used in wireless access points) and only write a Linux device driver (which is much simpler than a Windows device driver, by the way) for a bunch of RAM you somehow 'grafted' onto the board. Now your peripheral immediately works with Windows networking, NFS, etc.... cool! ...but the downside is that every I/O request has to go through the entire Linux networking stack, so your latency is going to be much higher (probably milliseconds...), although with some tweaking you'll still have a tremendous throughput advantage over rotating media.

---Joel Kolstad

WINterms.

Also google the term "dickless workstation".

Tim.

Right, this is the other approach - I'd much prefer this, really - my background is software and fs drivers sound a lot easier to me than hardware.

I had ruled this out because I didn't think I could build a node with large RAM without making it a full blown system anyway - can you tell me a bit more about what this might look like? Could I scale it to 4GB / disk? Could I use off-the-shelf DDR DIMMS?

Thanks for any comments/answers!

-usman

Ah, the REAL question.

There are 64 bit motherboards that don't have the 4GB limitation, they just cost an order of magnitude more than the usual boards, and often don't accept the bargain basement cheap pc memory sticks.

You'd perhaps want to consider implementing all your OS drivers in VHDL, and maybe the whole OS. The layer after layer of software is likely to swamp the amount of time it takes.

Ask what the real typical throughput is for their drive, not some hypothetical burst rate.

It seems unlikly that you are going to get a working system that is cheaper than the commodity PC boards with 4 gig each.

I'm after plans for building a >300khz ADV converter. The use it in recording from an ultrasonic microphone to a portable computer.

The interest in being able to accurately record bat calls. I have heterodyne and frequency division systems for real time listening, but for accurate analysis an exact record of the sound is required.

Looking for a commercial solution leads to either a very expensive >10Mhz mains powered solution, or a cheap

This board (Tyan S2875S) takes 8GB (4 x 2GB sticks) and didn't cost anywhere near an order of magnitude more than any other 64bit board. I bought it a little over a year ago for about $200. It does take registered DIMMs, but apples to apples they aren't all that much more expensive at the smaller sizes ($70 vs $63 for 512MB, DDR400). I don't see 2GB unbuffered DIMMs though:

Why nolt add more processors? A 4-way Opteron board should be able to handle 32GB natively. ...as long as we're talking a money-pit. ;-)

The burst rates on the outter tracks should be sustainable. Of course if you have lotsa small files you have that software uglyness again. The trick is staying on the outer tracks. ;-)

Or not-so-commodity with 32GB each, for that matter.

The original poster was asking for a board that would support 32 gig. Those boards are out there but they aren't $200. Even fairly ordinary Opteron boards were $400+. And he was quoting commodity prices for fairly cheap ram but wanting a board that would run 32 gig.

Because he was asking for microsecond response to get blocks off the "disk" AND he didn't want to throw unlimited money at it, he was asking for sale price memory to be (roughly) the whole cost of the system and still get blocks off the disk in a microsecond.

And he isn't going to get a sustained rate of a block a microsecond. Half a dozen years ago someone was trying to get "real time" performance out of available "finger speed" OS. They did some measurements and found it not too surprising to have 100,000 microsecond variations in response.

Yup

"Farily ordinary Opteron boards" are $200, or less.

I understand. I want a Maseratti for $50 too. Would you design and build me one?

Wants, wants, wants. I want the above Maseratti too.

Sure, That makes sense, since 100msec is on the order of the OS's context switch time.

Dreamers are everywhere. ;-)

You can have lots of fast memory on a PC - that much is easy enough. Fast access to this memory will surely be the difficult part.

How fast can you move data over 1Gb ethernet when you factor in the efficiency of the protocols above the link layer? I bet modern hard disks would give this ramdisk scheme a run for it's money.

If you can offload some of the data processing onto the individual nodes such that data is moved there, processed, and the results moved back - then you may win.

Hope this helps.

Regards, Alan

Ooo... where and which ones? I thought I checked Pricewatch and only saw those that were 2x that or more. I wanted to try out Solaris 10 but that only works on Opterons and Sparc's. For me a slow 1-way Opteron and a board that would accept commodity cheap ram would have been enough, but I'm trying to not have high expectations.

He can certainly buy himself a server board that would support 32 gig. And he could put a 64 bit OS on it, open up his 32 gig file and put it in one big block of memory and skip all file system operations after that. That would give him microsecond access to the whole block.

Slower than that, but not abandoning the file system completely was the Solaris "mapping a filesystem into memory." That's the equivalent of "ram drive" in PC-speak. But that must pay for software overhead.

Well, that was him.

I think some of the Windows timings were substantially worse than that, not having pre-emptive switching at that point in time.

Everybody needs a hobby

I paid about $200 for this Tyan S2875S(single CPU) and the S2875 (dual CPU) was about $30 more (I wish I'd gone for it). Looking at NewEgg, it seems that socket-940 isn't all that popular anymore (they no longer carry Tyan), though they have a FoxCOnn board for $224. NewEgg lists the Tyan S2850 (also the AMD chipset) from for $218.

Why wouldn't Solaris run on an Athlon64? They're much cheaper and do take cheap RAM.

Understood.

Sure. "Cooperative multitasking" is oxymoronic.

;-)

The Philips ARM chips have 400 kHz A/D converters.

Wouter van Ooijen

-- ------------------------------------

Specific implementations, of course, are left as an exercise for the reader.