So many files, so few files.

Do you know what wget is? Maybe you should go wget some of these websites before it is too late!

--Mac

Oh dear me, Mac, I'm afraid you've gone and done it now! Imagine, all of Agilent's site on my computer. Along with APEX, Allegro, Analog Devices, and that doesn't even start on the A's. Hmm, "If left to run unchecked, it can easily fill up the disk." Acckk!

If you use the wrong command line, you will end up with *The Internet* on your disk!

lol!!

I hope it has a store-as-plain-text option.

NT

I wish I could do that. Once I'll have all "The Internet" on my old 1Go drive, I'll just have to del*.* and then have a multi-teraoctets drive for free.

Hey, I may even DL "The Internet" several times in different directories for more space.

Yes, it's sometimes true, but I've found that places I've relied upon in the past have gone dark. The bottom line, I think, is, if you need it, you'd better get and maintain a personal copy.

How much longer before the HD-DVD technologies become available as PC peripheral drives that can be used for general data archival?

Now that I've gotten a camcorder, DVDs aren't adequate for storing all the data. They were fine for my photos and general PC storage needs before the video-cam.

Just stock up on floppies :-)

Seriously, if you know the filenames Google may find a copy ferreted away somewhere, usually on some .edu or other.

- YD.

Forget that, and take a look at InPhase - 200G per disc

Well that will be great when/if it becomes a commercial and affordable product. HD-DVD though is much closer to prime-time:

I just hope the DRM/IP mess won't cause these to be useless, or only useable on Windows, or some other stupidity.

IIUI InPhase is already shipping prototypes and has got a number of bigname companies taking it seriously. I would not be suprised if it suddenly trumps Bluray and HDDVD. It could still become a 3-way fight.

Information on the Net is available only provided somebody has already placed it there.

The question arises, how reliable is the information source?. Is the source reliable and honest? It's anybody's guess.

There's FAR too much confidence placed in the reliability of Google.

It's so easy for political parties to place distorted mis-information and deliberate lies on the net. As we can be absolutely certain it has already been done.

They who own and control the world's communications systems, radio, TV, telephone, Internet, sattelites, will rule the Earth.

They will eventually takeover from the oil, gas and energy corporations and the insurance internationals.

Bush and Blair are mere pawns. You'll see!

Probably OT, but the best compression I found for home video is brutal, cold hearted editing. Eighteen years of home videos fit on two DVDs. :) They're watchable now, but I still see clips that can be removed without loss.

Got any that you could send to "America's Stupidest Videos?" ;-P

Cheers! Rich

Well, I have a 240 GB RAID at home. It wouldn't cost all that much to upgrade it to a TB. I would imagine that you could easily get quite a few of the sites you want on a disk array you could easily afford.

;-)

--Mac

Indeed, I'm sure you're right. But then what? As the months and years go by, one would want a program to automatically crawl the site and add updated material. Note, I say add, since one wouldn't want to follow the site by deleting material. But this brings up the issue of how to maintain the links, etc. For example if a page is modified to add links to new material, but drop links to older, one would want both versions available, but with what relationship? Everything would be fine for the first few week or months, but the structure would slowly begin to break, if not skillfully handled.

How does the way-back machine handle these issues?

Oh boy, given the size of the media, if this comes in below $1000 per recorder and $10 per slice media. Bluray and HDDVD don't stand a chance. HDDVD is already on the ropes vs dual layer DVD (9.4 GB per slice) with $50 recorders and under $1 media.

That's a good point. Thinking off the top of my head, the main thing you want is pdf's (maybe?). You could search for and delete redundant ones (automatically, of course). Meanwhile, you could just update the html (using wget) but never delete any pdf's (unless they are redundant). If some of the pdf's get orphaned from their links, that is OK, because you could just use google desktop or whatever it is called to index all your own disk space.

That is a good question whose answer I don't know. Must be a database of some sort.

Anyway, you are probably right. It is not really practical, and I guess I should give up fighting the battle.

At least in your case you have a lot of good karma, and you can always post here when you are looking for something with a reasonable expectation that someone else might have it and be willing to provide it to you in some form. ;-)

--Mac

Since the htmls are usually pretty small compared with the pdfs, you could do the Wayback Machine thing--just keep snapshots of the link pages, with appropriately modified pdf file names to avoid overwriting them for a revision change. That way, you could ask for the latest datasheet, or the latest as of June 2005, or whenever.

Cheers,

Phil Hobbs