So many files, so few files.

Winfield Hill · 2006-01-24T01:37:28+00:00

Ah, there's so much critical information in the world that we need at our fingertips. One simply can't rely on finding it on the net, or depending on a manufacturer to keep the files available, so one really has to keep copies in one's own computer(s), right? So many files. Whew, take my Keithley instrument folder: 396 files in 70 folders taking 532MB of space. But so few files, and so much critical missing information. Take my Agilent HP folders. Instruments: 725 files in 110 folders, 1043MB total, but really sadly lacking in the stuff I really need. Agilent semiconductors: 343 files, 86MB, but so few files; I really know I'm in trouble now that ownership has changed. That's usually when the website gets re-designed and the old stuff goes away. :-( It's probably still on their web-server computers, but the links to find it are broken. Sad, so sad. Well, so little time, better go get some more files.-- Thanks, - Win

Jan 24, 2006 22 Replies

Rate this thread:

Rich Grise 20 years ago

An ordinary data base, where each datum is, instead of a single file, a list of the file and all of its previous versions, which are identified by appending the file's original date into its name. (you'd rename today's yesterday's version either with its original date, if you can remember it, or today's date, since the new one won't have a date in its name - that would be the "superseded" date - as part of the "update" script. Yeah, today's date would be easiest.)

e.g:

A: Amplifiers 1. Transistor a. common emitter i. Circuit-A Circuit-A.20050618 Circuit-A.20041303 b. Circuit-B Circuit-B.20050213 2. Tube ... B: Oscillator ...

and so on.

Figuring out categories, of course, is left as an exercise for the reader. ;-)

Good Luck! Rich

Vote

John Devereux 20 years ago

You could use a standard revision control system to handle this. E.g. like svn, cvs, bitkeeper. Periodically (every month say), use something like wget to synchronize your copy with the remote site, then use the revision control tool to save the snapshot. You will then be able to take your copy of the site backwards and forwards in time. The revision control tool only stores the *changes* to the files, so you do not need space for N copies of the site.

Of course this is still quite a big job, especially for multiple sites! You could use scripting to automate a lot of it. I imagine the main problem would be "dynamic" sites that use javascript, or get you to fill out a form, in order to access data.

Don't know... some specialised version of the above, perhaps.

John Devereux

Vote

Robert Latest 20 years ago

On 25 Jan 2006 05:42:48 -0800, Winfield Hill wrote in Msg.

Not only that -- kepp in mind that websites are less and less just pseudo-static html files but often have javascript and other (often dysfunctional) "goodies" in them: this is where automated website mirroring tools hopelessly get lost.

robert

Vote

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required