[Dirvish] Job checkpoint/restart
keithl at kl-ic.com
Tue Jun 14 06:28:05 UTC 2005
On Mon, Jun 13, 2005 at 10:55:44PM -0700, Brian Martin wrote:
> This seems to work, but it's awful! And it only works if I know the outage
> is coming. Is there any better way to do this?
If you have slow connections, I suggest you move things with sneakernet
to start with. Bring over a hard disk in a USB cage, or if they are
distant, FedEx the disk. Oregon Health Sciences University and CalTech
have a big medical imaging project going, with 2TB data sets moving
back and forth - rather than using the internet, they just ship servers
Otherwise, make your vaults small. You can use excludes to chop out
parts of large directories, and start your trees pretty deep. You are
doing all this partly so you can restore drives, so keep the vaults
small enough to move *back*.
In general, it is not good to try to move data sets that take a long
time with any technology, including Rsync. Too many things can go
Keith Lofstrom keithl at keithl.com Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs
More information about the Dirvish