[Dirvish] One of the vaults broken - init enough?

Wed Sep 29 10:41:26 UTC 2021

Thank you very much again for all your information. Although I tried my 
best, I couldn't save the old backup sequence anymore. My impression was 
that at some point I might have deleted or corrupted one of the database 
information files for dirvish, so it couldn't make out when and what 
(for this single backup). Even another initiation wouldn't help.

So I decided to delete all existing backups and their database 
information files and leave the disc like a brand new one with only the 
default backup information structure for dirvish. Then I started an 
initiation for each part backup and now it has been running ok for 
couple of days.

Anyway I could gain a good refresh of my dirvish knowledge :)

Regards
Rolf

Am 08.09.21 um 15:14 schrieb Isaac W Traxler:
> Presumably you had a working abckup at some point in the past and 
> something went wrong. You now probably have numerous backup directories 
> where a portion are still good but some number have littlle of no data.
> 
> The first goal is to get a good current backup. Before anythong elase, 
> make sure no backups are running for home (as suggested by others). Now 
> run a backup  init to get a good backup of home. If this fails, tell us 
> ehat the log  files say. Once that works, tyour future backups should be 
> fine.
> 
> This leaves the question of the exising backup directories. One approach 
> would be to do nothing and simply let them eventually get purged out. A 
> better approach would be to identify the failed backups and eliminate 
> them (I go to the home directory and do a "du -sh *".  This command will 
> take a while, but it will report the amount of space that each backup is 
> taking. It should be obvious which ones failed (much smaller in space). 
> You can confirmthey failed by looking at the summary files. I normally 
> delete failed backup directories (so I don't assume they are there to 
> restore from).
> 
> The next issue is disk usage. Each init does a "full" backup and does 
> not take advantage of dirvish's (rsync's) ability to simply backup 
> changes. This results in duplicate copies of files. If space is of no 
> issue, you can not worry about this (and eventually as they age out this 
> problem will fix itself). When this issue happens, I go to the directory 
> above (my tree looks like /backups/server/file_system -- so I "cd 
> /backups/server"). and then I run "hardlink file_systyem" and wait for 
> it to finish (this may take a long time also). the hardlink command will 
> identify all duplicate files and replace duplicates with hardlinks to 
> one copy of the data (essentially what Dirvish/rsync does when making 
> the backups). This will recover any "lost" space due to duplicate files.
> 
> In fact, I make it a practice to periodcally run hardlink on every vault 
> to eliminate any duplicate files as a normal part of maintenance.
> 
> Isaac Traxler
> LSU/LONI HPC Storage & Infrastructure Manager
> traxler at lsu.edu
> 
> ------------------------------------------------------------------------
> *From:* dirvish <dirvish-bounces at lists.dirvish.org> on behalf of 
> Rolf-Werner Eilert <rwe-sse at osnanet.de>
> *Sent:* Wednesday, September 8, 2021 5:17 AM
> *To:* dirvish at dirvish.org <dirvish at dirvish.org>
> *Subject:* [Dirvish] One of the vaults broken - init enough?
> I hope this list is still alive...
> 
> Some time ago I received a
> 
> 21:45:44 /usr/sbin/dirvish --vault backup-home
> backup-home:default:20210727-2145:
> /mnt/backups/backup-home/dirvish/lock_file already exists
> 
> on the /home vault. After some days I looked it up and yes, I found a
> lock file and deleted it, but it had no effect. So I decided to initiate
> the /home vault again, then let it backup (only /home) once and went
> into holidays.
> 
> Since then I get
> 
> 21:24:51 /usr/sbin/dirvish --vault backup-home
> backup-home:default:20210830-2124: ERROR: no images for branch default found
> 
> Guess I messed it up, but how can I re-initiate the /home? Of course,
> the cronjob starts dirvish-runall, so do I have to expire all of them
> first, then initiate again?
> 
> Thank you for your help!
> 
> Rolf
> 
> 
> _______________________________________________
> dirvish mailing list
> dirvish at lists.dirvish.org
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.dirvish.org%2Flistinfo%2Fdirvish&data=04%7C01%7Ctraxler%40lsu.edu%7Cb71e92782b944fece56d08d972b1d5b2%7C2d4dad3f50ae47d983a09ae2b1f466f8%7C0%7C0%7C637666930839365745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=bqzuMRKhLDAtRMs8%2BjCgyi5Uc%2FXbmhzGTfhzFaIbEa8%3D&reserved=0 
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.dirvish.org%2Flistinfo%2Fdirvish&data=04%7C01%7Ctraxler%40lsu.edu%7Cb71e92782b944fece56d08d972b1d5b2%7C2d4dad3f50ae47d983a09ae2b1f466f8%7C0%7C0%7C637666930839365745%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=bqzuMRKhLDAtRMs8%2BjCgyi5Uc%2FXbmhzGTfhzFaIbEa8%3D&reserved=0>
> 
> _______________________________________________
> dirvish mailing list
> dirvish at lists.dirvish.org
> https://lists.dirvish.org/listinfo/dirvish
> 