From james at netinertia.co.uk Tue Jun 5 17:20:13 2012 From: james at netinertia.co.uk (James O'Gorman) Date: Tue, 5 Jun 2012 17:20:13 +0100 Subject: [Box Backup] www.boxbackup.org is now IPv6 enabled Message-ID: <20120605162013.GN2523@netinertia.co.uk> Hi everyone, Just a quick note to say I've just finished moving the main web and Subversion server to a new machine, and everything is now IPv6 enabled in time for the World IPv6 Launch :-) http://www.worldipv6launch.org/ On the off-chance you do find any issues with the new system, do let me know. James From cam.lafit at azerttyu.net Tue Jun 5 17:28:31 2012 From: cam.lafit at azerttyu.net (cam.lafit at azerttyu.net) Date: Tue, 5 Jun 2012 18:28:31 +0200 Subject: [Box Backup] www.boxbackup.org is now IPv6 enabled In-Reply-To: <20120605162013.GN2523@netinertia.co.uk> References: <20120605162013.GN2523@netinertia.co.uk> Message-ID: Hi Nice New :) From achim+box at qustodium.net Tue Jun 5 20:16:03 2012 From: achim+box at qustodium.net (Achim J. Latz) Date: Tue, 05 Jun 2012 21:16:03 +0200 Subject: [Box Backup] www.boxbackup.org is now IPv6 enabled In-Reply-To: <20120605162013.GN2523@netinertia.co.uk> References: <20120605162013.GN2523@netinertia.co.uk> Message-ID: <4FCE5AF3.6020007@qustodium.net> Hello James: On 05/06/2012 18:20, James O'Gorman wrote: > Hi everyone, > > Just a quick note to say I've just finished moving the main web and > Subversion server to a new machine, and everything is now IPv6 enabled > in time for the World IPv6 Launch :-) > > http://www.worldipv6launch.org/ > > On the off-chance you do find any issues with the new system, do let me > know. Thanks for the work, and for the heads-up about the "event". Does anybody know if apart from its web site, Box Backup itself is IPv6-ready? Thanks, Achim -- Achim J. Latz, Qustodium Internet Security achim.latz at qustodium.net ? http://www.qustodium.net Data Encryption ? Backup Automatisation ? E-Mail Protection From james at netinertia.co.uk Tue Jun 5 23:40:25 2012 From: james at netinertia.co.uk (James O'Gorman) Date: Tue, 5 Jun 2012 23:40:25 +0100 Subject: [Box Backup] www.boxbackup.org is now IPv6 enabled In-Reply-To: <4FCE5AF3.6020007@qustodium.net> References: <20120605162013.GN2523@netinertia.co.uk> <4FCE5AF3.6020007@qustodium.net> Message-ID: <20120605224025.GO2523@netinertia.co.uk> On Tue, Jun 05, 2012 at 09:16:03PM +0200, Achim J. Latz wrote: > Does anybody know if apart from its web site, Box Backup itself is > IPv6-ready? Good question! I've had a glance at the server code (lib/server/Socket.cpp) and I think the answer is "no" - there are only entries for AF_INET and AF_UNIX. I think IPv6 requires AF_INET6. Please don't take this as a definitive answer as I'm not a C++ expert :-) I'm sure Chris will be able to confirm. James From misc.lists at fsck.ch Thu Jun 7 19:52:29 2012 From: misc.lists at fsck.ch (Toby) Date: Thu, 07 Jun 2012 20:52:29 +0200 Subject: [Box Backup] boxbackup bahaviour when renaming directories Message-ID: <4FD0F86D.1020600@fsck.ch> Hi I recently renamed a bunch of directories on my machine running bbackupd. I was hopeing (but not counting on it) that the files contained in those directories would not have to be re-uploaded. Now when I checked with bbackupquery, I noticed that both the old and the new directories where there, both still contained files, and nothing was marked deleted. The new directory and the new files had different IDs, so I guess they were reuploaded. Now why wheren't the old directories marked deleted? Does this take some time? Thanks in advance, Toby From chris at qwirx.com Thu Jun 7 20:37:13 2012 From: chris at qwirx.com (Chris Wilson) Date: Thu, 7 Jun 2012 20:37:13 +0100 (BST) Subject: [Box Backup] boxbackup bahaviour when renaming directories In-Reply-To: <4FD0F86D.1020600@fsck.ch> References: <4FD0F86D.1020600@fsck.ch> Message-ID: Hi Toby, On Thu, 7 Jun 2012, Toby wrote: > I recently renamed a bunch of directories on my machine running > bbackupd. I was hopeing (but not counting on it) that the files > contained in those directories would not have to be re-uploaded. > > Now when I checked with bbackupquery, I noticed that both the old and > the new directories where there, both still contained files, and nothing > was marked deleted. The new directory and the new files had different > IDs, so I guess they were reuploaded. > > Now why wheren't the old directories marked deleted? Does this take some > time? Good question. Please could you let us know what version of Box Backup you're running, and run bbackupd with the "-o /tmp/bbackupd.log -Otrace" and send the logfile to me privately, or at least the part that relates to the directory that you deleted? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From thimo at g4t3.de Fri Jun 8 12:17:17 2012 From: thimo at g4t3.de (Thimo Langbehn) Date: Fri, 08 Jun 2012 13:17:17 +0200 Subject: [Box Backup] Wiki account requested + Bacula correction Message-ID: <4FD1DF3D.7030501@g4t3.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear Box Backup Admins, I humbly request a Wiki account for the user "Thimo" to help improve the documentation on my journey to use this system. Further, there is a small error in the backup system comparison [1] regarding Bacula, which is indeed capable of encrypting the data on the source system (by the FileServer daemon) using OpenSSL [2]. This option is not available on all distributions due to the license problem (advertising clause) with OpenSSL, nevertheless, the system supports it. It is even possible to add a second (Master) key that will be able to decode the encrypted data again. Since Bacula uses a DB-based catalog, the filenames and paths of the encrypted jobs are still accessible, but I would argue that this is a minor point. Cheers and thank you for this interesting system, Thimo Langbehn (Germany) [1] http://www.boxbackup.org/wiki/BoxComparison [2] http://www.bacula.org/5.2.x-manuals/en/main/main/Data_Encryption.html -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/R3z0ACgkQLoHAUMxCwiWMVgCdGnpsw5kNqFWEL+Q5sLdbJL/a z5oAoLVfnsIairGh0AKqYpcJOg86MjdI =5MB9 -----END PGP SIGNATURE----- From dave at bdisystems.co.uk Fri Jun 8 15:19:27 2012 From: dave at bdisystems.co.uk (dave bamford) Date: Fri, 08 Jun 2012 15:19:27 +0100 Subject: [Box Backup] Housekeeping still crashing Message-ID: <1339165167.3720.37.camel@crusty.backed-up.net> Hi I am running Box Backup Store Server vRELEASE_0_11rc8_3092 on Debian stable and I am still getting problems with housekeeping. Its been running very well for several weeks with the bbstored -V command so about a week ago I switched to running bbstored -v. I then noticed one particular account was at the hard limit and housekeeping was not getting round to freeing up space so I ran a check fix on the account here are the logs > Jun 5 12:56:23 bart bbstoreaccounts[1151]: NOTICE: Will fix errors encountered during checking. > Jun 5 12:56:23 bart bbstoreaccounts[1151]: Checking store account ID 0x00001000... > Jun 5 12:56:23 bart bbstoreaccounts[1151]: Phase 1, check objects... > Jun 5 13:43:25 bart bbstoreaccounts[1151]: Max dir starting ID is 0x72800 > Jun 5 13:43:49 bart bbstoreaccounts[1151]: WARNING: Spurious file backup/00001000/refcount.db found, deleting > Jun 5 21:45:15 bart bbstoreaccounts[1151]: Phase 2, check directories... > Jun 5 22:13:32 bart bbstoreaccounts[1151]: WARNING: File ID 0x6788a has different container ID, probably moved > Jun 5 22:13:39 bart bbstoreaccounts[1151]: WARNING: File ID 0x6c8e8 has different container ID, probably moved > Jun 5 22:13:40 bart bbstoreaccounts[1151]: WARNING: File ID 0x6c6da has different container ID, probably moved > Jun 5 22:13:40 bart bbstoreaccounts[1151]: WARNING: File ID 0x6c6d9 has different container ID, probably moved > Jun 5 22:15:40 bart bbstoreaccounts[1151]: WARNING: File ID 0x68452 has different container ID, probably moved > Jun 5 22:16:05 bart bbstoreaccounts[1151]: WARNING: File ID 0x661cd has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x64292 has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x6fa8 has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x69a5c has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x69a60 has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x69a63 has different container ID, probably moved > Jun 5 22:16:27 bart bbstoreaccounts[1151]: WARNING: File ID 0x69a6a has different container ID, probably moved > Jun 5 22:16:46 bart bbstoreaccounts[1151]: WARNING: File ID 0x6726d has different container ID, probably moved > Jun 5 22:16:46 bart bbstoreaccounts[1151]: WARNING: File ID 0x63042 has different container ID, probably moved > Jun 5 22:16:46 bart bbstoreaccounts[1151]: WARNING: File ID 0x661cf has different container ID, probably moved > Jun 5 22:16:46 bart bbstoreaccounts[1151]: WARNING: File ID 0x6884a has different container ID, probably moved > Jun 5 22:16:46 bart bbstoreaccounts[1151]: WARNING: File ID 0x66292 has different container ID, probably moved > Jun 5 22:16:54 bart bbstoreaccounts[1151]: WARNING: File ID 0x69d56 has different container ID, probably moved > Jun 5 22:17:00 bart bbstoreaccounts[1151]: WARNING: File ID 0x6ca23 has different container ID, probably moved > Jun 5 22:17:13 bart bbstoreaccounts[1151]: Phase 3, check root... > Jun 5 22:17:13 bart bbstoreaccounts[1151]: Phase 4, fix unattached objects... > Jun 5 22:17:13 bart bbstoreaccounts[1151]: Phase 5, fix unrecovered inconsistencies... > Jun 5 22:17:13 bart bbstoreaccounts[1151]: Phase 6, regenerate store info... > Jun 5 22:17:13 bart bbstoreaccounts[1151]: WARNING: Soft limit for account changed to ensure housekeeping doesn't delete files on next run. > Jun 5 22:17:13 bart bbstoreaccounts[1151]: WARNING: Hard limit for account changed to ensure housekeeping doesn't delete files on next run. > Jun 5 22:17:13 bart bbstoreaccounts[1151]: NOTICE: New store info file written successfully. > Jun 5 22:17:13 bart bbstoreaccounts[1151]: WARNING: Finished checking store account ID 0x00001000: 1 errors found > Jun 5 22:17:13 bart bbstoreaccounts[1151]: WARNING: You should now use bbackupquery on the client machine to examine the store. 2 days later housekeeping falls over trying to rename a file in that account > Jun 7 23:49:24 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20245 from dir 0x1ee99 > Jun 7 23:49:26 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20246 from dir 0x1ee99 > Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to rename file: /backups/box/backup/00001000/ee/01/o99.rfwX to /backups/box/backup/00001000/ee/01/o99.rfw: No such file or directory (2) > Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) > Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to delete file: /backups/box/backup/00001000/ee/01/o99.rfwX: No such file or directory (2) > Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) > Anyone any ideas? Also I changed the limits on the account between checking and housekeeping failing and there was also an ERROR reported > Jun 6 12:55:17 bart bbstoreaccounts[12599]: NOTICE: Limits on account 0x00001000 changed to 38273024 soft, 41943040 hard. > Jun 7 05:05:17 bart bbstored/hk[2139]: ERROR: Housekeeping on account 0x00001063 threw exception, aborting run for this account: BackupStore PatchChainInfoBadInDirectory (A directory contains inconsistent information. Run bbstoreaccounts check to fix it.) (4/67) Thanks Dave Bamford From chris at qwirx.com Fri Jun 8 15:31:32 2012 From: chris at qwirx.com (Chris Wilson) Date: Fri, 8 Jun 2012 15:31:32 +0100 (BST) Subject: [Box Backup] Housekeeping still crashing In-Reply-To: <1339165167.3720.37.camel@crusty.backed-up.net> References: <1339165167.3720.37.camel@crusty.backed-up.net> Message-ID: Hi Dave, On Fri, 8 Jun 2012, dave bamford wrote: > I am running Box Backup Store Server vRELEASE_0_11rc8_3092 on Debian > stable and I am still getting problems with housekeeping. Its been > running very well for several weeks with the bbstored -V command so > about a week ago I switched to running bbstored -v. ... > 2 days later housekeeping falls over trying to rename a file in that account > >> Jun 7 23:49:24 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20245 from dir 0x1ee99 >> Jun 7 23:49:26 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20246 from dir 0x1ee99 >> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to rename file: /backups/box/backup/00001000/ee/01/o99.rfwX to /backups/box/backup/00001000/ee/01/o99.rfw: No such file or directory (2) >> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) >> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to delete file: /backups/box/backup/00001000/ee/01/o99.rfwX: No such file or directory (2) >> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) Could you try running it with -V again? That way I hope I'll get a stack trace. > Also I changed the limits on the account between checking and housekeeping failing and there was also an ERROR reported > >> Jun 6 12:55:17 bart bbstoreaccounts[12599]: NOTICE: Limits on account 0x00001000 changed to 38273024 soft, 41943040 hard. >> Jun 7 05:05:17 bart bbstored/hk[2139]: ERROR: Housekeeping on account 0x00001063 threw exception, aborting run for this account: BackupStore PatchChainInfoBadInDirectory (A directory contains inconsistent information. Run bbstoreaccounts check to fix it.) (4/67) 1063 is a different account, so that doesn't seem to be related? Did you try checking and fixing that account? Cheres, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From dave at bdisystems.co.uk Fri Jun 8 23:32:31 2012 From: dave at bdisystems.co.uk (Dave Bamford) Date: Fri, 8 Jun 2012 23:32:31 +0100 Subject: [Box Backup] Housekeeping still crashing In-Reply-To: References: <1339165167.3720.37.camel@crusty.backed-up.net> Message-ID: <88DD9DFE-4391-41B2-9004-EA0240FDAAE1@bdisystems.co.uk> Hi chris Checking 1063 right now. Will start with -V but ran for over a month without problems. Is it possible that the verbosity may change the results? Will keep you posted Thanks Dave On 8 Jun 2012, at 15:31, Chris Wilson wrote: > Hi Dave, > > On Fri, 8 Jun 2012, dave bamford wrote: > >> I am running Box Backup Store Server vRELEASE_0_11rc8_3092 on Debian >> stable and I am still getting problems with housekeeping. Its been >> running very well for several weeks with the bbstored -V command so >> about a week ago I switched to running bbstored -v. > ... >> 2 days later housekeeping falls over trying to rename a file in that account >> >>> Jun 7 23:49:24 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20245 from dir 0x1ee99 >>> Jun 7 23:49:26 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20246 from dir 0x1ee99 >>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to rename file: /backups/box/backup/00001000/ee/01/o99.rfwX to /backups/box/backup/00001000/ee/01/o99.rfw: No such file or directory (2) >>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) >>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to delete file: /backups/box/backup/00001000/ee/01/o99.rfwX: No such file or directory (2) >>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) > > Could you try running it with -V again? That way I hope I'll get a stack trace. > >> Also I changed the limits on the account between checking and housekeeping failing and there was also an ERROR reported >> >>> Jun 6 12:55:17 bart bbstoreaccounts[12599]: NOTICE: Limits on account 0x00001000 changed to 38273024 soft, 41943040 hard. >>> Jun 7 05:05:17 bart bbstored/hk[2139]: ERROR: Housekeeping on account 0x00001063 threw exception, aborting run for this account: BackupStore PatchChainInfoBadInDirectory (A directory contains inconsistent information. Run bbstoreaccounts check to fix it.) (4/67) > > 1063 is a different account, so that doesn't seem to be related? Did you try checking and fixing that account? > > Cheres, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup From david at dward.us Sat Jun 9 02:42:00 2012 From: david at dward.us (David Ward) Date: Sat, 09 Jun 2012 11:42:00 +1000 Subject: [Box Backup] Housekeeping still crashing In-Reply-To: <88DD9DFE-4391-41B2-9004-EA0240FDAAE1@bdisystems.co.uk> References: <1339165167.3720.37.camel@crusty.backed-up.net> <88DD9DFE-4391-41B2-9004-EA0240FDAAE1@bdisystems.co.uk> Message-ID: <4FD2A9E8.1030409@dward.us> Good point Dave. I have noticed over the years with some executables that hang, when I run them prefixed with strace, they work. Particularly network applications. No science in my response here, but my theory is the added delay of strace helps the timing of execution. Perhaps someone knows better. Regards David Ward m: 0410 472 531 skype: DaveQB twitter: DaveQB14 www: www.dward.us On 09/06/12 08:32, Dave Bamford wrote: > Hi chris > > Checking 1063 right now. Will start with -V but ran for over a month without problems. Is it possible that the verbosity may change the results? > > Will keep you posted > > Thanks > > Dave > > On 8 Jun 2012, at 15:31, Chris Wilson wrote: > >> Hi Dave, >> >> On Fri, 8 Jun 2012, dave bamford wrote: >> >>> I am running Box Backup Store Server vRELEASE_0_11rc8_3092 on Debian >>> stable and I am still getting problems with housekeeping. Its been >>> running very well for several weeks with the bbstored -V command so >>> about a week ago I switched to running bbstored -v. >> ... >>> 2 days later housekeeping falls over trying to rename a file in that account >>> >>>> Jun 7 23:49:24 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20245 from dir 0x1ee99 >>>> Jun 7 23:49:26 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20246 from dir 0x1ee99 >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to rename file: /backups/box/backup/00001000/ee/01/o99.rfwX to /backups/box/backup/00001000/ee/01/o99.rfw: No such file or directory (2) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to delete file: /backups/box/backup/00001000/ee/01/o99.rfwX: No such file or directory (2) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) >> Could you try running it with -V again? That way I hope I'll get a stack trace. >> >>> Also I changed the limits on the account between checking and housekeeping failing and there was also an ERROR reported >>> >>>> Jun 6 12:55:17 bart bbstoreaccounts[12599]: NOTICE: Limits on account 0x00001000 changed to 38273024 soft, 41943040 hard. >>>> Jun 7 05:05:17 bart bbstored/hk[2139]: ERROR: Housekeeping on account 0x00001063 threw exception, aborting run for this account: BackupStore PatchChainInfoBadInDirectory (A directory contains inconsistent information. Run bbstoreaccounts check to fix it.) (4/67) >> 1063 is a different account, so that doesn't seem to be related? Did you try checking and fixing that account? >> >> Cheres, Chris. >> -- >> _____ __ _ >> \ __/ / ,__(_)_ | Chris Wilson Cambs UK | >> / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | >> \__/_/_/_//_/___/ | We are GNU : free your mind & your software | >> _______________________________________________ >> Boxbackup mailing list >> Boxbackup at boxbackup.org >> http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup From david at dward.us Sat Jun 9 02:41:36 2012 From: david at dward.us (David Ward) Date: Sat, 09 Jun 2012 11:41:36 +1000 Subject: [Box Backup] Housekeeping still crashing In-Reply-To: <88DD9DFE-4391-41B2-9004-EA0240FDAAE1@bdisystems.co.uk> References: <1339165167.3720.37.camel@crusty.backed-up.net> <88DD9DFE-4391-41B2-9004-EA0240FDAAE1@bdisystems.co.uk> Message-ID: <4FD2A9D0.9060207@dward.us> Good point Dave. I have noticed over the years with some executables that hang, when I run them prefixed with strace, they work. Particularly network applications. No science in my response here, but my theory is the added delay of strace helps the timing of execution. Perhaps someone knows better. Regards David Ward m: 0410 472 531 skype: DaveQB twitter: DaveQB14 www: www.dward.us On 09/06/12 08:32, Dave Bamford wrote: > Hi chris > > Checking 1063 right now. Will start with -V but ran for over a month without problems. Is it possible that the verbosity may change the results? > > Will keep you posted > > Thanks > > Dave > > On 8 Jun 2012, at 15:31, Chris Wilson wrote: > >> Hi Dave, >> >> On Fri, 8 Jun 2012, dave bamford wrote: >> >>> I am running Box Backup Store Server vRELEASE_0_11rc8_3092 on Debian >>> stable and I am still getting problems with housekeeping. Its been >>> running very well for several weeks with the bbstored -V command so >>> about a week ago I switched to running bbstored -v. >> ... >>> 2 days later housekeeping falls over trying to rename a file in that account >>> >>>> Jun 7 23:49:24 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20245 from dir 0x1ee99 >>>> Jun 7 23:49:26 bart bbstored/hk[2139]: Housekeeping removed deleted file 0x20246 from dir 0x1ee99 >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to rename file: /backups/box/backup/00001000/ee/01/o99.rfwX to /backups/box/backup/00001000/ee/01/o99.rfw: No such file or directory (2) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: ERROR: Failed to delete file: /backups/box/backup/00001000/ee/01/o99.rfwX: No such file or directory (2) >>>> Jun 7 23:49:28 bart bbstored/hk[2139]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) >> Could you try running it with -V again? That way I hope I'll get a stack trace. >> >>> Also I changed the limits on the account between checking and housekeeping failing and there was also an ERROR reported >>> >>>> Jun 6 12:55:17 bart bbstoreaccounts[12599]: NOTICE: Limits on account 0x00001000 changed to 38273024 soft, 41943040 hard. >>>> Jun 7 05:05:17 bart bbstored/hk[2139]: ERROR: Housekeeping on account 0x00001063 threw exception, aborting run for this account: BackupStore PatchChainInfoBadInDirectory (A directory contains inconsistent information. Run bbstoreaccounts check to fix it.) (4/67) >> 1063 is a different account, so that doesn't seem to be related? Did you try checking and fixing that account? >> >> Cheres, Chris. >> -- >> _____ __ _ >> \ __/ / ,__(_)_ | Chris Wilson Cambs UK | >> / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | >> \__/_/_/_//_/___/ | We are GNU : free your mind & your software | >> _______________________________________________ >> Boxbackup mailing list >> Boxbackup at boxbackup.org >> http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup From achim+box at qustodium.net Sat Jun 9 10:00:31 2012 From: achim+box at qustodium.net (Achim) Date: Sat, 09 Jun 2012 11:00:31 +0200 Subject: [Box Backup] Boxi compile errors Message-ID: Hello list: I get the following compilation errors for Boxi with a fresh checkout from SVN: $ ./make-image-headers.pl $ ./configure-mingw.sh $ make [..] make[2]: Entering directory `/usr/local/src/boxi_svn_20120609/src' [CXX] ComparePanel.cc In file included from ComparePanel.cc:35: ../include/CompareProgressPanel.h: In member function `virtual void CompareProgr essPanel::NotifyLocalDirAccessFailed(const std::string&, const std::string&)': ../include/CompareProgressPanel.h:122: error: `GetNativeErrorMessage' undeclared (first use this function) ../include/CompareProgressPanel.h:122: error: (Each undeclared identifier is rep orted only once for each function it appears in.) make[2]: *** [ComparePanel.o] Error 1 make[2]: Leaving directory `/usr/local/src/boxi_svn_20120609/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/boxi_svn_20120609' make: *** [all] Error 2 Best regards, Achim From achim+box at qustodium.net Sat Jun 9 16:21:57 2012 From: achim+box at qustodium.net (Achim) Date: Sat, 09 Jun 2012 17:21:57 +0200 Subject: [Box Backup] Boxi compile errors In-Reply-To: References: Message-ID: <8b077bb9d1d64a3cfe1d607c0a93eba0@qustodium.net> Hello again: I managed to fix this compile error: On 09.06.2012 11:00, Achim wrote: > make[2]: Entering directory `/usr/local/src/boxi_svn_20120609/src' > [CXX] ComparePanel.cc > In file included from ComparePanel.cc:35: > ../include/CompareProgressPanel.h: In member function `virtual void > CompareProgr > essPanel::NotifyLocalDirAccessFailed(const std::string&, const > std::string&)': > ../include/CompareProgressPanel.h:122: error: `GetNativeErrorMessage' > undeclared > (first use this function) > ../include/CompareProgressPanel.h:122: error: (Each undeclared > identifier is rep > orted only once for each function it appears in.) > make[2]: *** [ComparePanel.o] Error 1 > make[2]: Leaving directory `/usr/local/src/boxi_svn_20120609/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/usr/local/src/boxi_svn_20120609' > make: *** [all] Error 2 by changing line 122 in include/CompareProgressPanel.h like so: // wxString(GetNativeErrorMessage().c_str(), wxString(GetErrorMessage(GetLastError()).c_str(), (appears to be an artefact from R3064 ) Now I get to the following error: [CXX] BackupProgressPanel.cc In file included from ../boxbackup/bin/bbackupd/BackupClientContext.h:17, from BackupProgressPanel.cc:45: ../boxbackup/lib/backupstore/BackupStoreFile.h: In static member function `stati c void* BackupStoreFile::CodingChunkAlloc(int)': ../boxbackup/lib/backupstore/BackupStoreFile.h:167: warning: cast from pointer t o integer of different size BackupProgressPanel.cc: In member function `virtual void BackupProgressPanel::No tifyIDMapsSetup(BackupClientContext&)': BackupProgressPanel.cc:217: error: invalid initialization of non-const reference of type 'std::list >&' from a temporary of type 'std::list >' make: *** [BackupProgressPanel.o] Error 1 so I removed a trailing '&' from line 217 in BackupProgressPanel.cc, and compile continues. Next error is: [CXX] TestWithServer.cc In file included from ../boxbackup/bin/bbackupd/BackupClientContext.h:17, from ../boxbackup/bin/bbackupd/BackupDaemon.h:17, from ../include/RestoreProgressPanel.h:32, from TestWithServer.cc:69: ../boxbackup/lib/backupstore/BackupStoreFile.h: In static member function `stati c void* BackupStoreFile::CodingChunkAlloc(int)': ../boxbackup/lib/backupstore/BackupStoreFile.h:167: warning: cast from pointer t o integer of different size TestWithServer.cc: In member function `virtual void TestBackupStoreDaemon::Conne ction(SocketStreamTLS&)': TestWithServer.cc:139: error: no matching function for call to `BackupStoreConte xt::BackupStoreContext(int32_t&, TestBackupStoreDaemon&)' ../boxbackup/lib/backupstore/BackupStoreContext.h:52: note: candidates are: Back upStoreContext::BackupStoreContext(const BackupStoreContext&) ../boxbackup/lib/backupstore/BackupStoreContext.h:49: note: Back upStoreContext::BackupStoreContext(int32_t, HousekeepingInterface&, const std::s tring&) make: *** [TestWithServer.o] Error 1 To fix this, I added a third parameter "TEST" to TestWithServer.cc:139 in the call to BackupStoreContext. This finally results in a compiled boxi.exe. Perhaps Chris could update the Boxi code to reflect this? At the same time, I see the follwing warning several times, but it does not appear to have an impact: ../boxbackup/lib/backupstore/BackupStoreFile.h: In static member function `stati c void* BackupStoreFile::CodingChunkAlloc(int)': ../boxbackup/lib/backupstore/BackupStoreFile.h:167: warning: cast from pointer t o integer of different size Best regards, Achim From misc.lists at fsck.ch Sun Jun 10 10:50:24 2012 From: misc.lists at fsck.ch (Toby) Date: Sun, 10 Jun 2012 11:50:24 +0200 Subject: [Box Backup] boxbackup bahaviour when renaming directories In-Reply-To: References: <4FD0F86D.1020600@fsck.ch> Message-ID: <4FD46DE0.2040403@fsck.ch> On 6/7/12 9:37 PM, Chris Wilson wrote: > Hi Toby, > > On Thu, 7 Jun 2012, Toby wrote: > >> I recently renamed a bunch of directories on my machine running >> bbackupd. I was hopeing (but not counting on it) that the files >> contained in those directories would not have to be re-uploaded. >> >> Now when I checked with bbackupquery, I noticed that both the old and >> the new directories where there, both still contained files, and >> nothing was marked deleted. The new directory and the new files had >> different IDs, so I guess they were reuploaded. >> >> Now why wheren't the old directories marked deleted? Does this take >> some time? > > Good question. Please could you let us know what version of Box Backup > you're running, and run bbackupd with the "-o /tmp/bbackupd.log -Otrace" > and send the logfile to me privately, or at least the part that relates > to the directory that you deleted? There is no -o and no -O option in my version: Box Backup Client v0.11rc8+2714, (c) Ben Summers and contributors 2003-2010 Usage: bbackupd [options] [config file] Options: -c Use the specified configuration file. If -c is omitted, the last argument is the configuration file, or else the default [/etc/boxbackup/bbackupd.conf] -D Debugging mode, do not fork, one process only, one client only -F Do not fork into background, but fork to serve multiple clients -k Keep console open after fork, keep writing log messages to it -K Stop writing log messages to console while daemon is running -P Show process ID (PID) in console output -q Run more quietly, reduce verbosity level by one, can repeat -Q Run at minimum verbosity, log nothing -v Run more verbosely, increase verbosity level by one, can repeat -V Run at maximum verbosity, log everything -W Set verbosity to error/warning/notice/info/trace/everything -t Tag console output with specified marker -T Timestamp console output -U Timestamp console output with microseconds From dave at bdisystems.co.uk Sun Jun 10 15:24:51 2012 From: dave at bdisystems.co.uk (dave bamford) Date: Sun, 10 Jun 2012 15:24:51 +0100 Subject: [Box Backup] No housekeeping? In-Reply-To: <1327339595.6225.44.camel@crusty.backed-up.net> References: <1327339595.6225.44.camel@crusty.backed-up.net> Message-ID: <1339338291.3720.42.camel@crusty.backed-up.net> Housekeeping has crashed again but this time I had the -V switch on Here are the results... Hope they mean something. Regards Dave > Jun 10 12:38:24 bart bbstored/hk[11295]: ERROR: Failed to rename file: /backups/box/backup/00001000/28/03/o4a.rfwX to /backups/box/backup/00001000/28/03/o4a.rfw: No such file or directory (2) > Jun 10 12:38:24 bart bbstored/hk[11295]: Obtained 10 stack frames. > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 0: bbstored(DumpStackBacktrace()+0x26) [0x4dbff6] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 1: bbstored(RaidFileWrite::Commit(bool)+0x503) [0x46f4d3] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 2: bbstored(HousekeepStoreAccount::DeleteFile(long, long, BackupStoreDirectory&, std::string const&, long)+0x106) [0x43c9a6] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 3: bbstored(HousekeepStoreAccount::DeleteFiles()+0x114) [0x43e5a4] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 4: bbstored(HousekeepStoreAccount::DoHousekeeping(bool)+0x632) [0x43f122] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 5: bbstored(BackupStoreDaemon::RunHousekeepingIfNeeded()+0x446) [0x424f56] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 6: bbstored(BackupStoreDaemon::HousekeepingProcess()+0xaf) [0x425daf] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 7: bbstored(BackupStoreDaemon::Run()+0x278) [0x435d18] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 8: bbstored(Daemon::Main(std::string const&)+0x1582) [0x499c22] > Jun 10 12:38:24 bart bbstored/hk[11295]: Stack frame 9: bbstored(Daemon::Main(char const*, int, char const**)+0x24c) [0x4969cc] > Jun 10 12:38:24 bart bbstored/hk[11295]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(325) > Jun 10 12:38:25 bart bbstored/hk[11295]: ERROR: Failed to delete file: /backups/box/backup/00001000/28/03/o4a.rfwX: No such file or directory (2) > Jun 10 12:38:25 bart bbstored/hk[11295]: Obtained 10 stack frames. > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 0: bbstored(DumpStackBacktrace()+0x26) [0x4dbff6] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 1: bbstored(RaidFileWrite::Discard()+0x338) [0x46af78] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 2: bbstored(RaidFileWrite::~RaidFileWrite()+0x1a) [0x46b8da] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 3: bbstored(HousekeepStoreAccount::DeleteFile(long, long, BackupStoreDirectory&, std::string const&, long)+0xd75) [0x43d615] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 4: bbstored(HousekeepStoreAccount::DeleteFiles()+0x114) [0x43e5a4] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 5: bbstored(HousekeepStoreAccount::DoHousekeeping(bool)+0x632) [0x43f122] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 6: bbstored(BackupStoreDaemon::RunHousekeepingIfNeeded()+0x446) [0x424f56] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 7: bbstored(BackupStoreDaemon::HousekeepingProcess()+0xaf) [0x425daf] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 8: bbstored(BackupStoreDaemon::Run()+0x278) [0x435d18] > Jun 10 12:38:25 bart bbstored/hk[11295]: Stack frame 9: bbstored(Daemon::Main(std::string const&)+0x1582) [0x499c22] > Jun 10 12:38:25 bart bbstored/hk[11295]: WARNING: Exception thrown: RaidFileException(OSError) at RaidFileWrite.cpp(380) On Mon, 2012-01-23 at 17:26 +0000, dave bamford wrote: > Hi Peter > > I had a similar problem and it turned out Housekeeping was aborting on a > corrupt account and never restarting until I restarted bbstored. > Try starting bbstored -v and check the logs > > Regards > > Dave Bamford > > On Mon, 2012-01-23 at 17:43 +0100, Peter Hall wrote: > > Hi fellow boxbackup users, > > > > > > My store is full, and it seems housekeeping isn't running! > > > > > > # bbstoreaccounts info 1 > > Account ID: 0x00000001 > > Last object ID: 0x1106496 > > Used: 243199923 blocks, 927.73 GB, 99% | > > *************** | > > Old files: 3849882 blocks, 14.69 GB, 1% | > > | > > Deleted files: 127698533 blocks, 487.13 GB, 52% |******** > > | > > Directories: 451091 blocks, 1.72 GB, 0% | > > | > > Soft limit: 230400000 blocks, 878.91 GB, 94% | > > *************** | > > Hard limit: 243200000 blocks, 927.73 GB, 100% | > > ****************| > > Client store marker: 17851542 > > > > > > Plenty of deleted files to remove and make space for new ones, but > > it's been like this for days now. > > > > > > I've tried restarting the server, and drastically lowering the > > housekeeping limit: > > # grep -i keep /etc/boxbackup/bbstored.conf > > TimeBetweenHousekeeping = 120 > > > > > > But still no cleaning of the old files. > > > > > > All the references to housekeeping I can find in the logs are: > > client=0x00000001[29150]: WARNING: Reference count database is missing > > or corrupted, creating a new one, expect housekeeping to find and fix > > problems with reference counts later. > > > > > > I have run a fix of the account with "bbstoreaccounts check 1 fix". It > > found and fixed three errors on the first run, subsequent runs find no > > errors. > > > > > > bbstored seems to be doing something though, it takes around 45% > > memory and 50% cpu if I consult 'top'. > > > > > > Is it normal that it can take several days before I see any > > housekeeping progress on a store of this size? > > > > > > Thanks in advance, > > Peter > > _______________________________________________ > > Boxbackup mailing list > > Boxbackup at boxbackup.org > > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup > > > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup From chris at qwirx.com Mon Jun 11 22:16:26 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 11 Jun 2012 22:16:26 +0100 (BST) Subject: [Box Backup] No housekeeping? In-Reply-To: <1339338291.3720.42.camel@crusty.backed-up.net> References: <1327339595.6225.44.camel@crusty.backed-up.net> <1339338291.3720.42.camel@crusty.backed-up.net> Message-ID: Hi Dave, On Sun, 10 Jun 2012, dave bamford wrote: > Housekeeping has crashed again but this time I had the -V switch on > > Here are the results... > Hope they mean something. Yes, thanks! And thanks for your patience! This helped me to catch a double fault condition that I wasn't aware of, which is most likely the cause of housekeeping terminating. I think I have fixed this double fault in Subversion. Please could you test? The root cause remains unfixed, because I don't understand how the raidfile /backups/box/backup/00001000/28/03/o4a.rfwX, which we previously opened and locked, in an account that we have locked for housekeeping, no longer exists (according to the error message) when it comes to commit our changes. Could anything else be modifying the directory where this account is stored? Does it have an unusual filesystem? What do you have in the /backups/box/backup/00001000/28/03 directory? Can you send me privately the complete log from bbstored housekeeping on this account? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From chris at qwirx.com Mon Jun 11 23:01:47 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 11 Jun 2012 23:01:47 +0100 (BST) Subject: [Box Backup] Boxi compile errors In-Reply-To: <8b077bb9d1d64a3cfe1d607c0a93eba0@qustodium.net> References: <8b077bb9d1d64a3cfe1d607c0a93eba0@qustodium.net> Message-ID: Hi Achim, On Sat, 9 Jun 2012, Achim wrote: > This finally results in a compiled boxi.exe. Perhaps Chris could update > the Boxi code to reflect this? Thanks, I think your fixes are all correct, I've committed them. > At the same time, I see the follwing warning several times, but it does not > appear to have an impact: > > ../boxbackup/lib/backupstore/BackupStoreFile.h: In static member function > `stati > c void* BackupStoreFile::CodingChunkAlloc(int)': > ../boxbackup/lib/backupstore/BackupStoreFile.h:167: warning: cast from > pointer to integer of different size Are you compiling on a 64-bit machine by any chance? I think these are some of the 64-bit problems that Charles spotted and hopefully fixed in his branch, but which are not merged back into trunk yet. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From dave at bdisystems.co.uk Mon Jun 11 23:12:44 2012 From: dave at bdisystems.co.uk (dave bamford) Date: Mon, 11 Jun 2012 23:12:44 +0100 Subject: [Box Backup] No housekeeping? In-Reply-To: References: <1327339595.6225.44.camel@crusty.backed-up.net> <1339338291.3720.42.camel@crusty.backed-up.net> Message-ID: <1339452764.3720.55.camel@crusty.backed-up.net> Hi Chris I will test this over the next few days. I am wondering if the bbstoreaccounts check removes this file. The store is mounted NFS using nfs4 /home/backups 10.100.100.0/24:(rw,sync,fsid=0,no_root_squash,no_subtree_check) The store filesystem is xfs. The only other thing going on is a rsync between this store and a backup store. Ideally it would be nice to write to both stores simultaneously, but I don't know if this is possible with box. Here are the contents of the directory ls -al /home/backups/box/backup/00001000/28/03 total 44 drwxr-xr-x 2 _bbstored _bbstored 20 Jun 10 18:03 . drwxr-xr-x 9 _bbstored _bbstored 8192 Jun 3 17:10 .. -rw-r--r-- 1 _bbstored _bbstored 31505 Jun 10 12:38 o4a.rfw Will send the log privately. Thanks Dave On Mon, 2012-06-11 at 22:16 +0100, Chris Wilson wrote: > Hi Dave, > > On Sun, 10 Jun 2012, dave bamford wrote: > > > Housekeeping has crashed again but this time I had the -V switch on > > > > Here are the results... > > Hope they mean something. > > Yes, thanks! And thanks for your patience! > > This helped me to catch a double fault condition that I wasn't aware of, > which is most likely the cause of housekeeping terminating. > > I think I have fixed this double fault in Subversion. Please could you > test? > > The root cause remains unfixed, because I don't understand how the > raidfile /backups/box/backup/00001000/28/03/o4a.rfwX, which we previously > opened and locked, in an account that we have locked for housekeeping, no > longer exists (according to the error message) when it comes to commit our > changes. > > Could anything else be modifying the directory where this account is > stored? Does it have an unusual filesystem? What do you have in the > /backups/box/backup/00001000/28/03 directory? Can you send me privately > the complete log from bbstored housekeeping on this account? > > Cheers, Chris. From chris at qwirx.com Mon Jun 11 23:19:46 2012 From: chris at qwirx.com (Chris Wilson) Date: Mon, 11 Jun 2012 23:19:46 +0100 (BST) Subject: [Box Backup] No housekeeping? In-Reply-To: <1339452764.3720.55.camel@crusty.backed-up.net> References: <1327339595.6225.44.camel@crusty.backed-up.net> <1339338291.3720.42.camel@crusty.backed-up.net> <1339452764.3720.55.camel@crusty.backed-up.net> Message-ID: Hi Dave, On Mon, 11 Jun 2012, dave bamford wrote: > I am wondering if the bbstoreaccounts check removes this file. If you run it while housekeeping is also running on the same account, then it might. The locking should prevent that, but perhaps it's broken somehow. > The store is mounted NFS using nfs4 /home/backups > 10.100.100.0/24:(rw,sync,fsid=0,no_root_squash,no_subtree_check) The > store filesystem is xfs. > > The only other thing going on is a rsync between this store and a backup > store. Which side is the read-only side of the rsync? Is it the NFS server? So the rsync shouldn't be modifying files on this side. It might be NFS, though. I've seen weird things happen with some recent Linux kernels as NFS clients. What OS is running on the NFS server? > Ideally it would be nice to write to both stores simultaneously, > but I don't know if this is possible with box. Not exactly. You can backup to two independent stores simultaneously, but of course you use twice as much bandwidth by doing that. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From dave at bdisystems.co.uk Tue Jun 12 00:08:18 2012 From: dave at bdisystems.co.uk (Dave Bamford) Date: Tue, 12 Jun 2012 00:08:18 +0100 Subject: [Box Backup] No housekeeping? In-Reply-To: References: <1327339595.6225.44.camel@crusty.backed-up.net> <1339338291.3720.42.camel@crusty.backed-up.net> <1339452764.3720.55.camel@crusty.backed-up.net> Message-ID: <6E0EF39F-F8D0-4DCD-B1A2-FAE55F60D591@bdisystems.co.uk> Hi Chris Is there a way to make housekeeping just run on a specific account? It takes quite a while for housekeeping to get round all the store currently occupies about 3.5Tb Over about 20 accounts. Rsync is reading the active store and writing to the inactive store. Both stores are physical machines running Debian Squeeze. The bbstored server is running on a VM using a LVM image with KVM as the hyper visor, on a separate physical machine. The VM has a real world network port and an internal network port for the stores, bridging to 2 physical network cards, both running at 1gb. All are running Squeeze. I was thinking that it may be possible to configure the raid file facility in box to write to two stores simultaneously. They are just NFS mounted file systems. Regards Dave Sent from my iPad On 11 Jun 2012, at 23:19, Chris Wilson wrote: > Hi Dave, > > On Mon, 11 Jun 2012, dave bamford wrote: > >> I am wondering if the bbstoreaccounts check removes this file. > > If you run it while housekeeping is also running on the same account, then it might. The locking should prevent that, but perhaps it's broken somehow. > >> The store is mounted NFS using nfs4 /home/backups 10.100.100.0/24:(rw,sync,fsid=0,no_root_squash,no_subtree_check) The store filesystem is xfs. >> >> The only other thing going on is a rsync between this store and a backup >> store. > > Which side is the read-only side of the rsync? Is it the NFS server? So the rsync shouldn't be modifying files on this side. It might be NFS, though. I've seen weird things happen with some recent Linux kernels as NFS clients. What OS is running on the NFS server? > >> Ideally it would be nice to write to both stores simultaneously, >> but I don't know if this is possible with box. > > Not exactly. You can backup to two independent stores simultaneously, but of course you use twice as much bandwidth by doing that. > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/cgi-bin/mailman/listinfo/boxbackup From achim+box at qustodium.net Wed Jun 13 11:52:51 2012 From: achim+box at qustodium.net (Achim J. Latz) Date: Wed, 13 Jun 2012 12:52:51 +0200 Subject: [Box Backup] Boxi compile errors In-Reply-To: References: <8b077bb9d1d64a3cfe1d607c0a93eba0@qustodium.net> Message-ID: <4FD87103.8040707@qustodium.net> Hello Chris: On 12/06/2012 00:01, Chris Wilson wrote: > > At the same time, I see the follwing warning several times, but it does not appear to have an impact: >> >> ../boxbackup/lib/backupstore/BackupStoreFile.h: In static member function `stati >> c void* BackupStoreFile::CodingChunkAlloc(int)': >> ../boxbackup/lib/backupstore/BackupStoreFile.h:167: warning: cast from pointer to integer of different size > Are you compiling on a 64-bit machine by any chance? I think these are > some of the 64-bit problems that Charles spotted and hopefully fixed in > his branch, but which are not merged back into trunk yet. For now this is a fresh SVN checkout on Cygwin/MinGW on 32bit windows XP: nothing that did not work in the past... 64bit compatibility would be nice in any case ;-) Cheers, Achim -- Achim J. Latz, Qustodium Internet Security achim.latz at qustodium.net ? http://www.qustodium.net Data Encryption ? Backup Automatisation ? E-Mail Protection From chris at qwirx.com Fri Jun 29 23:44:13 2012 From: chris at qwirx.com (Chris Wilson) Date: Fri, 29 Jun 2012 23:44:13 +0100 (BST) Subject: [Box Backup] No housekeeping? In-Reply-To: <6E0EF39F-F8D0-4DCD-B1A2-FAE55F60D591@bdisystems.co.uk> References: <1327339595.6225.44.camel@crusty.backed-up.net> <1339338291.3720.42.camel@crusty.backed-up.net> <1339452764.3720.55.camel@crusty.backed-up.net> <6E0EF39F-F8D0-4DCD-B1A2-FAE55F60D591@bdisystems.co.uk> Message-ID: Hi Dave, On Tue, 12 Jun 2012, Dave Bamford wrote: > Is there a way to make housekeeping just run on a specific account? > It takes quite a while for housekeeping to get round all the store currently occupies about 3.5Tb Sorry for the long delay. I finally got some programming time to implement this option. You can now use the bbstoreaccounts "housekeep" command to run housekeeping even if bbstored is not running, provided that the account is not locked. > Rsync is reading the active store and writing to the inactive store. > Both stores are physical machines running Debian Squeeze. The bbstored > server is running on a VM using a LVM image with KVM as the hyper visor, > on a separate physical machine. The VM has a real world network port and > an internal network port for the stores, bridging to 2 physical network > cards, both running at 1gb. All are running Squeeze. I'm kind of running out of ideas, but I can think of a few things you can try. You could try running bbstoreaccounts housekeep on both bbstored servers simultaneously. The locking should prevent the second one from starting. If it doesn't, then the locking is broken on NFS and we need to investigate that. Is bbstored actually running on both servers? If so, could you try stopping it on the inactive one to see if that makes the problem go away? Is the error only happening for one client? If so, is there anything different about that client? Is it configured to load balance between the two store servers? Is its account much larger than the others? Is it possible to eliminate NFS temporarily, for example by running bbstored on the NFS server, or moving this account to a local disk or iSCSI mount on the active bbstored server? > I was thinking that it may be possible to configure the raid file > facility in box to write to two stores simultaneously. They are just NFS > mounted file systems. The built-in software raid facility requires three servers for RAID over NFS, so I'm afraid it won't work with two. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From chris at qwirx.com Fri Jun 29 23:48:52 2012 From: chris at qwirx.com (Chris Wilson) Date: Fri, 29 Jun 2012 23:48:52 +0100 (BST) Subject: [Box Backup] boxbackup bahaviour when renaming directories In-Reply-To: <4FD46DE0.2040403@fsck.ch> References: <4FD0F86D.1020600@fsck.ch> <4FD46DE0.2040403@fsck.ch> Message-ID: Hi Toby, On Sun, 10 Jun 2012, Toby wrote: > On 6/7/12 9:37 PM, Chris Wilson wrote: >> On Thu, 7 Jun 2012, Toby wrote: >> >>> I recently renamed a bunch of directories on my machine running >>> bbackupd. I was hopeing (but not counting on it) that the files >>> contained in those directories would not have to be re-uploaded. >>> >>> Now when I checked with bbackupquery, I noticed that both the old and >>> the new directories where there, both still contained files, and >>> nothing was marked deleted. The new directory and the new files had >>> different IDs, so I guess they were reuploaded. >>> >>> Now why wheren't the old directories marked deleted? Does this take >>> some time? >> >> Good question. Please could you let us know what version of Box Backup >> you're running, and run bbackupd with the "-o /tmp/bbackupd.log -Otrace" >> and send the logfile to me privately, or at least the part that relates >> to the directory that you deleted? > > There is no -o and no -O option in my version: > > Box Backup Client v0.11rc8+2714, (c) Ben Summers and contributors 2003-2010 Sorry, that feature was introduced in r2806. You can still log to a file by running bbackupd with the -V option and redirecting the output to a file. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software |