From boxbackup at fluffy.co.uk Fri Feb 1 00:06:04 2008 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Fri, 01 Feb 2008 00:06:04 +0000 Subject: [Box Backup] Announcing 0.11 Release Candidate 2 In-Reply-To: References: <479F8A82.9030703@guno.nl> <47A0D5FC.7000606@guno.nl> <47A1917C.1010504@hickinbottom.com> Message-ID: <47A2626C.8050304@hickinbottom.com> This is a multi-part message in MIME format. --------------090406040500070209050304 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit OK. Attached is a small patch to add a little helpful output if the tests fail. Stuart Chris Wilson wrote: > Hi Stuart, > > On Thu, 31 Jan 2008, Stuart Hickinbottom wrote: > > >> It's easy to forget - I've done it myself more than once. I wonder >> whether, if any tests fail, we could get a reminder printed to check the >> common things: >> >> 1. Check that no other instances of bbstored or bbackupd are running >> on this machine. >> 2. That there isn't a firewall blocking the incoming or outgoing >> ports that are used. >> 3. That there is sufficient space in the filesystem the tests are >> being run from. >> >> I think such a reminder would help avoid the common questions here. >> >> Chris, if it would help I could knock up a patch for that, and add a >> couple of notes to the build/test section in the wiki. >> > > Please go ahead, thanks! > > Chris. > --------------090406040500070209050304 Content-Type: text/plain; name="failure-text.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="failure-text.patch" --- runtest.pl.in.orig 2008-01-31 23:18:58.000000000 +0000 +++ runtest.pl.in 2008-01-31 23:17:58.000000000 +0000 @@ -71,6 +71,23 @@ # report results print "--------\n",join("\n", at results),"\n"; +if ($exit_code != 0) +{ + print <<__E; + +One or more tests have failed. Common things to check that may cause tests +to incorrectly fail are: + +* Check that no instances of bbstored or bbackupd are already running + on this machine. +* Make sure there isn't a firewall blocking the incoming or outgoing + ports that are used by bbstored and bbackupd. +* Check that there is sufficient space in the filesystem that the tests + are being run from. + +__E +} + exit $exit_code; sub runtest --------------090406040500070209050304-- From boxbackup at fluffy.co.uk Fri Feb 1 00:54:41 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 1 Feb 2008 00:54:41 +0000 (GMT) Subject: [Box Backup] Announcing 0.11 Release Candidate 2 In-Reply-To: <47A2626C.8050304@hickinbottom.com> References: <479F8A82.9030703@guno.nl> <47A0D5FC.7000606@guno.nl> <47A1917C.1010504@hickinbottom.com> <47A2626C.8050304@hickinbottom.com> Message-ID: Hi Stuart, On Fri, 1 Feb 2008, Stuart Hickinbottom wrote: > OK. Attached is a small patch to add a little helpful output if the tests > fail. Thanks! Applied with minor changes to wording. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 1 04:30:29 2008 From: boxbackup at fluffy.co.uk (Alex Harper) Date: Thu, 31 Jan 2008 20:30:29 -0800 Subject: [Box Backup] Announcing 0.11 Release Candidate 2 In-Reply-To: Message-ID: Hi Chris, > Looks like your box is too fast for me :-) I can't reproduce it but I > think I know where the problem is, and I checked in a blind fix. > > Please could you test the latest trunk to see if I've fixed the problem > successfully? Seems like your guess was correct, trunk is now passing all tests for me. I gather it was a timing issue? FWIW before I saw your response I had gone ahead and run tests on OS X x86 (Xeon, Core 2) and PPC (G5) on 10.4.11 and 10.5.1. Tests were consistently passing on PPC and older x86, but failures were consistent on the newer x86 machines. Thanks for the fast fix, Alex -- Alex Harper aharper at foobox.net "Generally speaking, things have gone about as far as they can reasonably go when things have got about as bad as they can reasonably get." - Tom Stoppard From boxbackup at fluffy.co.uk Fri Feb 1 09:14:45 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 1 Feb 2008 09:14:45 +0000 (GMT) Subject: [Box Backup] Announcing 0.11 Release Candidate 2 In-Reply-To: References: Message-ID: Hi Alex, On Thu, 31 Jan 2008, Alex Harper wrote: > > Please could you test the latest trunk to see if I've fixed the > > problem successfully? > > Seems like your guess was correct, trunk is now passing all tests for > me. Great, thanks! > I gather it was a timing issue? Yes, timing/race. A cache invalidated by timestamps on files, a filesystem that only stores timestamps with 1 second resolution and a machine fast enough to change the files and access the directory again within the same second, add up to this bug. > FWIW before I saw your response I had gone ahead and run tests on OS X > x86 (Xeon, Core 2) and PPC (G5) on 10.4.11 and 10.5.1. Tests were > consistently passing on PPC and older x86, but failures were consistent > on the newer x86 machines. I only tested on PPC macs because that's all I have access to. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 1 11:10:11 2008 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Fri, 01 Feb 2008 12:10:11 +0100 Subject: [Box Backup] Boxbackup 0.11rc2 - Apple PPC G4 1400 Mhz - Debian Etch (Stable) Message-ID: <47A2FE13.50902@kontrapunkt.com> Hello... Just compiled 0.11rc2 on an Apple PPC G4 1400 Mhz running Debian Etch (Stable). All tests passed as seen below. common: PASSED crypto: PASSED compress: PASSED raidfile: PASSED basicserver: PASSED backupstore: PASSED backupstorefix: PASSED backupstorepatch: PASSED backupdiff: PASSED bbackupd: PASSED Thanks, Tobias From boxbackup at fluffy.co.uk Sat Feb 2 02:45:14 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Sat, 2 Feb 2008 02:45:14 +0000 Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail Message-ID: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> Hi Chris, I thought I would try compiling RC2 on my Mac Book Pro - Core2Duo running OSX 10.5 (Leopard) -------- common: PASSED crypto: PASSED compress: PASSED raidfile: PASSED basicserver: PASSED backupstore: FAILED: 5 tests failed (first at testbackupstore.cpp:1201) backupstorefix: PASSED backupstorepatch: PASSED backupdiff: PASSED bbackupd: PASSED make: *** [release/common/test] Error 1 I know you have only been working on PPC, so im not sure its meant to work. Thought you would like the feedback anyhow :-) Regards Matt Brown From boxbackup at fluffy.co.uk Sat Feb 2 08:54:45 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sat, 2 Feb 2008 08:54:45 +0000 (GMT) Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail In-Reply-To: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> References: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> Message-ID: Hi Matt, On Sat, 2 Feb 2008, Matt Brown wrote: > I thought I would try compiling RC2 on my Mac Book Pro - Core2Duo > running OSX 10.5 (Leopard) > > -------- > common: PASSED > crypto: PASSED > compress: PASSED > raidfile: PASSED > basicserver: PASSED > backupstore: FAILED: 5 tests failed (first at testbackupstore.cpp:1201) > backupstorefix: PASSED > backupstorepatch: PASSED > backupdiff: PASSED > bbackupd: PASSED > make: *** [release/common/test] Error 1 > > I know you have only been working on PPC, so im not sure its meant to > work. Some people are having success running on Intel Macs. Please could you send me the test-backupstore.log file? > Thought you would like the feedback anyhow :-) Yes, thanks! Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Sat Feb 2 13:24:31 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sat, 2 Feb 2008 13:24:31 +0000 Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail In-Reply-To: References: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> Message-ID: <20080202132430.GC15350@netinertia.co.uk> On Sat, Feb 02, 2008 at 08:54:45AM +0000, Chris Wilson wrote: > Some people are having success running on Intel Macs. Please could you > send me the test-backupstore.log file? Trunk passes all tests (debug and release) on my Intel iMac running Leopard. James From boxbackup at fluffy.co.uk Sat Feb 2 22:41:34 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Sat, 2 Feb 2008 22:41:34 +0000 Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail In-Reply-To: <20080202132430.GC15350@netinertia.co.uk> References: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> <20080202132430.GC15350@netinertia.co.uk> Message-ID: > On Sat, Feb 02, 2008 at 08:54:45AM +0000, Chris Wilson wrote: >> Some people are having success running on Intel Macs. Please could >> you >> send me the test-backupstore.log file? > > Trunk passes all tests (debug and release) on my Intel iMac running > Leopard. These were what appeared to fail.... FAILURE: Condition [dir.GetNumberOfEntries() == UPLOAD_NUM + 3] failed at testbackupstore.cpp(1201) FAILURE: Condition [en->GetName() == dirname] failed at testbackupstore.cpp(1218) FAILURE: Condition [en->GetFlags() == BackupProtocolClientListDirectory::Flags_Dir] failed at testbackupstore.cpp(1219) FAILURE: Condition [en->GetObjectID() == subdirid] failed at testbackupstore.cpp(1220) FAILURE: Condition [en->GetModificationTime() == 0] failed at testbackupstore.cpp(1221) (Chris I will email the log) Regards Matt From boxbackup at fluffy.co.uk Sun Feb 3 13:16:46 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 3 Feb 2008 13:16:46 +0000 (GMT) Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail In-Reply-To: References: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> <20080202132430.GC15350@netinertia.co.uk> Message-ID: Hi Matt, On Sat, 2 Feb 2008, Matt Brown wrote: > >On Sat, Feb 02, 2008 at 08:54:45AM +0000, Chris Wilson wrote: > > >Some people are having success running on Intel Macs. Please could > > >you send me the test-backupstore.log file? > > > >Trunk passes all tests (debug and release) on my Intel iMac running > >Leopard. > > These were what appeared to fail.... > > FAILURE: Condition [dir.GetNumberOfEntries() == UPLOAD_NUM + 3] failed at > FAILURE: testbackupstore.cpp(1201) > FAILURE: Condition [en->GetName() == dirname] failed at > FAILURE: testbackupstore.cpp(1218) > FAILURE: Condition [en->GetFlags() == > FAILURE: BackupProtocolClientListDirectory::Flags_Dir] failed at > FAILURE: testbackupstore.cpp(1219) > FAILURE: Condition [en->GetObjectID() == subdirid] failed at > FAILURE: testbackupstore.cpp(1220) > FAILURE: Condition [en->GetModificationTime() == 0] failed at > FAILURE: testbackupstore.cpp(1221) I think I fixed that very same bug a few days ago. Please could you try with the latest trunk? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Sun Feb 3 15:11:18 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Sun, 3 Feb 2008 15:11:18 +0000 Subject: [Box Backup] 0.11rc2 - MAC OSX Leopard 10.5 - Tests Fail **RESOLVED** In-Reply-To: References: <235A6E45-03E2-4FA0-B7C4-F7B0600606FB@mbrown.co.uk> <20080202132430.GC15350@netinertia.co.uk> Message-ID: <83F7800D-32A7-4D3E-9483-B4311FDA0993@mbrown.co.uk> Hi Chris, > > I think I fixed that very same bug a few days ago. Please could you > try > with the latest trunk? Sorry must of missed that one .. Checked out revision 2087. -------- common: PASSED crypto: PASSED compress: PASSED raidfile: PASSED basicserver: PASSED backupstore: PASSED backupstorefix: PASSED backupstorepatch: PASSED backupdiff: PASSED bbackupd: PASSED make: Nothing to be done for `release'. make: Nothing to be done for `all'. All appears to be fine now, thanks. Regards Matt From boxbackup at fluffy.co.uk Wed Feb 6 04:22:06 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Wed, 6 Feb 2008 15:22:06 +1100 Subject: [Box Backup] Boxbackup on Ubuntu Message-ID: Hello list, I'm having some problems getting boxbackup to run. Starting the program, either manually, or by the startup script prints no messages at all. Using ps to examine the running processes shows nothing. It's very frustrating! I'm using the package provided by Ubuntu for my release (Gutsy 7.10), which is .10. I followed the excellent tutorial by xkr47 here: http://xkr47.outerspace.dyndns.org/howtos/boxbackup/ I've read the ConfigureAServer page on the wiki too. I'm sure I'm doing something silly. Is there any way to turn on logging, or make it print some debugging messages before it quietly fails? Is it worth upgrading to .11rc2? I don't mind building from source. Thanks, Timothy. From boxbackup at fluffy.co.uk Wed Feb 6 09:24:04 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 6 Feb 2008 09:24:04 +0000 (GMT) Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: Hi Timothy, On Wed, 6 Feb 2008, Timothy Wilson wrote: > I'm having some problems getting boxbackup to run. Starting the program, > either manually, or by the startup script prints no messages at all. > Using ps to examine the running processes shows nothing. It's very > frustrating! Try running "bbackupd SINGLEPROCESS". That should run in the foreground and print messages. Please also check your system logs. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Wed Feb 6 09:42:39 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Wed, 6 Feb 2008 20:42:39 +1100 Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: Ok thanks for that suggestion Chris, I'll give that a shot. On 06/02/2008, Chris Wilson wrote: > Hi Timothy, > > On Wed, 6 Feb 2008, Timothy Wilson wrote: > > > I'm having some problems getting boxbackup to run. Starting the program, > > either manually, or by the startup script prints no messages at all. > > Using ps to examine the running processes shows nothing. It's very > > frustrating! > > Try running "bbackupd SINGLEPROCESS". That should run in the foreground > and print messages. Please also check your system logs. > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed Feb 6 12:13:01 2008 From: boxbackup at fluffy.co.uk (Michiel van Baak) Date: Wed, 6 Feb 2008 13:13:01 +0100 Subject: [Box Backup] 0.11 ETA Message-ID: <20080206121300.GD16403@vanbaak.info> Hey. With all the possitive test reports out there is there any ETA on the final 0.11 release ? -- Michiel van Baak michiel at vanbaak.eu http://michiel.vanbaak.eu GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x71C946BD "Why is it drug addicts and computer aficionados are both called users?" From boxbackup at fluffy.co.uk Wed Feb 6 13:48:09 2008 From: boxbackup at fluffy.co.uk (Mikael Syska) Date: Wed, 06 Feb 2008 14:48:09 +0100 Subject: [Box Backup] 0.11 ETA In-Reply-To: <20080206121300.GD16403@vanbaak.info> References: <20080206121300.GD16403@vanbaak.info> Message-ID: <47A9BA99.40301@syska.dk> Think Chris Wilson said sometime around 27 Feb ... if all goes well. // ouT Michiel van Baak wrote: > Hey. > > With all the possitive test reports out there is there any > ETA on the final 0.11 release ? > > From boxbackup at fluffy.co.uk Wed Feb 6 14:11:16 2008 From: boxbackup at fluffy.co.uk (Michiel van Baak) Date: Wed, 6 Feb 2008 15:11:16 +0100 Subject: [Box Backup] 0.11 ETA In-Reply-To: <47A9BA99.40301@syska.dk> References: <20080206121300.GD16403@vanbaak.info> <47A9BA99.40301@syska.dk> Message-ID: <20080206141116.GE16403@vanbaak.info> On 14:48, Wed 06 Feb 08, Mikael Syska wrote: > Think Chris Wilson said sometime around 27 Feb ... if all goes well. great. Thanks for the info > > // ouT > > Michiel van Baak wrote: > >Hey. > > > >With all the possitive test reports out there is there any > >ETA on the final 0.11 release ? > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup -- Michiel van Baak michiel at vanbaak.eu http://michiel.vanbaak.eu GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x71C946BD "Why is it drug addicts and computer aficionados are both called users?" From boxbackup at fluffy.co.uk Wed Feb 6 20:20:55 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 6 Feb 2008 20:20:55 +0000 (GMT) Subject: [Box Backup] 0.11 ETA In-Reply-To: <20080206121300.GD16403@vanbaak.info> References: <20080206121300.GD16403@vanbaak.info> Message-ID: Hi Michiel, On Wed, 6 Feb 2008, Michiel van Baak wrote: > With all the possitive test reports out there is there any ETA on the > final 0.11 release ? ETA is currently the date announced along with 0.11rc2, i.e. 28th February. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Thu Feb 7 00:48:42 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 7 Feb 2008 11:48:42 +1100 Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: Sorry, please ignore the last message. This is my problem. Does this mean that my address can't be bound to? Timothy. root at userver:/etc/boxbackup/boxbackup-ca# bbstored SINGLEPROCESS -c /etc/boxbackup/server/bbstored.conf NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/server/bbstored.conf WARNING: Exception thrown: ServerException(ServerStreamBadListenAddrs) at ../../lib/server/ServerStream.h(238) WARNING: Exception thrown: ServerException(SocketCloseError) at ../../lib/server/SocketListen.h(150) FATAL: Terminating due to exception Server SocketCloseError (3/13) root at userver:/etc/boxbackup/boxbackup-ca# NOTICE: Terminating daemon On 07/02/2008, Timothy Wilson wrote: > Hello guys, > I'm getting the following error. Does this mean I haven't signed my > certificate correctly? Also, must I run the client on the server > machine to? I haven't configured bbackupd yet. Is this causing my > problems? > > root at userver:/etc/boxbackup# bbstored SINGLEPROCESS -c > /etc/boxbackup/server/bbstored.conf > NOTICE: Starting daemon, version 0.11rc2, config: > /etc/boxbackup/server/bbstored.conf > ERROR: SSL error during Load CA cert: error:02001002:system > library:fopen:No such file or directory > ERROR: SSL error during Load CA cert: error:2006D080:BIO > routines:BIO_new_file:no such file > ERROR: SSL error during Load CA cert: error:0B084002:x509 certificate > routines:X509_load_cert_crl_file:system lib > WARNING: Exception thrown: ServerException(TLSLoadTrustedCAsFailed) at > TLSContext.cpp(130) > FATAL: Terminating due to exception Server TLSLoadTrustedCAsFailed (3/27) > root at userver:/etc/boxbackup# NOTICE: Terminating daemon > > > On 06/02/2008, Timothy Wilson wrote: > > Ok thanks for that suggestion Chris, > > I'll give that a shot. > > > > On 06/02/2008, Chris Wilson wrote: > > > Hi Timothy, > > > > > > On Wed, 6 Feb 2008, Timothy Wilson wrote: > > > > > > > I'm having some problems getting boxbackup to run. Starting the program, > > > > either manually, or by the startup script prints no messages at all. > > > > Using ps to examine the running processes shows nothing. It's very > > > > frustrating! > > > > > > Try running "bbackupd SINGLEPROCESS". That should run in the foreground > > > and print messages. Please also check your system logs. > > > > > > Cheers, Chris. > > > -- > > > _____ __ _ > > > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > > > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > > > _______________________________________________ > > > boxbackup mailing list > > > boxbackup at fluffy.co.uk > > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > > > > From boxbackup at fluffy.co.uk Thu Feb 7 01:44:51 2008 From: boxbackup at fluffy.co.uk (Pablo Fernandez) Date: Wed, 06 Feb 2008 23:44:51 -0200 Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: <1202348691.19596.1.camel@laptop> Hi Check what you have in ListenAddress, make sure that address appears in ifconfig. Best regards, Pablo On Thu, 2008-02-07 at 11:48 +1100, Timothy Wilson wrote: > Sorry, please ignore the last message. This is my problem. Does this > mean that my address can't be bound to? > > Timothy. > > root at userver:/etc/boxbackup/boxbackup-ca# bbstored SINGLEPROCESS -c > /etc/boxbackup/server/bbstored.conf > NOTICE: Starting daemon, version 0.11rc2, config: > /etc/boxbackup/server/bbstored.conf > WARNING: Exception thrown: ServerException(ServerStreamBadListenAddrs) > at ../../lib/server/ServerStream.h(238) > WARNING: Exception thrown: ServerException(SocketCloseError) at > ../../lib/server/SocketListen.h(150) > FATAL: Terminating due to exception Server SocketCloseError (3/13) > root at userver:/etc/boxbackup/boxbackup-ca# NOTICE: Terminating daemon > > > > > On 07/02/2008, Timothy Wilson wrote: > > Hello guys, > > I'm getting the following error. Does this mean I haven't signed my > > certificate correctly? Also, must I run the client on the server > > machine to? I haven't configured bbackupd yet. Is this causing my > > problems? > > > > root at userver:/etc/boxbackup# bbstored SINGLEPROCESS -c > > /etc/boxbackup/server/bbstored.conf > > NOTICE: Starting daemon, version 0.11rc2, config: > > /etc/boxbackup/server/bbstored.conf > > ERROR: SSL error during Load CA cert: error:02001002:system > > library:fopen:No such file or directory > > ERROR: SSL error during Load CA cert: error:2006D080:BIO > > routines:BIO_new_file:no such file > > ERROR: SSL error during Load CA cert: error:0B084002:x509 certificate > > routines:X509_load_cert_crl_file:system lib > > WARNING: Exception thrown: ServerException(TLSLoadTrustedCAsFailed) at > > TLSContext.cpp(130) > > FATAL: Terminating due to exception Server TLSLoadTrustedCAsFailed (3/27) > > root at userver:/etc/boxbackup# NOTICE: Terminating daemon > > > > > > On 06/02/2008, Timothy Wilson wrote: > > > Ok thanks for that suggestion Chris, > > > I'll give that a shot. > > > > > > On 06/02/2008, Chris Wilson wrote: > > > > Hi Timothy, > > > > > > > > On Wed, 6 Feb 2008, Timothy Wilson wrote: > > > > > > > > > I'm having some problems getting boxbackup to run. Starting the program, > > > > > either manually, or by the startup script prints no messages at all. > > > > > Using ps to examine the running processes shows nothing. It's very > > > > > frustrating! > > > > > > > > Try running "bbackupd SINGLEPROCESS". That should run in the foreground > > > > and print messages. Please also check your system logs. > > > > > > > > Cheers, Chris. > > > > -- > > > > _____ __ _ > > > > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > > > > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > > > > _______________________________________________ > > > > boxbackup mailing list > > > > boxbackup at fluffy.co.uk > > > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > > > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu Feb 7 00:40:00 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 7 Feb 2008 11:40:00 +1100 Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: Hello guys, I'm getting the following error. Does this mean I haven't signed my certificate correctly? Also, must I run the client on the server machine to? I haven't configured bbackupd yet. Is this causing my problems? root at userver:/etc/boxbackup# bbstored SINGLEPROCESS -c /etc/boxbackup/server/bbstored.conf NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/server/bbstored.conf ERROR: SSL error during Load CA cert: error:02001002:system library:fopen:No such file or directory ERROR: SSL error during Load CA cert: error:2006D080:BIO routines:BIO_new_file:no such file ERROR: SSL error during Load CA cert: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib WARNING: Exception thrown: ServerException(TLSLoadTrustedCAsFailed) at TLSContext.cpp(130) FATAL: Terminating due to exception Server TLSLoadTrustedCAsFailed (3/27) root at userver:/etc/boxbackup# NOTICE: Terminating daemon On 06/02/2008, Timothy Wilson wrote: > Ok thanks for that suggestion Chris, > I'll give that a shot. > > On 06/02/2008, Chris Wilson wrote: > > Hi Timothy, > > > > On Wed, 6 Feb 2008, Timothy Wilson wrote: > > > > > I'm having some problems getting boxbackup to run. Starting the program, > > > either manually, or by the startup script prints no messages at all. > > > Using ps to examine the running processes shows nothing. It's very > > > frustrating! > > > > Try running "bbackupd SINGLEPROCESS". That should run in the foreground > > and print messages. Please also check your system logs. > > > > Cheers, Chris. > > -- > > _____ __ _ > > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > From boxbackup at fluffy.co.uk Thu Feb 7 17:36:09 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 7 Feb 2008 17:36:09 +0000 (GMT) Subject: [Box Backup] Boxbackup on Ubuntu In-Reply-To: References: Message-ID: Hi Timothy, On Thu, 7 Feb 2008, Timothy Wilson wrote: > I'm getting the following error. Does this mean I haven't signed my > certificate correctly? Also, must I run the client on the server > machine to? No, normally you wouldn't run the client on the server machine as you wouldn't be protecting much by backing up to the same machine. > ERROR: SSL error during Load CA cert: error:02001002:system > library:fopen:No such file or directory Looks like it can't find the TrustedCAsFile. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 8 09:26:21 2008 From: boxbackup at fluffy.co.uk (Alex Harper) Date: Fri, 08 Feb 2008 01:26:21 -0800 Subject: [Box Backup] Endian bug in rc2 and trunk... Message-ID: Hi, Apparent endian bug in rc2 (also present on trunk). Client (bbackupd) is x86 OS X client 10.5.1, server (bbstored) is PPC OS X Server 10.5.1. This is a fresh install, first backup of this account (or any account for that matter). bbstored extended logging: WARNING: Message from child process 1116: Incoming connection from 192.168.1.10 port 51883 Client certificate CN: BACKUP-5dd Send block allocation size is 1024 Receive Version(0x1) Receive Version(0x1) Send Version(0x1) Send Version(0x1) Receive Login(0x5dd,0) Receive Login(0x5dd,0) NOTICE: Login from Client ID 0x000005dd Read/Write Send LoginConfirmed(0,0x2,0xc80000,0x1180000) Send LoginConfirmed(0,0x2,0xc80000,0x1180000) Receive ListDirectory(0x100000000000000,0x2,0xc,false) Receive ListDirectory(0x100000000000000,0x2,0xc,false) WARNING: Exception thrown: RaidFileException(RaidFileDoesntExist) at RaidFileRead.cpp(1063) Connection statistics for BACKUP-5dd: IN=81 OUT=84 TOTAL=165 ERROR: Error in child process, terminating connection: RaidFile RaidFileDoesntExist (Error when accessing a file on the store. Check the store with bbstoreaccounts check.) (2/11) Forcing bbackupd to run on the x86 client under PPC emulation fixes the problem, with the corrected command: ListDirectory(0x1,0x2,0xc,false) I'm sure this is a trivial fix for 0.11 but it leaves me wondering about other lurking endian bugs. Of course, I completely understand Box comes with no guarantees and that my setup may be unusual. Would be it reasonable to force all my clients to run PPC binaries (in emulation if necessary)? While I could switch to an x86 server, that would just move the problem to the PPC clients. I'm wondering if forcing everyone to use network-order natively is a way to avoid pitfalls. Thanks for any advice, Alex -- Alex Harper aharper at foobox.net "Eternity is a terrible thought. I mean, where's it going to end?" - Tom Stoppard From boxbackup at fluffy.co.uk Fri Feb 8 12:14:56 2008 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Fri, 8 Feb 2008 12:14:56 +0000 Subject: [Box Backup] Endian bug in rc2 and trunk... Message-ID: <239F0FC2-7049-4EE5-B603-D48633A76423@fluffy.co.uk> Alex Harper wrote: > Apparent endian bug in rc2 (also present on trunk). Client > (bbackupd) is x86 > OS X client 10.5.1, server (bbstored) is PPC OS X Server 10.5.1. > [snip] > I'm sure this is a trivial fix for 0.11 but it leaves me wondering > about > other lurking endian bugs. Of course, I completely understand Box > comes with > no guarantees and that my setup may be unusual. > > Would be it reasonable to force all my clients to run PPC binaries (in > emulation if necessary)? While I could switch to an x86 server, > that would > just move the problem to the PPC clients. I'm wondering if forcing > everyone > to use network-order natively is a way to avoid pitfalls. I'm pretty sure this a big introduced after 0.10. I currently run 0.10 with different byte orderings on the client and the server, without difficulties, and for versions up to 0.10 I was developing everything across different endianesses. It is a little complicated to test though, the tests could be made to handle it more easily. There was one little endian bug in a previous version, but that was fixed and there's code to work around any data which was affected. Care has been taken to make endian issues automatically taken care of, for example, the autogenerated Protocol classes do it for you. But perhaps the changes between 0.10 and 0.11 should be reviewed, as I think the current developers use a single platform. This does seem a bit of a glaring error. Ben From boxbackup at fluffy.co.uk Fri Feb 8 16:00:46 2008 From: boxbackup at fluffy.co.uk (Alex Harper) Date: Fri, 08 Feb 2008 08:00:46 -0800 Subject: [Box Backup] Endian bug in rc2 and trunk... In-Reply-To: <239F0FC2-7049-4EE5-B603-D48633A76423@fluffy.co.uk> Message-ID: Hi Ben, > Care has been taken to make endian issues automatically taken care > of, for example, the autogenerated Protocol classes do it for you. I'm glad to hear that endianess was so carefully considered, and thus reassured I went back and double-checked my own work. The good news is that it all appears to be my fault :) I have CFLAGS=3D"-arch i386 -arch ppc" and similar LDFLAGS defined in my environment by default, so that I build OS X "universal" binaries of variou= s libraries I use for my own development. Unfortunately this apparently trick= s Box Backup's configure into undef'ing WORDS_BIGENDIAN for both architectures.=20 Running configure separately for each architecture gets WORDS_BIGENDIAN right and the resulting binaries appear to interoperate. I'm very sorry for the false alarm. Might I suggest that the endian logic at Box.h:140 test the gcc preprocesso= r magic __BIG_ENDIAN__ and __LITTLE_ENDIAN__ before falling back to trusting WORDS_BIGENDIAN? These are more likely to be definitive. Again, sorry for the false alarm, Alex -- Alex Harper aharper at foobox.net =B3We cross our bridges when we come to them and burn them behind us, with nothing to show for our progress except a memory of the smell of smoke, and a presumption that once our eyes watered.=B2 - Tom Stoppard From boxbackup at fluffy.co.uk Mon Feb 11 12:27:42 2008 From: boxbackup at fluffy.co.uk (Oli) Date: Mon, 11 Feb 2008 12:27:42 +0000 Subject: [Box Backup] Manually deleting files In-Reply-To: References: <4799B1F2.3010800@3ait.co.uk> <62EAD2CB-343A-49D8-9B3E-33172351B1E8@mbrown.co.uk> <4799C489.8090103@3ait.co.uk> Message-ID: <47B03F3E.2030503@3ait.co.uk> Hi Chris, > > Box Backup 0.11 (rc1) will do a backup run and delete excluded files even > if the store is full (or at least it should, please let me know if it > doesn't). Bad form posting over myself, I know, but just found something out which might be relevant: In case it wanted to delete excluded files AFTER the backup run, I tried commenting out all my servers config and try a run. I got this: Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: NOTICE: 4 redundant locations in root directory found, wi ll delete from store after 172800 seconds. Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Unused location in root: earth Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Unused location in root: venus Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Unused location in root: firewalls Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Unused location in root: jupiter Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Not deleting unused entries - too early (172799 seconds r emaining) Which kinda gives me another way out - I can comment 'earth' and have it delete it from the store, then put it back in to make it back it up again. Obviously this is a long way from ideal - That'll copy around 200Mb But it's only a solution if I don't have to wait 172799 seconds - ~50 hours. Can't see anywhere to change this :-( So it notices the lack of dir, but not the excludes. I've checked syntax again and paths relative the onsite backup - all correct... Cheers, -Oli -- Oli Comber Systems Developer 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ M: +44 (0)77255 82405 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 3aIT Limited is a company registered in England and Wales. CoReg: 3866698 VATReg: 771388600 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. 3aIT Limited From boxbackup at fluffy.co.uk Mon Feb 11 12:05:24 2008 From: boxbackup at fluffy.co.uk (Oli) Date: Mon, 11 Feb 2008 12:05:24 +0000 Subject: [Box Backup] Manually deleting files In-Reply-To: References: <4799B1F2.3010800@3ait.co.uk> <62EAD2CB-343A-49D8-9B3E-33172351B1E8@mbrown.co.uk> <4799C489.8090103@3ait.co.uk> Message-ID: <47B03A04.4070202@3ait.co.uk> Hi Chris, sorry to not get back to you straight away... Chris Wilson wrote: > Hi Oli and Matt, > > On Fri, 25 Jan 2008, Oli wrote: > >> Unfortunately the store has no old or 'deleted' files to delete, it's >> all 'current' :-( >> >> If I could convince it to do a backup run, it would probably mark as >> deleted a few. > > I think you didn't say what version of Box Backup you're running on the > client. > > Box Backup 0.11 (rc1) will do a backup run and delete excluded files even > if the store is full (or at least it should, please let me know if it > doesn't). I've just upgraded to 0.11RC2 on client and server, and I set up my excluded files list to include various directories and files which are very large. I had also previously increased the store size and done a backup run, unfortunately it filled up again - otherwise I was hoping to fix it this way first. Anyhoo, just upgraded and tried it, but unfortunately it doesn't seem to have gone through and deleted the newly excluded files, so my store is still full. Here's some logs... Client: Feb 11 11:44:16 io Box Backup[26907]: NOTICE: Using configuration file /etc/box/bbackupd.conf Feb 11 11:44:16 io Box Backup (bbackupd)[26904]: Incoming connection from local (UNIX socket) Feb 11 11:44:16 io Box Backup (bbackupd)[26904]: Connection from command socket Feb 11 11:44:16 io Box Backup[26907]: Daemon configuration summary: AutomaticBackup = false UpdateStor eInterval = 0 seconds MinimumFileAge = 60 seconds MaxUploadWait = 0 seconds Feb 11 11:44:16 io Box Backup (bbackupd)[26904]: NOTICE: Beginning scan of local files Feb 11 11:44:16 io Box Backup (bbackupd)[26904]: Opening connection to server '***' Feb 11 11:44:16 io Box Backup[26907]: Sync started... Feb 11 11:44:17 io Box Backup (bbackupd)[26904]: Connection made, login successful Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Receiving stream, size 240 Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Not deleting unused entries - none in list Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: NOTICE: About to notify administrator about event backup- start, running script '/etc/box/bbackupd/NotifySysadmin.sh backup-start' Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Scanning directory: ***censored... ;-) *** Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Skipping excluded directory: ***censored... ;-) *** Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Receiving stream, size 1073 Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Uploading complete file: ***censored... ;-) *** ar Feb 11 11:44:18 io Box Backup (bbackupd)[26904]: Sending stream, size uncertain Feb 11 11:45:39 io Box Backup (bbackupquery)[26989]: NOTICE: Box Backup Query Tool v0.11rc2, (c) Ben Summe rs and contributors 2003-2008 Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: WARNING: Exception thrown: ConnectionException(Conn_Proto col_UnexpectedReply) at autogen_BackupProtocolClient.cpp(1485) Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: NOTICE: About to notify administrator about event store-f ull, running script '/etc/box/bbackupd/NotifySysadmin.sh store-full' Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: ERROR: Failed to upload file: ***censored... ;-) *** rge.tar: server error: StorageLimitExceeded Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: ERROR: Failed to upload file: ***censored... ;-) *** rge.tar: caught exception: Connection Protocol_UnexpectedReply (Server probably reported an error.) (7/47) Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: NOTICE: About to notify administrator about event backup- error, running script '/etc/box/bbackupd/NotifySysadmin.sh backup-error' Feb 11 11:49:16 io Box Backup (bbackupd)[26904]: ERROR: Exception caught (Connection Protocol_UnexpectedRe ply (Server probably reported an error.) 7/47), reset state and waiting to retry... Feb 11 11:49:26 io Box Backup (bbackupd)[26904]: NOTICE: File statistics: total file size uploaded 9222144 0, bytes already on server 0, encoded size 82014870 Feb 11 11:49:26 io Box Backup[26907]: Sync finished. Interesting the client can't tell the file size, and the server doesn't give an error until it's finished... Server log: Feb 11 11:26:44 mars Box Backup (bbstored)[27204]: NOTICE: Starting daemon, version 0.11rc2, config: /etc/box/bbstored.conf Feb 11 11:32:37 mars Box Backup (bbstored)[27204]: WARNING: Message from child process 27233: Incoming connection from ***censored... ;-) *** port 61910 Feb 11 11:32:38 mars Box Backup (bbstored)[27233]: NOTICE: Login from Client ID 0x075ab23c Read/Write Feb 11 11:32:38 mars Box Backup (bbstored)[27233]: Sending stream, size 240 Feb 11 11:32:39 mars Box Backup (bbstored)[27233]: Sending stream, size 1073 Feb 11 11:32:39 mars Box Backup (bbstored)[27233]: Receiving stream, size uncertain Feb 11 11:32:46 mars Box Backup (bbstored)[27204]: WARNING: Message from child process 27234: Incoming connection from ***censored... ;-) *** 58757 Feb 11 11:32:46 mars Box Backup (bbstored)[27234]: NOTICE: Login from Client ID 0x075ab23c Read-only Feb 11 11:35:22 mars Box Backup (bbstored)[27234]: NOTICE: Session finished for Client ID 0x075ab23c Feb 11 11:36:25 mars Box Backup (bbstored)[27233]: WARNING: Exception thrown: BackupStoreException(AddedFileExceedsStorageLi mit) at BackupContext.cpp(569) Feb 11 11:36:25 mars Box Backup (bbstored)[27233]: NOTICE: Session finished for Client ID 0x075ab23c > > Box Backup 0.10 (stable release) will not do anything else after it > discovers that the store is full, unfortunately. > >> Actually, that is helpful... you've given me a way to mark some bits as >> deleted immediately once I've increased the store size without waiting >> interfering with a full backup run, then I can shrink it again to force >> the deletion of the old bits. > > Out of curiosity, what way was that? I didn't see it. Oh, I'm a bit of a BB noob... I didn't know how to trigger a manual backup :-) > >> I'd still quite like to be able to just zap the odd file/dir that I know >> doesn't need to be there any more (ie, I just excluded it on the client) > > I'm thinking about adding that feature to 0.11, even though it's in > feature freeze right now, so I shouldn't really. At least it should be > quite simple and easy to add. That would be seriously cool :-) > > If you're running 0.10, I'd advise/request you to upgrade to 0.11rc1 on > the client and see if that fixes the problem for you. > > Cheers, Chris. Cheers, -Oli -- Oli Comber Systems Developer 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ M: +44 (0)77255 82405 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 3aIT Limited is a company registered in England and Wales. CoReg: 3866698 VATReg: 771388600 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. 3aIT Limited From boxbackup at fluffy.co.uk Mon Feb 11 22:18:52 2008 From: boxbackup at fluffy.co.uk (=?iso-8859-2?Q?Mitja_Mu=BEeni=E8?=) Date: Mon, 11 Feb 2008 23:18:52 +0100 Subject: [Box Backup] 0.11rc2 - logic error in autoconf Message-ID: <20080211231915.56CC03B8@srv.oneis.co.uk> Hi! While testing 0.11rc2 on OpenBSD/i386 I noticed that the logic criteria for "__syscall needing definition" configure test were inverted. This test was never executed during configure and because of that, the regress tests failed in intercept.cpp. The following patch fixes that: --- infrastructure/m4/ax_func_syscall.m4.orig Tue Feb 12 00:11:49 2008 +++ infrastructure/m4/ax_func_syscall.m4 Tue Feb 12 00:12:19 2008 @@ -16,8 +16,8 @@ AC_DEFUN([AX_FUNC_SYSCALL], [ AC_CHECK_HEADERS([sys/syscall.h unistd.h]) AC_CHECK_FUNCS([syscall __syscall]) - if test "x$ac_cv_func_syscall" != "xyes" && - test "x$ac_cv_func___syscall" != "xyes"; then + if test "x$ac_cv_func_syscall" = "xyes" && + test "x$ac_cv_func___syscall" = "xyes"; then AC_CACHE_CHECK([for __syscall needing definition], [have___syscall_need_defn], [AC_RUN_IFELSE([AC_LANG_PROGRAM([[ $ac_includes_default (regenerate configure after applying) Regards, Mitja From boxbackup at fluffy.co.uk Tue Feb 12 03:49:17 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Tue, 12 Feb 2008 14:49:17 +1100 Subject: [Box Backup] Windows client help! Message-ID: Hello guys, I'm trying to get my Windows machine to backup to my Ubuntu server. My Ubuntu workstation is backing up fine, so I'm familiar with the steps required to configure a client. The problem is that my Windows client is complaining about not being able to load the SSL certs. I have them where the conf says they should be, I'm loading the right config file etc, I'm wondering if I need the SSL libraries on my Windows system. I'm using the "boxbackup-chris_general_1857-backup-client-mingw32.zip", which I found on the wiki. I think this is the latest Windows client build. However, there are no ssl dlls here, or have they been statically compiled? I don't know very much about this sort of thing I'm afraid. This page http://www.boxbackup.org/instguide/apb.html#id2596280 tells me that I need the openssl files. Where can I find them? Kind regards, Timothy. From boxbackup at fluffy.co.uk Tue Feb 12 09:46:53 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 12 Feb 2008 09:46:53 +0000 (GMT) Subject: [Box Backup] Windows client help! In-Reply-To: References: Message-ID: Hi Timothy, On Tue, 12 Feb 2008, Timothy Wilson wrote: > I'm trying to get my Windows machine to backup to my Ubuntu server. My > Ubuntu workstation is backing up fine, so I'm familiar with the steps > required to configure a client. The problem is that my Windows client > is complaining about not being able to load the SSL certs. I have them > where the conf says they should be, I'm loading the right config file > etc, I'm wondering if I need the SSL libraries on my Windows system. No, it's statically compiled in. The paths must be wrong. What does your configuration file say and what do the error messages say? > I'm using the "boxbackup-chris_general_1857-backup-client-mingw32.zip", > which I found on the wiki. I think this is the latest Windows client > build. However, there are no ssl dlls here, or have they been statically > compiled? I don't know very much about this sort of thing I'm afraid. > > This page http://www.boxbackup.org/instguide/apb.html#id2596280 tells me > that I need the openssl files. Where can I find them? I think those instructions are outdated. I've never needed or distributed those files. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Tue Feb 12 13:29:43 2008 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Tue, 12 Feb 2008 14:29:43 +0100 Subject: [Box Backup] rc2 - Possible error in NotifySysadmin.sh Message-ID: <47B19F47.4060800@kontrapunkt.com> Hello... When running NotifySysadmin.sh with no argument, the instructions returned are: Usage: /usr/local/bin/bbackupd-config Shouldnt the correct message be ?: Usage: /etc/box/bbackupd/NotifySysadmin.sh Thanks, Tobias From boxbackup at fluffy.co.uk Tue Feb 12 13:40:06 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 12 Feb 2008 13:40:06 +0000 Subject: [Box Backup] rc2 - Possible error in NotifySysadmin.sh In-Reply-To: <47B19F47.4060800@kontrapunkt.com> References: <47B19F47.4060800@kontrapunkt.com> Message-ID: <20080212134006.GJ15350@netinertia.co.uk> Hi Tobias, On Tue, Feb 12, 2008 at 02:29:43PM +0100, Tobias Balle-Petersen wrote: > Hello... > > When running NotifySysadmin.sh with no argument, the instructions returned > are: > Usage: /usr/local/bin/bbackupd-config > > > > Shouldnt the correct message be ?: > Usage: /etc/box/bbackupd/NotifySysadmin.sh > Thanks, I've corrected this in trunk. James From boxbackup at fluffy.co.uk Tue Feb 12 16:48:26 2008 From: boxbackup at fluffy.co.uk (Pablo Fernandez) Date: Tue, 12 Feb 2008 14:48:26 -0200 Subject: [Box Backup] Stats Message-ID: <1202834906.5715.3.camel@laptop> Hi I would like to have some more information about what this means: Blocks used: 19052161 (37211.25Mb) Blocks used by old files: 2557740 (4995.59Mb) Blocks used by deleted files: 3102255 (6059.09Mb) Blocks used by directories: 6889 (13.46Mb) What does directories mean there? The total size of the current files would be used - old - deleted - directories? Thanks Best regards, Pablo From boxbackup at fluffy.co.uk Tue Feb 12 17:36:04 2008 From: boxbackup at fluffy.co.uk (Eric Cronin) Date: Tue, 12 Feb 2008 12:36:04 -0500 Subject: [Box Backup] Mac NotifySysadmin.sh customizations Message-ID: <2E7062BF-4E57-474E-9DA3-CBA72F1D5D34@gizmolabs.org> This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --Apple-Mail-8--955218133 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Has anyone running box on Mac OS X done any customization of the notification script? I'm thinking particularly Growl notification... Right now the script is somewhat useless on vanilla mac's due to postfix being unconfigured and disabled out of the box, and I'd like to fix it to do something for the MacPorts update. Thanks, Eric --Apple-Mail-8--955218133 content-type: application/pgp-signature; x-mac-type=70674453; name=PGP.sig content-description: This is a digitally signed message part content-disposition: inline; filename=PGP.sig content-transfer-encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQEcBAEBCAAGBQJHsdkEAAoJEIgz4Q+coYsS06QH/iEprjnyW07VB53wrQ2AV/RA 7e7mnZCC1Q5/ey1hhyKl2XxIpqcfOV/EaWrdCloDpx3Aqkk8t+0RAnUvHP53o1XG UODKdJeQyfXljnMlwCm0fAmj1DekGayEGufum4CcvZK9MSbj90bUOLWTmyT1aEIy AFbUt8GHpqTsnYENqSFG5AhgEWME825+USQgPYhlziYecEt8K+Mdftc7C5dVloh3 sBGcmIQJFDuaOwUtnnsV8Oo8atyCoqRwAeV7W8EgopKYws8pLQNWTr1tHjKWs1gy lj8i4KCkAAjsnXkjusyqgjq5exOaFa74asy9h14GzsQnS2AOETGhPBxwBnODwKk= =qazP -----END PGP SIGNATURE----- --Apple-Mail-8--955218133-- From boxbackup at fluffy.co.uk Tue Feb 12 17:58:25 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 12 Feb 2008 17:58:25 +0000 Subject: [Box Backup] Mac NotifySysadmin.sh customizations In-Reply-To: <2E7062BF-4E57-474E-9DA3-CBA72F1D5D34@gizmolabs.org> References: <2E7062BF-4E57-474E-9DA3-CBA72F1D5D34@gizmolabs.org> Message-ID: <20080212175824.GL15350@netinertia.co.uk> Hi Eric, On Tue, Feb 12, 2008 at 12:36:04PM -0500, Eric Cronin wrote: > Has anyone running box on Mac OS X done any customization of the > notification script? I'm thinking particularly Growl notification... Right > now the script is somewhat useless on vanilla mac's due to postfix being > unconfigured and disabled out of the box, and I'd like to fix it to do > something for the MacPorts update. I'm not running Box on OS X, but are you sure it's not working? The script just runs /usr/sbin/sendmail, which doesn't require a locally-enabled SMTP service. Ben also wrote most of these scripts on OS X so they should work... When you say Growl notification, do you mean the MailMe action? Otherwise I'm not sure it would be much use, unless someone were watching the console 24x7 :-) James From boxbackup at fluffy.co.uk Tue Feb 12 22:23:54 2008 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Tue, 12 Feb 2008 14:23:54 -0800 Subject: [Box Backup] Stats In-Reply-To: <1202834906.5715.3.camel@laptop> References: <1202834906.5715.3.camel@laptop> Message-ID: <47B21C7A.9070903@reedtz.com> On 2/12/08 8:48 AM, Pablo Fernandez wrote: > Hi > > I would like to have some more information about what this means: > > Blocks used: 19052161 (37211.25Mb) > Blocks used by old files: 2557740 (4995.59Mb) > Blocks used by deleted files: 3102255 (6059.09Mb) > Blocks used by directories: 6889 (13.46Mb) > > What does directories mean there? The total size of the current files > would be used - old - deleted - directories? > Here is my (somewhat educated) understanding of the numbers: - Blocks used: The total number of blocks used by box for this account - Blocks used by old files: The number of blocks used by older versions of files. I.e. there is one or more newer versions of this file, and this number denotes how much is 'not current', but still retained, in case you want to retrieve older versions of the file(s). - Blocks used by deleted files: The number of blocks used by files that have been deleted on the client machine. Again, these files are still retrievable from the server, and this tells you how much space those deleted files are taking up. - Blocks used by directories: The actual space taken up by directories on the server. So your math is correct, AFAIK. current files = used - old - deleted - directories; HTH, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Tue Feb 12 23:02:07 2008 From: boxbackup at fluffy.co.uk (Eric Cronin) Date: Tue, 12 Feb 2008 18:02:07 -0500 Subject: [Box Backup] Mac NotifySysadmin.sh customizations In-Reply-To: <20080212175824.GL15350@netinertia.co.uk> References: <2E7062BF-4E57-474E-9DA3-CBA72F1D5D34@gizmolabs.org> <20080212175824.GL15350@netinertia.co.uk> Message-ID: <71405AFB-A7B4-4EC2-AD04-2EFDE7323EFB@gizmolabs.org> On Feb 12, 2008, at 12:58 PM, James O'Gorman wrote: > Hi Eric, > > On Tue, Feb 12, 2008 at 12:36:04PM -0500, Eric Cronin wrote: >> Has anyone running box on Mac OS X done any customization of the >> notification script? I'm thinking particularly Growl >> notification... Right >> now the script is somewhat useless on vanilla mac's due to postfix >> being >> unconfigured and disabled out of the box, and I'd like to fix it to >> do >> something for the MacPorts update. > > I'm not running Box on OS X, but are you sure it's not working? The > script just runs /usr/sbin/sendmail, which doesn't require a > locally-enabled SMTP service. Ben also wrote most of these scripts > on OS > X so they should work... /usr/sbin/sendmail (which is the postfix variety) is less than reliable on a stock osx install I've found, even before most ISPs started blocking outgoing port 25... About 75% of the time alerts sit in the mailq until I "postfix start" on my laptop. The script works fine if the osx box is more of a "server" (and then has a unfiltered IP, configured hostname, postfix running for retries etc). > When you say Growl notification, do you mean the MailMe action? > Otherwise I'm not sure it would be much use, unless someone were > watching the console 24x7 :-) For desktops/laptops running boxbackup someone's watching the console pretty regularly, especially if the growl setting is not to auto- dismiss alerts from bbackupd... Doesn't make sense all the time or as a default, but is nice when the conditions are there. Thanks, Eric From boxbackup at fluffy.co.uk Wed Feb 13 05:14:42 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Wed, 13 Feb 2008 00:14:42 -0500 Subject: [Box Backup] Stats In-Reply-To: <47B21C7A.9070903@reedtz.com> References: <1202834906.5715.3.camel@laptop> <47B21C7A.9070903@reedtz.com> Message-ID: <74d01c7a0802122114v1d8632cetfe273335bc43d11c@mail.gmail.com> Hi all, Here's an interesting account, in this regard: Blocks used: 41138268 (40174.09Mb) Blocks used by old files: 30521871 (29806.51Mb) Blocks used by deleted files: 20616549 (20133.35Mb) Block soft limit: 51200000 (50000.00Mb) Block hard limit: 76800000 (75000.00Mb) Notice how old + deleted > used. So, it appears that files that are both old and deleted are counted twice. I have not been able to figure out a way disentangle them from the server side. See my feature request dated 2007-02-25 on http://www.boxbackup.org/trac/wiki/FeatureRequests Take care, Pete From boxbackup at fluffy.co.uk Wed Feb 13 07:05:14 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Wed, 13 Feb 2008 07:05:14 +0000 Subject: [Box Backup] Mac NotifySysadmin.sh customizations In-Reply-To: <71405AFB-A7B4-4EC2-AD04-2EFDE7323EFB@gizmolabs.org> References: <2E7062BF-4E57-474E-9DA3-CBA72F1D5D34@gizmolabs.org> <20080212175824.GL15350@netinertia.co.uk> <71405AFB-A7B4-4EC2-AD04-2EFDE7323EFB@gizmolabs.org> Message-ID: <20080213070514.GM15350@netinertia.co.uk> Hi Eric, On Tue, Feb 12, 2008 at 06:02:07PM -0500, Eric Cronin wrote: > /usr/sbin/sendmail (which is the postfix variety) is less than reliable on a > stock osx install I've found, even before most ISPs started blocking > outgoing port 25... About 75% of the time alerts sit in the mailq until I > "postfix start" on my laptop. > > The script works fine if the osx box is more of a "server" (and then has a > unfiltered IP, configured hostname, postfix running for retries etc). Gotcha. > > When you say Growl notification, do you mean the MailMe action? > > Otherwise I'm not sure it would be much use, unless someone were > > watching the console 24x7 :-) > > For desktops/laptops running boxbackup someone's watching the console pretty > regularly, especially if the growl setting is not to auto-dismiss alerts > from bbackupd... Doesn't make sense all the time or as a default, but is > nice when the conditions are there. Makes sense. I'll have a look later and see if I can knock something up. It should be pretty straightforward, although it will add yet another OS-dependent script into the distribution... James From boxbackup at fluffy.co.uk Thu Feb 14 01:49:12 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 14 Feb 2008 12:49:12 +1100 Subject: [Box Backup] Windows client help! In-Reply-To: References: Message-ID: Hello guys, Thanks for your reply Chris. These are the error messages: NOTICE: Starting daemon, version chris_general_1857, config: C:\Program Files\Box Backup\bbackupd.conf ERROR: SSL error during Load certificates: error:02001003:system library:fopen:No such process ERROR: SSL error during Load certificates: error:20074002:BIO routines:FILE_CTRL:system lib ERROR: SSL error during Load certificates: error:140DC002:SSL routines:SSL_CTX_use_certificate_chain_file:system lib WARNING: Exception thrown: ServerException(TLSLoadCertificatesFailed) at TLSContext.cpp(79) FATAL: Terminating due to exception Server TLSLoadCertificatesFailed (3/25) ERROR: Failed to ConnectNamedPipe(\\.\pipe\C:\Program Files\Box Backup\bbackupd.sock): The pipe has been ended. (109) Here is the relevant bit of my bbackupd.conf: AccountNumber = 0000001 KeysFile = C:\Program Files\Box Backupbbackupd\1-FileEncKeys.raw CertificateFile = C:\Program Files\Box Backupbbackupd\1-cert.pem PrivateKeyFile = C:\Program Files\Box Backupbbackupd\1-key.pem TrustedCAsFile = C:\Program Files\Box Backupbbackupd\serverCA.pem DataDirectory = C:\Program Files\Box Backup CommandSocket = C:\Program Files\Box Backup\bbackupd.sock Server { PidFile = C:\Program Files\Box Backup\bbackupd.pid } Is anything glaringly wrong? I've just ommited some things that I don't think are relevant (such as my backup locations etc). I appreciate your assistance! Kind regards, Timothy. From boxbackup at fluffy.co.uk Thu Feb 14 06:04:41 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 14 Feb 2008 17:04:41 +1100 Subject: [Box Backup] Windows client help! In-Reply-To: References: Message-ID: Whoops, just noticed I had wrong paths to my certificates. Gah I'm silly! Sorry, I thought I fixed that error, but that was in another conf file...never mind. I'll try to be more careful next time :( Timothy. From boxbackup at fluffy.co.uk Tue Feb 19 16:06:45 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Tue, 19 Feb 2008 17:06:45 +0100 Subject: [Box Backup] Run several config-files as one Client Message-ID: <47BAFE95.6080001@my-cooper.de> Hi, I have several jails/virtual servers running on one machine. I want to configure BoxBackup to stop the first jail, make a backup, start the server again, then stop the next one etc. I was thinking to make a shell-script which calls the bbackupctl-command with a different config-file each but as the same client-id (which belongs to the whole machine). Here I could include the different folders for each jail/virtual server. Is that possible or how would you realize this? Thanks, Ben. From boxbackup at fluffy.co.uk Tue Feb 19 18:59:47 2008 From: boxbackup at fluffy.co.uk (Michiel van Baak) Date: Tue, 19 Feb 2008 19:59:47 +0100 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <47BAFE95.6080001@my-cooper.de> References: <47BAFE95.6080001@my-cooper.de> Message-ID: <20080219185946.GF20991@vanbaak.info> On 17:06, Tue 19 Feb 08, Benedikt Niessen wrote: > Hi, > > I have several jails/virtual servers running on one machine. > > I want to configure BoxBackup to stop the first jail, make a backup, > start the server again, then stop the next one etc. > > I was thinking to make a shell-script which calls the bbackupctl-command > with a different config-file each but as the same client-id (which > belongs to the whole machine). Here I could include the different > folders for each jail/virtual server. > > Is that possible or how would you realize this? I have several xen boxen that run a bunch of virtual machines. What I did is install a bbackupd on every virtual machine with their own account id. That way you'll have backups without downtime of virtual machines. Also, once a vm/jail needs to be moved to another piece of hardware you dont have to worry about the backupschema, because it has it's own client. -- Michiel van Baak michiel at vanbaak.eu http://michiel.vanbaak.eu GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x71C946BD "Why is it drug addicts and computer aficionados are both called users?" From boxbackup at fluffy.co.uk Tue Feb 19 19:06:22 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Tue, 19 Feb 2008 20:06:22 +0100 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <20080219185946.GF20991@vanbaak.info> References: <47BAFE95.6080001@my-cooper.de> <20080219185946.GF20991@vanbaak.info> Message-ID: <47BB28AE.9010600@my-cooper.de> Hi, This is a not so good option for me as I share most of the files between the jails so I would safe the files for every jail. Thats a minor alternative. Any other hints? But thanks anyway... Ben. Michiel van Baak schrieb: > On 17:06, Tue 19 Feb 08, Benedikt Niessen wrote: >> Hi, >> >> I have several jails/virtual servers running on one machine. >> >> I want to configure BoxBackup to stop the first jail, make a backup, >> start the server again, then stop the next one etc. >> >> I was thinking to make a shell-script which calls the bbackupctl-command >> with a different config-file each but as the same client-id (which >> belongs to the whole machine). Here I could include the different >> folders for each jail/virtual server. >> >> Is that possible or how would you realize this? > > I have several xen boxen that run a bunch of virtual > machines. > What I did is install a bbackupd on every virtual machine > with their own account id. That way you'll have backups > without downtime of virtual machines. > Also, once a vm/jail needs to be moved to another piece of > hardware you dont have to worry about the backupschema, > because it has it's own client. > From boxbackup at fluffy.co.uk Tue Feb 19 19:46:06 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 19 Feb 2008 19:46:06 +0000 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <47BAFE95.6080001@my-cooper.de> References: <47BAFE95.6080001@my-cooper.de> Message-ID: <20080219194606.GN15350@netinertia.co.uk> Hi Ben, On Tue, Feb 19, 2008 at 05:06:45PM +0100, Benedikt Niessen wrote: > Hi, > > I have several jails/virtual servers running on one machine. > > I want to configure BoxBackup to stop the first jail, make a backup, start > the server again, then stop the next one etc. > > I was thinking to make a shell-script which calls the bbackupctl-command > with a different config-file each but as the same client-id (which belongs > to the whole machine). Here I could include the different folders for each > jail/virtual server. > > Is that possible or how would you realize this? Assuming you mean FreeBSD's jails, and are running a fairly recent release, you could probably do something like this: #!/bin/sh # Suck in the rc.conf settings . /etc/defaults/rc.conf source_rc_confs # Loop through $jail list, stop each jail, run a sync and start it again for jail in $jail_list; do /etc/rc.d/jail stop $jail bbackupctl -c /usr/local/etc/box/bbackupd_$jail.conf sync /etc/rc.d/jail start $jail done NB: This is totally untested, but should give you the basic idea :-) Not quite sure why you want to stop the jail though - can't you just have a single bbackupd.conf with a location for each jail and have it backup automatically as usual? James From boxbackup at fluffy.co.uk Tue Feb 19 19:57:32 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Tue, 19 Feb 2008 20:57:32 +0100 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <20080219194606.GN15350@netinertia.co.uk> References: <47BAFE95.6080001@my-cooper.de> <20080219194606.GN15350@netinertia.co.uk> Message-ID: <47BB34AC.6010904@my-cooper.de> > Not quite sure why you want to stop the jail though - can't you just > have a single bbackupd.conf with a location for each jail and have it > backup automatically as usual? That was my idea. Im just a little confused about the client id thing. My idea was to name the server with the id 1 and have a copy for each bbackupd.conf jail, bbackupd_$jail.conf as you named it. The only difference would be the foldername inside the config. I have not had a close look or any experience with boxbackup so I dont know if this would work. But if it works and BoxBackup is not "confused", then sure this is my way to go. Thanks for your help. James O'Gorman schrieb: > Hi Ben, > > On Tue, Feb 19, 2008 at 05:06:45PM +0100, Benedikt Niessen wrote: >> Hi, >> >> I have several jails/virtual servers running on one machine. >> >> I want to configure BoxBackup to stop the first jail, make a backup, start >> the server again, then stop the next one etc. >> >> I was thinking to make a shell-script which calls the bbackupctl-command >> with a different config-file each but as the same client-id (which belongs >> to the whole machine). Here I could include the different folders for each >> jail/virtual server. >> >> Is that possible or how would you realize this? > > Assuming you mean FreeBSD's jails, and are running a fairly recent > release, you could probably do something like this: > > #!/bin/sh > > # Suck in the rc.conf settings > . /etc/defaults/rc.conf > source_rc_confs > > # Loop through $jail list, stop each jail, run a sync and start it again > for jail in $jail_list; do > /etc/rc.d/jail stop $jail > bbackupctl -c /usr/local/etc/box/bbackupd_$jail.conf sync > /etc/rc.d/jail start $jail > done > > NB: This is totally untested, but should give you the basic idea :-) > > Not quite sure why you want to stop the jail though - can't you just > have a single bbackupd.conf with a location for each jail and have it > backup automatically as usual? > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Feb 19 20:15:14 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 19 Feb 2008 20:15:14 +0000 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <47BB34AC.6010904@my-cooper.de> References: <47BAFE95.6080001@my-cooper.de> <20080219194606.GN15350@netinertia.co.uk> <47BB34AC.6010904@my-cooper.de> Message-ID: <20080219201513.GO15350@netinertia.co.uk> On Tue, Feb 19, 2008 at 08:57:32PM +0100, Benedikt Niessen wrote: > That was my idea. Im just a little confused about the client id thing. My > idea was to name the server with the id 1 and have a copy for each > bbackupd.conf jail, bbackupd_$jail.conf as you named it. The only difference > would be the foldername inside the config. The client id is just a reference for the box that's being backed up. In this case, the host server (i.e. the one which contains the jails) could have a single client id, and then when you wish to restore a file for a jail, you just retrieve it from that jail's location in the backup, like: $ bbackupquery query > cd jail1/etc query > get rc.conf /root/rc.conf.restored Object ID xxxxxxxx fetched sucessfully. query > cd /jail2 query > restore home /newdisk/home This is probably the easiest way to manage your jails, rather than having a separate client account for each jail. I have my jails backed up this way, and it works fine :-) > I have not had a close look or any experience with boxbackup so I dont know > if this would work. But if it works and BoxBackup is not "confused", then > sure this is my way to go. As far as Box cares, they're just files on a disk. You might want to look at the Exclude patterns if you don't want to backup the OS binaries (e.g. /bin, /sbin) from each jail. Here's my Location directive in bbackupd.conf: jails { Path = /data/jails ExcludeDirsRegex = .+/[s]?bin$ } HTH, James From boxbackup at fluffy.co.uk Tue Feb 19 20:22:31 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Tue, 19 Feb 2008 21:22:31 +0100 Subject: [Box Backup] Run several config-files as one Client In-Reply-To: <20080219201513.GO15350@netinertia.co.uk> References: <47BAFE95.6080001@my-cooper.de> <20080219194606.GN15350@netinertia.co.uk> <47BB34AC.6010904@my-cooper.de> <20080219201513.GO15350@netinertia.co.uk> Message-ID: <47BB3A87.8080607@my-cooper.de> Thats exactly what I was looking for. Thanks for that hint. I will go this way and give BoxBackup a try. Thanks for your help. James O'Gorman schrieb: > On Tue, Feb 19, 2008 at 08:57:32PM +0100, Benedikt Niessen wrote: >> That was my idea. Im just a little confused about the client id thing. My >> idea was to name the server with the id 1 and have a copy for each >> bbackupd.conf jail, bbackupd_$jail.conf as you named it. The only difference >> would be the foldername inside the config. > > The client id is just a reference for the box that's being backed up. In > this case, the host server (i.e. the one which contains the jails) could > have a single client id, and then when you wish to restore a file for a > jail, you just retrieve it from that jail's location in the backup, > like: > > $ bbackupquery > query > cd jail1/etc > query > get rc.conf /root/rc.conf.restored > Object ID xxxxxxxx fetched sucessfully. > query > cd /jail2 > query > restore home /newdisk/home > > This is probably the easiest way to manage your jails, rather than > having a separate client account for each jail. > > I have my jails backed up this way, and it works fine :-) > >> I have not had a close look or any experience with boxbackup so I dont know >> if this would work. But if it works and BoxBackup is not "confused", then >> sure this is my way to go. > > As far as Box cares, they're just files on a disk. You might want to > look at the Exclude patterns if you don't want to backup the OS binaries > (e.g. /bin, /sbin) from each jail. > > Here's my Location directive in bbackupd.conf: > > jails > { > Path = /data/jails > ExcludeDirsRegex = .+/[s]?bin$ > } > > HTH, > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Feb 19 23:54:30 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Wed, 20 Feb 2008 10:54:30 +1100 Subject: [Box Backup] Adding files from usb hdds Message-ID: Hello guys, I've been having some success with boxbackup, so thank you for writing and maintaining the program! I now have a problem where I have a couple of remote machines that I want to move to boxbackup, but they have more data than I care to download. So rather than sucking down multiple gigabytes of data through my small internet pipe, I was thinking I could just backup the targets to a usb hdd, and then plug it into the server to sync off. But how to get the data into the right account? The approach I was thinking of taking, would be to install the backup client on the server, create a new account for the remote machine, but point the backup config file's targets to the mounted hdd instead of the remote machine. However, I think that will muck up the paths; ie it will be backing up /mnt/hdd/home rather than /home. Is there a way to rename the paths, or am I going about this the wrong way? Is there a better approach? I haven't seen anything about this on the wiki :( Kind regards, Timothy. From boxbackup at fluffy.co.uk Wed Feb 20 06:47:44 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 07:47:44 +0100 Subject: [Box Backup] Transfer very slow Message-ID: <47BBCD10.70306@my-cooper.de> Hi, I tried to use BoxBackup as discussed yesterday and seems is working fine so far. My problem now is that the transfer between Server and Client is very very slow At the moment its running at around 4kb/sec at fast DSL line (max 480KB/s down) and a dedicated server which has a 100MBit connection. I found on the web that I should check the IO and for both servers I checked using "top" but everything normal, no high load. I used the standard config. Is there another way to check where is the problem or is there an option I could use? Im afraid backing up would need hours... Thanks for your help, Ben. From boxbackup at fluffy.co.uk Wed Feb 20 07:35:18 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 08:35:18 +0100 Subject: [Box Backup] BB not transfering all files/directories Message-ID: <47BBD836.9030809@my-cooper.de> Hi, After my first backup I checked the result to see if it worked. But seems not. I ran the sync-command several times and the Client Store Marker is changed from 0 to some other value, so seems it is finished. This is my configuration for the paths to backup, but e.g. /usr/local is not complete, /home is emtpy, the jailsfiles are empty... For me it looks like a permission problem. The service is running as _bbstored but I ran the bbackupctl -c ... sync as root. So what can/do I have to change to give the service all needed permissions? BackupLocations { etc { Path = /etc } root { Path = /root } boot { Path = /boot } usrlocal { Path = /usr/local } home { Path = /home } databackup { Path = /data/backup } jailsfiles { Path = /data/jails/_files } jailsflavour { Path = /data/jails/flavours } } Thanks for your help or any hint. Do I have to add _bbstored to the same group or sth? From boxbackup at fluffy.co.uk Wed Feb 20 07:41:11 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 08:41:11 +0100 Subject: [Box Backup] Transfer very slow In-Reply-To: <47BBCD10.70306@my-cooper.de> References: <47BBCD10.70306@my-cooper.de> Message-ID: <47BBD997.1090204@my-cooper.de> Seems the speed problem is solved. Its still quite instable but I guess this is a "problem" of the encryption, small files, sync etc. My connection was very just very busy... Benedikt Niessen schrieb: > Hi, > > I tried to use BoxBackup as discussed yesterday and seems is working > fine so far. My problem now is that the transfer between Server and > Client is very very slow > > At the moment its running at around 4kb/sec at fast DSL line (max > 480KB/s down) and a dedicated server which has a 100MBit connection. > > I found on the web that I should check the IO and for both servers I > checked using "top" but everything normal, no high load. > > I used the standard config. Is there another way to check where is the > problem or is there an option I could use? > > Im afraid backing up would need hours... > > Thanks for your help, > Ben. > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Feb 20 09:48:50 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 10:48:50 +0100 Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: <47BBD836.9030809@my-cooper.de> References: <47BBD836.9030809@my-cooper.de> Message-ID: <47BBF782.1080707@my-cooper.de> I mixed sth, on the client its surely running as root, not as _bbstored. The Server is running as _bbstored. Sorry for the confusing statement... Benedikt Niessen schrieb: > Hi, > After my first backup I checked the result to see if it worked. But > seems not. I ran the sync-command several times and the Client Store > Marker is changed from 0 to some other value, so seems it is finished. > > This is my configuration for the paths to backup, but e.g. /usr/local is > not complete, /home is emtpy, the jailsfiles are empty... For me it > looks like a permission problem. > > The service is running as _bbstored but I ran the bbackupctl -c ... sync > as root. So what can/do I have to change to give the service all needed > permissions? > > BackupLocations > { > etc > { > Path = /etc > } > root > { > Path = /root > } > boot > { > Path = /boot > } > usrlocal > { > Path = /usr/local > } > home > { > Path = /home > } > databackup > { > Path = /data/backup > } > jailsfiles > { > Path = /data/jails/_files > } > jailsflavour > { > Path = /data/jails/flavours > } > } > > Thanks for your help or any hint. Do I have to add _bbstored to the same > group or sth? > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Feb 20 09:51:27 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 20 Feb 2008 09:51:27 +0000 (GMT) Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: <47BBD836.9030809@my-cooper.de> References: <47BBD836.9030809@my-cooper.de> Message-ID: Hi Benedikt, On Wed, 20 Feb 2008, Benedikt Niessen wrote: > After my first backup I checked the result to see if it worked. But > seems not. I ran the sync-command several times and the Client Store > Marker is changed from 0 to some other value, so seems it is finished. > > This is my configuration for the paths to backup, but e.g. /usr/local is > not complete, /home is emtpy, the jailsfiles are empty... For me it > looks like a permission problem. That would not have caused it to abort. Please check your system logs. What version of Box Backup are you running? Cheers, Chris. -- _ __ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Feb 20 10:30:58 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 11:30:58 +0100 Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: References: <47BBD836.9030809@my-cooper.de> Message-ID: <47BC0162.6020009@my-cooper.de> Hm, I found this: > Exception caught (Connection SocketConnectError (Probably a network > issue between client and server, bad hostname, or server not running.) > 7/15), reset state and waiting to retry... But shouldnt it work for the second try when I run it? I repeated several times... Thanks for your help. Ben. Chris Wilson schrieb: > Hi Benedikt, > > On Wed, 20 Feb 2008, Benedikt Niessen wrote: > >> After my first backup I checked the result to see if it worked. But >> seems not. I ran the sync-command several times and the Client Store >> Marker is changed from 0 to some other value, so seems it is finished. >> >> This is my configuration for the paths to backup, but e.g. /usr/local >> is not complete, /home is emtpy, the jailsfiles are empty... For me it >> looks like a permission problem. > > That would not have caused it to abort. Please check your system logs. > What version of Box Backup are you running? > > Cheers, Chris. From boxbackup at fluffy.co.uk Wed Feb 20 11:14:26 2008 From: boxbackup at fluffy.co.uk (Benedikt Niessen) Date: Wed, 20 Feb 2008 12:14:26 +0100 Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: References: <47BBD836.9030809@my-cooper.de> Message-ID: <47BC0B92.8080700@my-cooper.de> I accidentally removed the 0000001-Directory (1 is the id of my only client) but now I get the following messages even I reinstalled completely new (the bbstored-Server) > bbstored[32451]: SSL err during Load certificates: > error:0200100D:system library:fopen:Permission denied > bbstored[32451]: SSL err during Load > certificates: error:20074002:BIO routines:FILE_CTRL:system lib > bbstored[32451]: SSL err during Load > certificates: error:140DC002:SSL > routines:SSL_CTX_use_certificate_chain_file:system lib > bbstored[32451]: bbstored: terminating due to > exception Server TLSLoadCertificatesFailed (3/25) > bbstored/hk[32452]: bbstored: terminating due to > exception Common OSFileError (Error accessing a file. Check > permissions.) (1/9) I checked all permissions, changed to 777 for files and folders, checked the folders, reinstalled boxbackup-server completely but no use. It just doesnt want to work anymore. I checked for a debugoption but the FreeBSD-Port doesnt offer any and the "ExtensiveLogging" is set to yes but doesnt help at all. For today I will not try to fix it anymore. Maybe somebody knows what is wrong and how to recover the backup-server... Thanks anyway, Ben. Chris Wilson schrieb: > Hi Benedikt, > > On Wed, 20 Feb 2008, Benedikt Niessen wrote: > >> After my first backup I checked the result to see if it worked. But >> seems not. I ran the sync-command several times and the Client Store >> Marker is changed from 0 to some other value, so seems it is finished. >> >> This is my configuration for the paths to backup, but e.g. /usr/local >> is not complete, /home is emtpy, the jailsfiles are empty... For me it >> looks like a permission problem. > > That would not have caused it to abort. Please check your system logs. > What version of Box Backup are you running? > > Cheers, Chris. From boxbackup at fluffy.co.uk Wed Feb 20 17:33:13 2008 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Wed, 20 Feb 2008 09:33:13 -0800 Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: <47BC0B92.8080700@my-cooper.de> References: <47BBD836.9030809@my-cooper.de> <47BC0B92.8080700@my-cooper.de> Message-ID: <47BC6459.1000900@reedtz.com> On 2/20/08 3:14 AM, Benedikt Niessen wrote: > I accidentally removed the 0000001-Directory (1 is the id of my only > client) but now I get the following messages even I reinstalled > completely new (the bbstored-Server) > > > bbstored[32451]: SSL err during Load certificates: > > error:0200100D:system library:fopen:Permission denied > > bbstored[32451]: SSL err during Load > > certificates: error:20074002:BIO routines:FILE_CTRL:system lib > > bbstored[32451]: SSL err during Load > > certificates: error:140DC002:SSL > > routines:SSL_CTX_use_certificate_chain_file:system lib > > bbstored[32451]: bbstored: terminating due to > > exception Server TLSLoadCertificatesFailed (3/25) > > bbstored/hk[32452]: bbstored: terminating due to > > exception Common OSFileError (Error accessing a file. Check > > permissions.) (1/9) > > I checked all permissions, changed to 777 for files and folders, > checked the folders, reinstalled boxbackup-server completely but no > use. It just doesnt want to work anymore. It seems like there is a problem accessing the certificate files. Those are usually kept in /etc/box/bbstored/ca/* Does the _bbstored user have access to those directories and/or files? The config file to look at is /etc/box/bbstored.conf (or wherever you keep it). This has references to the certificate files you seem to be having permission problems with. Take a look at those files... > I checked for a debugoption but the FreeBSD-Port doesnt offer any and > the "ExtensiveLogging" is set to yes but doesnt help at all. I'm sure this is just a typo, but just to be sure, the parameter for extended logging is "ExtendedLogging". Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Wed Feb 20 18:35:30 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 20 Feb 2008 18:35:30 +0000 (GMT) Subject: [Box Backup] Adding files from usb hdds In-Reply-To: References: Message-ID: Hi Timothy, On Wed, 20 Feb 2008, Timothy Wilson wrote: > I now have a problem where I have a couple of remote machines that I > want to move to boxbackup, but they have more data than I care to > download. So rather than sucking down multiple gigabytes of data through > my small internet pipe, I was thinking I could just backup the targets > to a usb hdd, and then plug it into the server to sync off. > > But how to get the data into the right account? The approach I was > thinking of taking, would be to install the backup client on the > server, create a new account for the remote machine, but point the > backup config file's targets to the mounted hdd instead of the remote > machine. However, I think that will muck up the paths; ie it will be > backing up /mnt/hdd/home rather than /home. Is there a way to rename > the paths, or am I going about this the wrong way? Is there a better > approach? It shouldn't mess up any paths because the root directory of the account contains only location names, and locations don't know anything about the real physical path that they're backed up from. So you can set the location path to /mnt/hdd/home in bbackupd.conf on the server, back up on the server, and then configure the client's bbackupd.conf to back up a location with the same name and with path = /home, and it should just work. > I haven't seen anything about this on the wiki :( I've added an account for you on the wiki, I'll email the password separately, please feel free to add it (perhaps under Common Tasks). Cheers, Chris. -- _ __ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Feb 20 18:50:29 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 20 Feb 2008 18:50:29 +0000 (GMT) Subject: [Box Backup] BB not transfering all files/directories In-Reply-To: <47BC6459.1000900@reedtz.com> References: <47BBD836.9030809@my-cooper.de> <47BC0B92.8080700@my-cooper.de> <47BC6459.1000900@reedtz.com> Message-ID: Hi Benedikt and Per, On Wed, 20 Feb 2008, Per Reedtz Thomsen wrote: > On 2/20/08 3:14 AM, Benedikt Niessen wrote: >> I accidentally removed the 0000001-Directory (1 is the id of my only >> client) but now I get the following messages even I reinstalled completely >> new (the bbstored-Server) >> >> > bbstored[32451]: SSL err during Load certificates: >> > error:0200100D:system library:fopen:Permission denied >> > bbstored[32451]: SSL err during Load >> > certificates: error:20074002:BIO routines:FILE_CTRL:system lib >> > bbstored[32451]: SSL err during Load >> > certificates: error:140DC002:SSL >> > routines:SSL_CTX_use_certificate_chain_file:system lib >> > bbstored[32451]: bbstored: terminating due to >> > exception Server TLSLoadCertificatesFailed (3/25) >> > bbstored/hk[32452]: bbstored: terminating due to >> > exception Common OSFileError (Error accessing a file. Check >> > permissions.) (1/9) >> >> I checked all permissions, changed to 777 for files and folders, checked >> the folders, reinstalled boxbackup-server completely but no use. It just >> doesnt want to work anymore. Please check the CertificateFile path in bbstored.conf. Check that you can list the file indicated there even when running as _bbstored user. For example: sudo -u _bbstored ls /etc/box/bbstored/server.name.pem > It seems like there is a problem accessing the certificate files. Those are > usually kept in /etc/box/bbstored/ca/* Actually they are not usually in the "ca" directory, that's for CA files and really they should not even be on the bbstored server at all. See [http://www.boxbackup.org/trac/wiki/CertificatesAndAccountsManagement]: "Setting Up the Certificate Authority... It's best to do this on a machine other than your server, probably without direct network access. The contents of this directory control who can access your backup store server." >> I checked for a debugoption but the FreeBSD-Port doesnt offer any and the >> "ExtensiveLogging" is set to yes but doesnt help at all. I've improved the debugging messages in the trunk to show the actual path that it was trying to access. If you still can't get it to work, please try the latest development version from trunk. See [http://www.boxbackup.org/trac/wiki/SourceCodeRepository] for details. Cheers, Chris. -- _ __ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Feb 20 23:10:34 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 21 Feb 2008 10:10:34 +1100 Subject: [Box Backup] Adding files from usb hdds In-Reply-To: References: Message-ID: Thanks very much Chris! Will do :) On 21/02/2008, Chris Wilson wrote: > Hi Timothy, > > > On Wed, 20 Feb 2008, Timothy Wilson wrote: > > > I now have a problem where I have a couple of remote machines that I > > want to move to boxbackup, but they have more data than I care to > > download. So rather than sucking down multiple gigabytes of data through > > my small internet pipe, I was thinking I could just backup the targets > > to a usb hdd, and then plug it into the server to sync off. > > > > But how to get the data into the right account? The approach I was > > thinking of taking, would be to install the backup client on the > > server, create a new account for the remote machine, but point the > > backup config file's targets to the mounted hdd instead of the remote > > machine. However, I think that will muck up the paths; ie it will be > > backing up /mnt/hdd/home rather than /home. Is there a way to rename > > the paths, or am I going about this the wrong way? Is there a better > > approach? > > > It shouldn't mess up any paths because the root directory of the account > contains only location names, and locations don't know anything about the > real physical path that they're backed up from. So you can set the > location path to /mnt/hdd/home in bbackupd.conf on the server, back up on > the server, and then configure the client's bbackupd.conf to back up a > location with the same name and with path = /home, and it should just > work. > > > > I haven't seen anything about this on the wiki :( > > > I've added an account for you on the wiki, I'll email the password > separately, please feel free to add it (perhaps under Common Tasks). > > > Cheers, Chris. > -- > _ __ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed Feb 20 23:31:12 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 20 Feb 2008 23:31:12 +0000 (GMT) Subject: [Box Backup] Manually deleting files In-Reply-To: <47B03F3E.2030503@3ait.co.uk> References: <4799B1F2.3010800@3ait.co.uk> <62EAD2CB-343A-49D8-9B3E-33172351B1E8@mbrown.co.uk> <4799C489.8090103@3ait.co.uk> <47B03F3E.2030503@3ait.co.uk> Message-ID: Hi Oli, Sorry for the delay in replying. On Mon, 11 Feb 2008, Oli wrote: >> Box Backup 0.11 (rc1) will do a backup run and delete excluded files even >> if the store is full (or at least it should, please let me know if it >> doesn't). > > In case it wanted to delete excluded files AFTER the backup run, I tried > commenting out all my servers config and try a run. I got this: > > Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: NOTICE: 4 redundant > locations in root directory found, will delete from store after 172800 > seconds. > Feb 11 12:12:44 io Box Backup (bbackupd)[26904]: Not deleting unused > entries - too early (172799 seconds remaining) > > Which kinda gives me another way out - I can comment 'earth' and have it > delete it from the store, then put it back in to make it back it up > again. Obviously this is a long way from ideal - That'll copy around 200Mb > > But it's only a solution if I don't have to wait 172799 seconds - ~50 > hours. Can't see anywhere to change this :-( There is a new option, DeleteRedundantLocationsAfter. For example, DeleteRedundantLocationsAfter = 10 will delete unused locations after 10 seconds. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Thu Feb 21 01:13:39 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 21 Feb 2008 12:13:39 +1100 Subject: [Box Backup] Boxbackup user Message-ID: Hello again everyone, I was wondering, is boxbackup hardcoded to use the _bbstored user? I've searched on the wiki, but can't find anything about this. I know that the port is hard coded, so maybe the user is too. The reason is my distribution doesn't seem to like the _ at the beginning. I could force it, but I don't want to break anything else. I'm asking this because I'm getting the raidfile 8/2 error, which is permissions error. But if I su to my bbstored user, I can touch files and make dirs, in the directory specified in my raidfile, so I'm sure it's not a permissions error. Unless it needs access to other things too? Changing the User option to root works, but I would prefer not to run with these permissions. Kind regards, Timothy. From boxbackup at fluffy.co.uk Thu Feb 21 08:45:42 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 21 Feb 2008 08:45:42 +0000 (GMT) Subject: [Box Backup] Boxbackup user In-Reply-To: References: Message-ID: Hi Timothy, On Thu, 21 Feb 2008, Timothy Wilson wrote: > I was wondering, is boxbackup hardcoded to use the _bbstored user? I've > searched on the wiki, but can't find anything about this. I know that > the port is hard coded, so maybe the user is too. The reason is my > distribution doesn't seem to like the _ at the beginning. I could force > it, but I don't want to break anything else. > > I'm asking this because I'm getting the raidfile 8/2 error, which is > permissions error. But if I su to my bbstored user, I can touch files > and make dirs, in the directory specified in my raidfile, so I'm sure > it's not a permissions error. Unless it needs access to other things > too? > > Changing the User option to root works, but I would prefer not to run > with these permissions. The user is not hardcoded. It's stored in the User option in bbstored.conf, as you saw. What file do you get the raidfile 8/2 error on? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Thu Feb 21 17:50:59 2008 From: boxbackup at fluffy.co.uk (Jody Belka) Date: Thu, 21 Feb 2008 17:50:59 +0000 Subject: [Box Backup] Box Backup 0.11rc2 on Windows? Message-ID: <20080221175059.GF7980@pimb.org> Hi all, After building boxbackup -rc2 successfully on a linux installation, I thought I'd try it out with a Windows client. Unfortunately, I didn't have much luck with building it myself on Windows. I was wondering if anyone else might have been successful, and have a build packaged somewhere that they could make available? J -- Jody Belka knew (at) pimb (dot) org From boxbackup at fluffy.co.uk Thu Feb 21 22:26:10 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 09:26:10 +1100 Subject: [Box Backup] Boxbackup user In-Reply-To: References: Message-ID: Hello Chris, It was complaining about a lock file. I removed it, restarted the daemon and it all worked. But the file name wasn't appearing on the console, it was only after I got your message that I thought to look in the logs.. Sorry for such a silly problem. I'm still getting used to box. Kind regards, Timothy. On 21/02/2008, Chris Wilson wrote: > Hi Timothy, > > > On Thu, 21 Feb 2008, Timothy Wilson wrote: > > > I was wondering, is boxbackup hardcoded to use the _bbstored user? I've > > searched on the wiki, but can't find anything about this. I know that > > the port is hard coded, so maybe the user is too. The reason is my > > distribution doesn't seem to like the _ at the beginning. I could force > > it, but I don't want to break anything else. > > > > I'm asking this because I'm getting the raidfile 8/2 error, which is > > permissions error. But if I su to my bbstored user, I can touch files > > and make dirs, in the directory specified in my raidfile, so I'm sure > > it's not a permissions error. Unless it needs access to other things > > too? > > > > Changing the User option to root works, but I would prefer not to run > > with these permissions. > > > The user is not hardcoded. It's stored in the User option in > bbstored.conf, as you saw. > > What file do you get the raidfile 8/2 error on? > > Cheers, Chris. > -- > > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Feb 21 22:34:15 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 09:34:15 +1100 Subject: [Box Backup] Possible date bug? Message-ID: Hello again list, I'm sorry for all the emails I'm sending to the list recently, but as a new user, I'm still getting to grips with box. On the plus side, you got a wiki article out of me...I think I still owe you guys a lot more though for making such an excellent program! Onto my problem. I've noticed that on every single one of my backup client machines, both Windows and Unix, the timestamp in the restore tool is reported as: "1970-01-01T10:00:00", for every file, every folder, everywhere. I looked at the 'know issues' page on the wiki, but I couldn't see anything to do with dates. I'd like to have a more recent backup than 40 odd years ago :) Is there something I need to do to get the dates working correctly? I'm using 0.11RC2 from source on Unix, and Build 1857 for Windows. Thank you very much, Timothy. From boxbackup at fluffy.co.uk Fri Feb 22 00:30:29 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 22 Feb 2008 00:30:29 +0000 (GMT) Subject: [Box Backup] Box Backup 0.11rc2 on Windows? In-Reply-To: <20080221175059.GF7980@pimb.org> References: <20080221175059.GF7980@pimb.org> Message-ID: Hi Jody, On Thu, 21 Feb 2008, Jody Belka wrote: > After building boxbackup -rc2 successfully on a linux installation, I > thought I'd try it out with a Windows client. Unfortunately, I didn't > have much luck with building it myself on Windows. Sorry, Win32 support is not tested yet in rc2 (my fault). I'll get around to it before release. > I was wondering if anyone else might have been successful, and have a > build packaged somewhere that they could make available? Please try one of these: http://www.boxbackup.org/trac/wiki/WindowsClientReleases Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 22 00:32:35 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 22 Feb 2008 00:32:35 +0000 (GMT) Subject: [Box Backup] Boxbackup user In-Reply-To: References: Message-ID: Hi Timothy, On Fri, 22 Feb 2008, Timothy Wilson wrote: > It was complaining about a lock file. I removed it, restarted the daemon > and it all worked. But the file name wasn't appearing on the console, it > was only after I got your message that I thought to look in the logs.. I thought I'd fixed all places where an error like that could happen without reporting it on the console. Which version are you running and what was the error in the logs (including surrounding lines)? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 22 00:34:47 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 22 Feb 2008 00:34:47 +0000 (GMT) Subject: [Box Backup] Possible date bug? In-Reply-To: References: Message-ID: Hi Timothy, On Fri, 22 Feb 2008, Timothy Wilson wrote: > I'm sorry for all the emails I'm sending to the list recently, but as a > new user, I'm still getting to grips with box. On the plus side, you got > a wiki article out of me...I think I still owe you guys a lot more > though for making such an excellent program! Onto my problem. > > I've noticed that on every single one of my backup client machines, both > Windows and Unix, the timestamp in the restore tool is reported as: > "1970-01-01T10:00:00", for every file, every folder, everywhere. I > looked at the 'know issues' page on the wiki, but I couldn't see > anything to do with dates. I'd like to have a more recent backup than 40 > odd years ago :) Timestamps are not stored for directories, but if you get that on files as well, please tell me which Unix platform and which version of Windows you're running on? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 22 00:42:38 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 11:42:38 +1100 Subject: [Box Backup] Box Backup 0.11rc2 on Windows? In-Reply-To: References: <20080221175059.GF7980@pimb.org> Message-ID: Hello Jody, If it's not too urgent, I'll have a Windows client how to on the wiki next week, if you can wait. I've got some notes on what I did to get it working, but it needs cleaning up before I can post it. Stay tuned! Timothy. On 22/02/2008, Chris Wilson wrote: > Hi Jody, > > > On Thu, 21 Feb 2008, Jody Belka wrote: > > > After building boxbackup -rc2 successfully on a linux installation, I > > thought I'd try it out with a Windows client. Unfortunately, I didn't > > have much luck with building it myself on Windows. > > > Sorry, Win32 support is not tested yet in rc2 (my fault). I'll get around > to it before release. > > > > I was wondering if anyone else might have been successful, and have a > > build packaged somewhere that they could make available? > > > Please try one of these: > > > http://www.boxbackup.org/trac/wiki/WindowsClientReleases > > > Cheers, Chris. > > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Feb 21 22:17:43 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 09:17:43 +1100 Subject: [Box Backup] Box Backup 0.11rc2 on Windows? In-Reply-To: <20080221175059.GF7980@pimb.org> References: <20080221175059.GF7980@pimb.org> Message-ID: Hello Jody, There is a link to a binary here: http://www.boxbackup.org/trac/wiki/WindowsClientReleases Just get the most recent build. I have it working successfully on 2 Windows clients, and I'm going to add more over the next few days. You should always check out the wiki! It's has so much good stuff in it, you must write down / tattoo-into-your-arm / bookmark the url. Kind regards, Timothy On 22/02/2008, Jody Belka wrote: > Hi all, > > After building boxbackup -rc2 successfully on a linux installation, I thought > I'd try it out with a Windows client. Unfortunately, I didn't have much luck > with building it myself on Windows. > > I was wondering if anyone else might have been successful, and have a build > packaged somewhere that they could make available? > > > > J > > -- > Jody Belka > knew (at) pimb (dot) org > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Fri Feb 22 00:40:26 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 11:40:26 +1100 Subject: [Box Backup] Possible date bug? In-Reply-To: References: Message-ID: Hello Chris, Thanks for your reply! Is there any reason why directories don't have a timestamp? Is there a technical reason? A planned feature? Or just a case of "it's not quite done yet" :) I don't want to push you! I would like to know when my most recent backup was, and generally I just look at the timestamps of the directories. But obviously I need to find another way with boxbackup. Kind regards, Timothy. On 22/02/2008, Chris Wilson wrote: > Hi Timothy, > > > On Fri, 22 Feb 2008, Timothy Wilson wrote: > > > I'm sorry for all the emails I'm sending to the list recently, but as a > > new user, I'm still getting to grips with box. On the plus side, you got > > a wiki article out of me...I think I still owe you guys a lot more > > though for making such an excellent program! Onto my problem. > > > > I've noticed that on every single one of my backup client machines, both > > Windows and Unix, the timestamp in the restore tool is reported as: > > "1970-01-01T10:00:00", for every file, every folder, everywhere. I > > looked at the 'know issues' page on the wiki, but I couldn't see > > anything to do with dates. I'd like to have a more recent backup than 40 > > odd years ago :) > > > Timestamps are not stored for directories, but if you get that on files > as well, please tell me which Unix platform and which version of Windows > you're running on? > > Cheers, Chris. > -- > > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Feb 21 22:58:02 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Fri, 22 Feb 2008 09:58:02 +1100 Subject: [Box Backup] Boxbackup user In-Reply-To: References: Message-ID: Whoops, bbstored starts, but I can't run bbackupquery on the clients! I'm getting this error message: root at userver:/backups# bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf check 2 WARNING: Failed to open lockfile: /backups/boxbackup/backup/00000002/write.lock WARNING: Exception thrown: CommonException(OSFileError) at NamedLock.cpp(136) Exception: Common OSFileError (Error accessing a file. Check permissions.) (1/9) But the permissions seem to be correct? root at userver:/backups# ls -l boxbackup/backup/00000002 | grep write -rw-r--r-- 1 bbstored root 0 2008-02-22 09:46 write.lock Initially bbstored was complaining about permission errors, so I ran it as root because I didn't get it...it had the correct permissions!, or so I thought. It couldn't get from / to the backup dir. So once I worked this out, I did a chown, and it can get to the write directory: I gave bbstored user a working shell, and cd'd one by one to make sure. It can even touch files, so I can't see what the problem is. But because I ran it as root, and now want to switch back to bbstored user, have I totally messed things up? Kind regards, Timothy. On 22/02/2008, Timothy Wilson wrote: > Hello Chris, > > It was complaining about a lock file. I removed it, restarted the > daemon and it all worked. But the file name wasn't appearing on the > console, it was only after I got your message that I thought to look > in the logs.. > > Sorry for such a silly problem. I'm still getting used to box. > > Kind regards, > > Timothy. > > > On 21/02/2008, Chris Wilson wrote: > > Hi Timothy, > > > > > > On Thu, 21 Feb 2008, Timothy Wilson wrote: > > > > > I was wondering, is boxbackup hardcoded to use the _bbstored user? I've > > > searched on the wiki, but can't find anything about this. I know that > > > the port is hard coded, so maybe the user is too. The reason is my > > > distribution doesn't seem to like the _ at the beginning. I could force > > > it, but I don't want to break anything else. > > > > > > I'm asking this because I'm getting the raidfile 8/2 error, which is > > > permissions error. But if I su to my bbstored user, I can touch files > > > and make dirs, in the directory specified in my raidfile, so I'm sure > > > it's not a permissions error. Unless it needs access to other things > > > too? > > > > > > Changing the User option to root works, but I would prefer not to run > > > with these permissions. > > > > > > The user is not hardcoded. It's stored in the User option in > > bbstored.conf, as you saw. > > > > What file do you get the raidfile 8/2 error on? > > > > Cheers, Chris. > > -- > > > > _____ __ _ > > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > From boxbackup at fluffy.co.uk Fri Feb 22 12:50:30 2008 From: boxbackup at fluffy.co.uk (Jody Belka) Date: Fri, 22 Feb 2008 12:50:30 +0000 Subject: [Box Backup] Box Backup 0.11rc2 on Windows? In-Reply-To: References: <20080221175059.GF7980@pimb.org> Message-ID: <20080222125030.GG7980@pimb.org> On Fri, Feb 22, 2008 at 09:17:43AM +1100, Timothy Wilson wrote: > There is a link to a binary here: > http://www.boxbackup.org/trac/wiki/WindowsClientReleases > Just get the most recent build. I have it working successfully on 2 > Windows clients, and I'm going to add more over the next few days. > > You should always check out the wiki! It's has so much good stuff in > it, you must write down / tattoo-into-your-arm / bookmark the url. I had found that page actually, but since the most recent build listed is from September of last year I'd ignored it previously. Is it worth trying, or should I wait for a bit? J -- Jody Belka knew (at) pimb (dot) org From boxbackup at fluffy.co.uk Fri Feb 22 19:15:29 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 22 Feb 2008 19:15:29 +0000 (GMT) Subject: [Box Backup] Possible date bug? In-Reply-To: References: Message-ID: Hi Timothy, On Fri, 22 Feb 2008, Timothy Wilson wrote: > Thanks for your reply! Is there any reason why directories don't have a > timestamp? Is there a technical reason? A planned feature? Or just a > case of "it's not quite done yet" :) I don't want to push you! > > I would like to know when my most recent backup was, and generally I > just look at the timestamps of the directories. But obviously I need to > find another way with boxbackup. This is what Ben said about the problem when I asked him: >> Ben, why is it that we zero the modification time on directories? Just >> because we don't diff/patch them? > > mtime on a directory is not useful, and would result in lots of updates > to metadata in the parent directory on the store. If we wanted to store directory timestamps, we could probably work around the problem of metadata updates on the parent by attaching new attributes to the directory itself. It might be useful to do this whenever a directory's contents on the store are updated, to indicate the recentness of a directory, which I think is what you're asking for. But nobody has proposed implementing this yet. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri Feb 22 19:27:17 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 22 Feb 2008 19:27:17 +0000 (GMT) Subject: [Box Backup] Box Backup 0.11rc2 on Windows? In-Reply-To: <20080222125030.GG7980@pimb.org> References: <20080221175059.GF7980@pimb.org> <20080222125030.GG7980@pimb.org> Message-ID: Hi Jody, On Fri, 22 Feb 2008, Jody Belka wrote: >> There is a link to a binary here: >> http://www.boxbackup.org/trac/wiki/WindowsClientReleases >> Just get the most recent build. I have it working successfully on 2 >> Windows clients, and I'm going to add more over the next few days. >> >> You should always check out the wiki! It's has so much good stuff in >> it, you must write down / tattoo-into-your-arm / bookmark the url. > > I had found that page actually, but since the most recent build listed > is from September of last year I'd ignored it previously. Is it worth > trying, or should I wait for a bit? Box Backup development is not that fast :-) It's the latest released build for Windows. It's almost 0.11. Try it. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Sat Feb 23 23:59:30 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Sun, 24 Feb 2008 10:59:30 +1100 Subject: [Box Backup] Possible date bug? In-Reply-To: References: Message-ID: Hello Chris, I appreciate you taking the time to ask Ben. Out of curiosity, what is the best way to determine how recent a backup is? I mean, even if it's something like just ls -l on the backup directory in the shell, as long as it give me an idea. Or is it just better to view the logs, to see what's failing (if anything). The reason I'm concerned is that I use boxbackup with remote sites, and the internet lines here in Australia varies from almost reliable to pathetic, so my current system of rsyncing directories with a cronjob often fails silently. I feel 98% confident that every machine I have on box (currently about 6, with more to come) can recover fully from a disaster, so this is a huge improvement from my current system. But that last 2% would be to know exactly when a backup occurred (or didn't). But if it's all in the logs, I'll go about reading them each morning :) All the best, Timothy. On 23/02/2008, Chris Wilson wrote: > Hi Timothy, > > On Fri, 22 Feb 2008, Timothy Wilson wrote: > > > > Thanks for your reply! Is there any reason why directories don't have a > > timestamp? Is there a technical reason? A planned feature? Or just a > > case of "it's not quite done yet" :) I don't want to push you! > > > > I would like to know when my most recent backup was, and generally I > > just look at the timestamps of the directories. But obviously I need to > > find another way with boxbackup. > > > This is what Ben said about the problem when I asked him: > > >> Ben, why is it that we zero the modification time on directories? Just > >> because we don't diff/patch them? > > > > mtime on a directory is not useful, and would result in lots of updates > > to metadata in the parent directory on the store. > > If we wanted to store directory timestamps, we could probably work around > the problem of metadata updates on the parent by attaching new attributes > to the directory itself. It might be useful to do this whenever a > directory's contents on the store are updated, to indicate the recentness > of a directory, which I think is what you're asking for. But nobody has > proposed implementing this yet. > > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Mon Feb 25 00:36:30 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 25 Feb 2008 00:36:30 +0000 (GMT) Subject: [Box Backup] Possible date bug? In-Reply-To: References: Message-ID: Hi Timothy, On Sun, 24 Feb 2008, Timothy Wilson wrote: > I appreciate you taking the time to ask Ben. No need, it was a copy of an answer that he provided several months ago when I asked him that question about timestamps on directories. > Out of curiosity, what is the best way to determine how recent a backup > is? I mean, even if it's something like just ls -l on the backup > directory in the shell, as long as it give me an idea. Unfortunately it's not so simple. Box Backup doesn't have the idea of a snapshot that encompasses all files at a particular date. It would conflict with not uploading files which have changed too recently, and also with resuming an interrupted backup. There are plans to implement a true snapshot behaviour in future, but at the moment, there is no easy way to find out when the last backup was run by looking at the server alone, only the most recent backup of each individual file. You could recursively list all files and pick the most recent file timestamp, but that's a lot of work. > Or is it just better to view the logs, to see what's failing (if > anything). I would always advise doing that if you suspect a problem. Box Backup client should notify you using the NotifyScript if a backup fails. Combine with "bbackupctl sync" to force a backup to occur at scheduled times, and therefore a notification if one of these fails. > The reason I'm concerned is that I use boxbackup with remote sites, and > the internet lines here in Australia varies from almost reliable to > pathetic, so my current system of rsyncing directories with a cronjob > often fails silently. rsync should not fail silently, it should at least generate an error if the backup does not complete. Box Backup definitely should, and if it doesn't notify you then I want to know about it. > I feel 98% confident that every machine I have on box (currently about > 6, with more to come) can recover fully from a disaster, so this is a > huge improvement from my current system. But that last 2% would be to > know exactly when a backup occurred (or didn't). Are you running regular scheduled compares? If not, then you should not have confidence in your backups, whatever the method, but especially not with beta software like Box Backup. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Wed Feb 27 01:51:50 2008 From: boxbackup at fluffy.co.uk (Alex Harper) Date: Tue, 26 Feb 2008 17:51:50 -0800 Subject: [Box Backup] File diff inverts block probability? BackupStoreFileDiff.cpp:515 Message-ID: Hi, I've been trying to understand Box's slow diff of some large files I'm backing up (compared with rsync). BackupStoreFileDiff.cpp:515 runs a loop of the various server side block sizes in reverse order, with the comment that its running from smallest to largest. I suspect this is no longer necessary, as it contradicts the comments at line 858. FindMostUsedSizes() seems to produce a block size array in block-size-frequency order, not size order. Thus running the loop in reverse would run from least common block size to most common block size. For a mostly unchanged file stored mostly as large blocks with just a few small blocks, this seems to eliminate most of the goodnessOfFit[] short-circuiting (there is a separate optimization miss with goodnessOfFit, but I'll leave that for another day). Reversing the loop seems logical, passes unittests and seems to work for me (so far, not a lot of testing). Speedup is 15-20% for my diffs. Is this a reasonable read of the code? Have I misunderstood something fundamental about the relationship between local block scans and server-side storage? Alex -- Alex Harper aharper at foobox.net "Generally speaking, things have gone about as far as they can reasonably go when things have got about as bad as they can reasonably get." - Tom Stoppard From boxbackup at fluffy.co.uk Wed Feb 27 13:50:51 2008 From: boxbackup at fluffy.co.uk (David Bamford) Date: Wed, 27 Feb 2008 13:50:51 -0000 (GMT) Subject: [Box Backup] running bbackupquery in a browser Message-ID: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> Hi I have managed to get a web page to work as a means of navigating through files and folders on the store using php to exec bbackupquery commands. But I have failed to get the "get" command to work to retrieve a file. I build a command like this /usr/local/bin/bbackupquery "cd \"/MyDocuments\"" "get \"Backup.bat\"" exit and it works fine from a terminal window, but if I try to exec it from the web page I get Error occured fetching file. Logging off... Exception: Connection Protocol_ObjTooBig (7/42) I have tried to give it a destination file, and to fetch by object id, but while the commands work fine from the command line they always fail from the web page. there is nothing in the error_log and php safe_mode is off. The configuration file is read correctly, and exec on other commands like "list" work very well. any help would be appreciated as this gives me the ability to browse and retrieve files graphically. Cheers Dave Bamford From boxbackup at fluffy.co.uk Wed Feb 27 16:23:15 2008 From: boxbackup at fluffy.co.uk (Pablo Fernandez) Date: Wed, 27 Feb 2008 14:23:15 -0200 Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> Message-ID: <1204129395.6630.5.camel@laptop> Hi Are you going to open source that? It would be very useful. Best regards, Pablo On Wed, 2008-02-27 at 13:50 +0000, David Bamford wrote: > Hi > I have managed to get a web page to work as a means of navigating through > files and folders on the store using php to exec bbackupquery commands. > > But I have failed to get the "get" command to work to retrieve a file. > > I build a command like this > > /usr/local/bin/bbackupquery "cd \"/MyDocuments\"" "get \"Backup.bat\"" exit > > and it works fine from a terminal window, but if I try to exec it from the > web page I get > Error occured fetching file. Logging off... Exception: Connection > Protocol_ObjTooBig (7/42) > > I have tried to give it a destination file, and to fetch by object id, but > while the commands work fine from the command line they always fail from > the web page. there is nothing in the error_log and php safe_mode is off. > > The configuration file is read correctly, and exec on other commands like > "list" work very well. > > any help would be appreciated as this gives me the ability to browse and > retrieve files graphically. > > Cheers > > Dave Bamford > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Feb 27 16:42:15 2008 From: boxbackup at fluffy.co.uk (David Bamford) Date: Wed, 27 Feb 2008 16:42:15 -0000 (GMT) Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <1204129395.6630.5.camel@laptop> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <1204129395.6630.5.camel@laptop> Message-ID: <20084.87.194.161.98.1204130535.squirrel@webmail.officenetworksystems.co.uk> ------=_20080227164215_93318 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit No problem code attached- bugs and all. Its very simple, just remove the bit that tried to get files back and you can use it to browse the store. You need a valid config file and keys of course. Dave > Hi > > Are you going to open source that? It would be very useful. > > Best regards, > > Pablo > > > On Wed, 2008-02-27 at 13:50 +0000, David Bamford wrote: >> Hi >> I have managed to get a web page to work as a means of navigating >> through >> files and folders on the store using php to exec bbackupquery commands. >> >> But I have failed to get the "get" command to work to retrieve a file. >> >> I build a command like this >> >> /usr/local/bin/bbackupquery "cd \"/MyDocuments\"" "get \"Backup.bat\"" >> exit >> >> and it works fine from a terminal window, but if I try to exec it from >> the >> web page I get >> Error occured fetching file. Logging off... Exception: Connection >> Protocol_ObjTooBig (7/42) >> >> I have tried to give it a destination file, and to fetch by object id, >> but >> while the commands work fine from the command line they always fail from >> the web page. there is nothing in the error_log and php safe_mode is >> off. >> >> The configuration file is read correctly, and exec on other commands >> like >> "list" work very well. >> >> any help would be appreciated as this gives me the ability to browse and >> retrieve files graphically. >> >> Cheers >> >> Dave Bamford >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > ------=_20080227164215_93318 Content-Type: application/x-php; name="index.php" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="index.php" PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtTGFuZ3VhZ2UiIGNvbnRl bnQ9ImVuLWdiIj4NCjx0aXRsZT5ib3ggYmFja3VwIHF1ZXJ5PC90aXRsZT4NCjwvaGVhZD4NCg0K PGJvZHk+DQo8P3BocA0KLy9wcm9ncmFtIHRvIGV4ZWN1dGUgYmFja3VwIGxpc3QgY29tbWFuZA0K aWYgKGlzc2V0KCRfR0VUWyd3ZCddKSl7DQoJZWNobyBleGVjKCcvdXNyL2xvY2FsL2Jpbi9iYmFj a3VwcXVlcnkgImxpc3QgLXQgXCInLiRfR0VUWyd3ZCddLidcIiIgZXhpdCcsICRyZXR2YWwpOw0K fQ0KZWxzZQl7DQoJaWYgKGlzc2V0KCRfR0VUWydmaWxlJ10pKXsNCgkJZWNobyAoJy91c3IvbG9j YWwvYmluL2JiYWNrdXBxdWVyeSAiY2QgXCInLiRfR0VUWydjZCddLidcIiIgImdldCBcIicuJF9H RVRbJ2ZpbGUnXS4nXCIiIGV4aXQnKTsNCgkJZWNobyBleGVjKCcvdXNyL2xvY2FsL2Jpbi9iYmFj a3VwcXVlcnkgImNkIFwiJy4kX0dFVFsnY2QnXS4nXCIiICJnZXQgLWkgMDAwMDAwMDcgL2hvbWUv ZGF2ZS9CYWNrdXAuYmF0IiBleGl0Jyk7IC8vLCRyZXR2YWwpOw0KLy8JCWVjaG8gZXhlYygnL3Vz ci9sb2NhbC9iaW4vYmJhY2t1cHF1ZXJ5ICJjZCBcIicuJF9HRVRbJ2NkJ10uJ1wiIiAiZ2V0IFwi Jy4kX0dFVFsnZmlsZSddLidcIiAvaG9tZS9kYXZlL3gueCIgZXhpdCcsICRyZXR2YWwpOw0KCQkk aT0xOw0KCQl3aGlsZSAoJHJldHZhbFskaV0gIT0gIlNlc3Npb24gZmluaXNoZWQuIiAmJiAkaSAh PSAyMCkJew0KCQllY2hvICgnPGJyLz4nLiRyZXR2YWxbJGldKTsNCgkJJGkrKzsNCgkJfQkNCgll eGl0KCk7DQoJfQ0KCWVsc2UJew0KCQllY2hvIGV4ZWMoJy91c3IvbG9jYWwvYmluL2JiYWNrdXBx dWVyeSAibGlzdCAtdCIgZXhpdCcsICRyZXR2YWwpOw0KCX0NCn0NCiRpPTE7DQpXaGlsZSAoJHJl dHZhbFskaV0gIT0gIlNlc3Npb24gZmluaXNoZWQuIikJew0KCS8vIHNraXAgZmlyc3QgOSBsaW5l cyB0aGVuIHB1dCBwYXJlbnQgZm9sZGVyDQoJaWYgKCRpPT05KQl7DQoJCWVjaG8gJzxici8+PGEg aHJlZj0iaW5kZXgucGhwP3dkPScuc3Vic3RyKCRfR0VUWyd3ZCddLDAsc3RycnBvcygkX0dFVFsn d2QnXSwiLyIpKS4nIj5QYXJlbnQgRm9sZGVyPC9hPic7DQoJfQ0KCS8vIGlmIGRpcmVjdG9yeSBm bGFnIGlzIHNldCBza2lwIGZpcnN0IDM2IGNoYXJzIGFuZCBhcHBlbmQgZGlyZWN0b3J5IHRvIGN1 cnJlbnQNCglpZiAoc3Vic3RyKCRyZXR2YWxbJGldLDEwLDEpID09ICdkJyl7DQoJCWVjaG8gJzxi ci8+PGEgaHJlZj0iaW5kZXgucGhwP3dkPScuJF9HRVRbJ3dkJ10uJy8nLnN1YnN0cigkcmV0dmFs WyRpXSwzNikuJyI+Jy5zdWJzdHIoJHJldHZhbFskaV0sMzYpLic8L2E+JzsNCgl9DQoJZWxzZQl7 CS8vIGxldCB0aGUgdXNlciByZWNvdmVyIHRoZSBmaWxlDQoJCWVjaG8gJzxici8+PGEgaHJlZj0i aW5kZXgucGhwP2NkPScuJF9HRVRbJ3dkJ10uJyZmaWxlPScuc3Vic3RyKCRyZXR2YWxbJGldLDM2 KS4nIj4nLiRyZXR2YWxbJGldLic8L2E+JzsNCgl9DQoJJGkrKzsNCn0NCj8+DQo8L2JvZHk+PC9o dG1sPg0K ------=_20080227164215_93318-- From boxbackup at fluffy.co.uk Wed Feb 27 17:14:35 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Wed, 27 Feb 2008 18:14:35 +0100 Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> Message-ID: <47C59A7B.2060303@trexler.at> David Bamford schrieb: > Hi > I have managed to get a web page to work as a means of navigating through > files and folders on the store using php to exec bbackupquery commands. Hi David, there is (was?) a similar project called "Boxbackup on rails" mentioned on this list exactly a year ago (-1 day) by Stefan Norlin. See: http://lists.warhead.org.uk/pipermail/boxbackup/2007-February/003277.html I don't know the status of that project, but it could make sense to combine your efforts. br Wolfgang From boxbackup at fluffy.co.uk Wed Feb 27 18:01:24 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Wed, 27 Feb 2008 18:01:24 +0000 Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <47C59A7B.2060303@trexler.at> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <47C59A7B.2060303@trexler.at> Message-ID: <20080227180124.GC15350@netinertia.co.uk> On Wed, Feb 27, 2008 at 06:14:35PM +0100, Wolfgang Trexler wrote: > there is (was?) a similar project called "Boxbackup on rails" mentioned on > this list exactly a year ago (-1 day) by Stefan Norlin. > > See: > http://lists.warhead.org.uk/pipermail/boxbackup/2007-February/003277.html > > I don't know the status of that project, but it could make sense to combine > your efforts. Ben also contributed a web app framework to the project about a year ago, which no one has done anything with yet... http://www.boxbackup.org/trac/browser/box/features/codeforintegration/docs/webappframework James From boxbackup at fluffy.co.uk Wed Feb 27 20:00:01 2008 From: boxbackup at fluffy.co.uk (Dave Bamford) Date: Wed, 27 Feb 2008 20:00:01 +0000 Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <20080227180124.GC15350@netinertia.co.uk> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <47C59A7B.2060303@trexler.at> <20080227180124.GC15350@netinertia.co.uk> Message-ID: <47C5C141.1070208@logical-progress.com> James O'Gorman wrote: > On Wed, Feb 27, 2008 at 06:14:35PM +0100, Wolfgang Trexler wrote: > >> there is (was?) a similar project called "Boxbackup on rails" mentioned on >> this list exactly a year ago (-1 day) by Stefan Norlin. >> >> See: >> http://lists.warhead.org.uk/pipermail/boxbackup/2007-February/003277.html >> >> I don't know the status of that project, but it could make sense to combine >> your efforts. >> > > Ben also contributed a web app framework to the project about a year > ago, which no one has done anything with yet... > > http://www.boxbackup.org/trac/browser/box/features/codeforintegration/docs/webappframework > > James I had a look at the Ruby on Rails at the time, but I don't think the source was available. I am looking for a quick and easy way of browsing and recovering files and I got stuck as to why I got the error. Its probably PHP related, perhaps I ought to try a PHP mailing list. I suspect I need some PHP code, maybe stream functions, to handle the file IO from bbackupquery, but it is way beyond me. Tried exec system and shell_exec. Had a look at the docs on popen but... Perhaps I need to set some environment variables. Dave Bamford From boxbackup at fluffy.co.uk Wed Feb 27 22:58:55 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 27 Feb 2008 22:58:55 +0000 (GMT) Subject: [Box Backup] File diff inverts block probability? BackupStoreFileDiff.cpp:515 In-Reply-To: References: Message-ID: Hi Alex, On Tue, 26 Feb 2008, Alex Harper wrote: > I've been trying to understand Box's slow diff of some large files I'm > backing up (compared with rsync). > > BackupStoreFileDiff.cpp:515 runs a loop of the various server side block > sizes in reverse order, with the comment that its running from smallest to > largest. I suspect this is no longer necessary, as it contradicts the > comments at line 858. > > FindMostUsedSizes() seems to produce a block size array in > block-size-frequency order, not size order. Thus running the loop in > reverse would run from least common block size to most common block > size. For a mostly unchanged file stored mostly as large blocks with > just a few small blocks, this seems to eliminate most of the > goodnessOfFit[] short-circuiting (there is a separate optimization miss > with goodnessOfFit, but I'll leave that for another day). > > Reversing the loop seems logical, passes unittests and seems to work for > me (so far, not a lot of testing). Speedup is 15-20% for my diffs. > > Is this a reasonable read of the code? Have I misunderstood something > fundamental about the relationship between local block scans and > server-side storage? Many thanks for looking into this and posting such a detailed and helpful report. I'm afraid I didn't write that code and I don't know it as well as I should, and I'm afraid I don't completely understand how it works. In particular, I have no idea whether scanning least or most frequently used block sizes first should actually minimise the amount of data to be transferred. I can understand reasons for scanning largest blocks first, but the original code didn't appear to do that either, quite the opposite. Given the limit on the number of matching blocks that we'll use in the recipe, I'd say it makes most sense to scan largest block sizes first. I'm afraid I can't either refute or support your assertion that changing the loop order results in faster diffs or transfers at this time. I'm not against doing it, although I'd like to see the new code tested heavily before going into a release. We currently re-read the file once for each block size that we intend to try. If we started by reading the most commonly-used block sizes instead of the least, as you propose, then potentially we could find all our matches in most commonly-used block sizes and reduce the number of times that we have to re-read the file (i.e. minimise local disk I/O). If I understand you correctly, the change you propose has a negligible or uncertain impact on network bandwidth use, and does reduce disk I/O, so I can see that as an argument in favour of your suggestion. But I think it would be even better to calculate weak checksums for all block sizes at the same time, during a single pass, and then make a second pass to identify which of those are real second-level checksum matches. The main disadvantage would be potentially storing a lot more matches in memory until the recipe is computed. I also think that this code should be rewritten to be optimised for readability :-) Then we could all argue about ways to improve it on the basis of a real understanding of what it actually does now :-) Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Thu Feb 28 00:17:13 2008 From: boxbackup at fluffy.co.uk (Alex Harper) Date: Wed, 27 Feb 2008 16:17:13 -0800 Subject: [Box Backup] File diff inverts block probability? BackupStoreFileDiff.cpp:515 In-Reply-To: Message-ID: Hi Chris, > I'm afraid I can't either refute or support your assertion that changing > the loop order results in faster diffs or transfers at this time. I'm not > against doing it, although I'd like to see the new code tested heavily > before going into a release. Of course, and to be clear, I'm not proposing this be included in 0.11. Its too fundamental to the reliability of the system. At this point all I'm trying to decide is if I've totally misunderstood the code. It seems I'm on the right track. However, it does bring up an interesting question. Lets say I had a patch to this code (like this loop change). You can assume it passes unittests, however, I already know that Box's unittests, while very impressive, don't cover all corner cases. I'm also just selfish enough with my free time to not want to have to take on the job of making the file diff unittests completely comprehensive :) What's a reasonable expectation for testing the patch? Do individual well-known active developer branches (like yours) have sufficient numbers of volunteer users that good coverage happens? > We currently re-read the file once for each block size that we intend to > try. If we started by reading the most commonly-used block sizes instead > of the least, as you propose, then potentially we could find all our > matches in most commonly-used block sizes and reduce the number of times > that we have to re-read the file (i.e. minimise local disk I/O). > > If I understand you correctly, the change you propose has a negligible or > uncertain impact on network bandwidth use, and does reduce disk I/O, so I > can see that as an argument in favour of your suggestion. Not quite. I was targeting a smaller fix at first, just a reversal of the loop. In the current implementation reading in block probability order (not always largest to smallest) doesn't actually change the IO profile. It still reads every byte at every block size. What it changes is that as block size probability goes down you spend more time in RollingChecksum:RollForwardSeveral() rolls (past all previously found big and high-probability blocks). Since RollingChecksum is less expensive than the remaining table looks and MD5 calcs the total wall time is reduced. There is another optimization miss where goodnessOfFit rolls you to a new offset that itself already has a better goodnessOfFit (goodnessOfFit check should loop) but that's a larger change, and I chose to roll it into a broader fix. > But I think it would be even better to calculate weak checksums for all > block sizes at the same time, during a single pass, and then make a second > pass to identify which of those are real second-level checksum matches. > The main disadvantage would be potentially storing a lot more matches in > memory until the recipe is computed. In fact, this is the core of my issue. In my problem case the file is a 1.5GB mail database stored on an encrypted filesystem. The CPU and IO load of the repeated file reads is huge. Minmum 15 minutes to diff with both cores running 100% and my laptop fans screaming. When the file content gets purged from file cache by other machine usage it gets even worse. Capping diff time isn't a good option since my goal is to not upload 1.5GB every time. I agree a one-read-pass diff is the right solution, and the comment at line 488 seems to validate that assumption. What I left out of my previous message is that I was looking at fixing the IO issue as well. I didn't mention it then because I don't want to overpromise, but my first draft passes unittests and runs the same diff 2.5 times faster. Unfortunately, right now it also has readability issues and produces different (seemingly correct) recipes versus the old code (making it hard to validate). I'll continue polishing, and assuming my understanding of block diff optimization (including block ordering) is correct I may have something worth talking about eventually. However, its the kind of thing that would require careful review by someone who understands BackupStoreFileDiff.cpp, better than I do. I'm hoping a volunteer will emerge :) Alex -- Alex Harper aharper at foobox.net "Generally speaking, things have gone about as far as they can reasonably go when things have got about as bad as they can reasonably get." - Tom Stoppard From boxbackup at fluffy.co.uk Thu Feb 28 09:00:34 2008 From: boxbackup at fluffy.co.uk (dave bamford) Date: Thu, 28 Feb 2008 09:00:34 +0000 Subject: [Box Backup] running bbackupquery in a browser (problem solved) In-Reply-To: <47C5C141.1070208@logical-progress.com> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <47C59A7B.2060303@trexler.at> <20080227180124.GC15350@netinertia.co.uk> <47C5C141.1070208@logical-progress.com> Message-ID: <47C67832.5040805@logical-progress.com> Turned out to be a protection issue. Apache didn't have the rights to write the file. It was the error message that misled me. Hopefully I can publish the php source soon. Thanks Dave Bamford From boxbackup at fluffy.co.uk Thu Feb 28 09:27:54 2008 From: boxbackup at fluffy.co.uk (Jarkko Haapalainen) Date: Thu, 28 Feb 2008 11:27:54 +0200 Subject: [Box Backup] running bbackupquery in a browser (problem solved) In-Reply-To: <47C67832.5040805@logical-progress.com> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <47C59A7B.2060303@trexler.at> <20080227180124.GC15350@netinertia.co.uk> <47C5C141.1070208@logical-progress.com> <47C67832.5040805@logical-progress.com> Message-ID: <14F0AE09-11FB-4555-9326-B858B396E26D@viidakko.fi> --Apple-Mail-9-397892027 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Hi, Tried to sent this some time ago, but here again. I made quick version of mine for browsing and restoring files from browser so look at the attached file. It only took ~1,5 hours so there is probably bugs, but is working nicely for me in my mac ;) Best regards, Jarkko --Apple-Mail-9-397892027 Content-Disposition: attachment; filename=bbackupquery.php Content-Type: text/php; x-unix-mode=0644; name="bbackupquery.php" Content-Transfer-Encoding: 7bit $path, 'files' => $matches); } function arr_to_html($m) { $html = ''; $path = $m['path']; foreach ($m['files'] as $object) { list ($line, $id, $flags, $datetime, $name) = $object; if (strpos($flags, 'd') === 1) { $link = sprintf('%s', "$path/$name", $name); } else { $link = sprintf('%s', "$path", $id, $name, $name); } $html .= "$id$flags$datetime$link"; } return $html; } function get_file($bquery, $chown, $tmpdir, $path, $file, $filename) { $new = "$tmpdir/".md5($file); $cmd = sprintf('%s \'cd "%s"\' \'get -i %s %s\' quit', $bquery, rawurldecode($path), rawurldecode($file), $new); $out = shell_exec($cmd); if (file_exists($new)) { exec("$chown '$new'"); $fp = fopen($new, 'rb'); header("Pragma: public"); header('Expires: '.gmdate('D, d M Y H:i:s').' GMT'); header("Cache-Control: private",false); header("Cache-Control: must-revalidate, post-check=0, pre-check=0"); header("Content-Type: application/octet-stream"); header("Content-Disposition: attachment; filename=\"".rawurldecode(utf8_decode($filename))."\""); header("Content-Length: " . filesize($new)); fpassthru($fp); unlink($new); exit; } else { echo "
ERROR: file restore failed\n";
		var_dump($cmd, $path, $file);
		echo "
"; } } if (!empty($_GET['get'])) { get_file($bquery, $chown, $tmpdir, $_GET['cwd'], $_GET['get'], $_GET['name']); exit; } ?> bbackupquery wrapper
id flags datetime name
--Apple-Mail-9-397892027 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit --Apple-Mail-9-397892027-- From boxbackup at fluffy.co.uk Thu Feb 28 08:30:34 2008 From: boxbackup at fluffy.co.uk (Jarkko Haapalainen) Date: Thu, 28 Feb 2008 10:30:34 +0200 Subject: [Box Backup] running bbackupquery in a browser In-Reply-To: <47C5C141.1070208@logical-progress.com> References: <11876.87.194.161.98.1204120251.squirrel@webmail.officenetworksystems.co.uk> <47C59A7B.2060303@trexler.at> <20080227180124.GC15350@netinertia.co.uk> <47C5C141.1070208@logical-progress.com> Message-ID: <520F49BE-7BEB-4B47-AFB9-BBCEC841F671@viidakko.fi> --Apple-Mail-4-394451532 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Hi Dave, I made quick version of mine for browsing and restoring files from browser so look at the attached file. It only took ~1,5 hours so there is probably bugs, but is working nicely for me in my mac ;) Best regards, Jarkko --Apple-Mail-4-394451532 Content-Disposition: attachment; filename=test.php Content-Type: text/php; x-unix-mode=0644; name="test.php" Content-Transfer-Encoding: 7bit $path, 'files' => $matches); } function arr_to_html($m) { $html = ''; $path = $m['path']; foreach ($m['files'] as $object) { list ($line, $id, $flags, $datetime, $name) = $object; if (strpos($flags, 'd') === 1) { $link = sprintf('%s', "$path/$name", $name); } else { $link = sprintf('%s', "$path", $id, $name, $name); } $html .= "$id$flags$datetime$link"; } return $html; } function get_file($bquery, $chown, $tmpdir, $path, $file, $filename) { $new = "$tmpdir/".md5($file); $cmd = sprintf('%s \'cd "%s"\' \'get -i %s %s\' quit', $bquery, rawurldecode($path), rawurldecode($file), $new); $out = shell_exec($cmd); if (file_exists($new)) { exec("$chown '$new'"); $fp = fopen($new, 'rb'); header("Pragma: public"); header('Expires: '.gmdate('D, d M Y H:i:s').' GMT'); header("Cache-Control: private",false); header("Cache-Control: must-revalidate, post-check=0, pre-check=0"); header("Content-Type: application/octet-stream"); header("Content-Disposition: attachment; filename=\"".rawurldecode(utf8_decode($filename))."\""); header("Content-Length: " . filesize($new)); fpassthru($fp); unlink($new); exit; } else { echo "
ERROR: file restore failed\n";
		var_dump($cmd, $path, $file);
		echo "
"; } } if (!empty($_GET['get'])) { get_file($bquery, $chown, $tmpdir, $_GET['cwd'], $_GET['get'], $_GET['name']); exit; } ?> bbackupquery wrapper
id flags datetime name
--Apple-Mail-4-394451532 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Dave Bamford kirjoitti 27.2.2008 kello 22.00: > I had a look at the Ruby on Rails at the time, but I don't think the > source was available. > > I am looking for a quick and easy way of browsing and recovering files > and I got stuck as to why I got the error. Its probably PHP related, > perhaps > I ought to try a PHP mailing list. > I suspect I need some PHP code, maybe stream functions, to handle > the file IO > from bbackupquery, but it is way beyond me. > Tried exec system and shell_exec. Had a look at the docs on popen > but... > > Perhaps I need to set some environment variables. > > Dave Bamford > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > --Apple-Mail-4-394451532-- From boxbackup at boxbackup.org Thu Feb 14 14:00:33 2008 From: boxbackup at boxbackup.org (Sean Laurent) Date: Thu, 14 Feb 2008 08:00:33 -0600 Subject: [Box Backup] Setup Help on Linux Message-ID: ------=_Part_1424_22004529.1202997633464 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Howdy all, I'm trying to setup BoxBackup 0.11 rc2 on Kubuntu 7.10 (Gutsy Gibbon) and I'm just not having much luck. I'm sure I've done something silly, but I can't figure out my mistake. Right now I have one client (192.168.2.100) and one server (192.168.2.50), both of which are running 0.11rc2 on Gutsy. When I run the client I get a connection error: ---snip--- $ sudo bbackupd -c /etc/boxbackup/bbackupd.conf -D -k -v -T 08:57:44 NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/bbackupd.conf 08:57:46 Incoming connection from local (UNIX socket) 08:57:46 Connection from command socket 08:57:46 NOTICE: Beginning scan of local files 08:57:46 Opening connection to server '192.168.2.50'... 08:57:46 ERROR: Failed to connect to socket (type 1, name 192.168.2.50, port 2201): Connection refused (111) 08:57:46 WARNING: Exception thrown: ConnectionException(Conn_SocketConnectError) at SocketStream.cpp(223) 08:57:46 NOTICE: About to notify administrator about event backup-error, running script '/etc/boxbackup/bbackupd/NotifySysadmin.sh backup-error' 08:57:46 ERROR: Exception caught (Connection SocketConnectError (Probably a network issue between client and server, bad hostname, or server not running.) 7/15), reset state and waiting to retry... 08:57:56 NOTICE: File statistics: total file size uploaded 0, bytes already on server 0, encoded size 0 ---snip--- Meanwhile, I'm monitoring syslog on the server, but absolutely nothing happens. It appears the client isn't making it that far. ---snip--- Feb 12 08:57:41 frodo Box Backup (bbstored)[22436]: NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/bbstored.conf ---snip--- I know the server is accessible, since I can SSH into it. I'm not positive the bbstored daemon is actually listening on the specified port (2201) and I don't know how to verify that. My client config looks like the following (comments removed): ---/etc/boxbackup/bbackup.conf--- StoreHostname = 192.168.2.50 AccountNumber = 0x1106 KeysFile = /etc/boxbackup/bbackupd/1106-FileEncKeys.raw CertificateFile = /etc/boxbackup/bbackupd/1106-cert.pem PrivateKeyFile = /etc/boxbackup/bbackupd/1106-key.pem TrustedCAsFile = /etc/boxbackup/bbackupd/serverCA.pem DataDirectory = /var/lib/bbackupd/ NotifyScript = /etc/boxbackup/bbackupd/NotifySysadmin.sh AutomaticBackup = no UpdateStoreInterval = 0 MinimumFileAge = 0 MaxUploadWait = 0 FileTrackingSizeThreshold = 65535 DiffingUploadSizeThreshold = 8192 MaximumDiffingTime = 120 ExtendedLogging = yes CommandSocket = /var/lib/bbackupd/bbackupd.sock Server { PidFile = /var/lib/bbackupd/bbackupd.pid } BackupLocations { home- { Path = /home/ ExcludeFilesRegex = .(mp3|MP3|ogg|OGG|avi|AVI) ExcludeDir = /home/sean/mount ExcludeDir = /home/sean/smb4k } etc- { Path = /etc/ } } ---end--- My server configuration looks like the following (with comments removed): ---/etc/boxbackup/bbstored.conf--- RaidFileConf = /etc/boxbackup/raidfile.conf AccountDatabase = /etc/boxbackup/bbstored/accounts.txt TimeBetweenHousekeeping = 900 Server { PidFile = /var/run/bbstored.pid User = bbstored ListenAddresses = inet:frodo CertificateFile = /etc/boxbackup/bbstored/frodo-cert.pem PrivateKeyFile = /etc/boxbackup/bbstored/frodo-key.pem TrustedCAsFile = /etc/boxbackup/bbstored/clientCA.pem } ---end--- Any suggestions would be greatly appreciated! Thanks!. -Sean -- A man is never drunk if he can lie on the floor without holding on. - Joe E. Lewis ------=_Part_1424_22004529.1202997633464 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Howdy all,
I'm trying to setup BoxBackup 0.11 rc2 on Kubuntu 7.10 (Gutsy Gibbon) and I'm just not having much luck. I'm sure I've done something silly, but I can't figure out my mistake.

Right now I have one client (192.168.2.100) and one server (192.168.2.50), both of which are running 0.11rc2 on Gutsy. When I run the client I get a connection error:

---snip---
$ sudo bbackupd -c /etc/boxbackup/bbackupd.conf -D -k -v -T
08:57:44 NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/bbackupd.conf
08:57:46 Incoming connection from local (UNIX socket)
08:57:46 Connection from command socket
08:57:46 NOTICE: Beginning scan of local files
08:57:46 Opening connection to server '192.168.2.50'...
08:57:46 ERROR: Failed to connect to socket (type 1, name 192.168.2.50, port 2201): Connection refused (111)
08:57:46 WARNING: Exception thrown: ConnectionException(Conn
_SocketConnectError) at SocketStream.cpp(223)
08:57:46 NOTICE: About to notify administrator about event backup-error, running script '/etc/boxbackup/bbackupd/NotifySysadmin.sh backup-error'
08:57:46 ERROR: Exception caught (Connection SocketConnectError (Probably a network issue between client and server, bad hostname, or server not running.) 7/15), reset state and waiting to retry...
08:57:56 NOTICE: File statistics: total file size uploaded 0, bytes already on server 0, encoded size 0
---snip---

Meanwhile, I'm monitoring syslog on the server, but absolutely nothing happens. It appears the client isn't making it that far.

---snip---
Feb 12 08:57:41 frodo Box Backup (bbstored)[22436]: NOTICE: Starting daemon, version 0.11rc2, config: /etc/boxbackup/bbstored.conf
---snip---

I know the server is accessible, since I can SSH into it. I'm not positive the bbstored daemon is actually listening on the specified port (2201) and I don't know how to verify that.

My client config looks like the following (comments removed):

---/etc/boxbackup/bbackup.conf---
StoreHostname = 192.168.2.50
AccountNumber = 0x1106
KeysFile = /etc/boxbackup/bbackupd/1106-FileEncKeys.raw
CertificateFile = /etc/boxbackup/bbackupd/1106-cert.pem
PrivateKeyFile = /etc/boxbackup/bbackupd/1106-key.pem
TrustedCAsFile = /etc/boxbackup/bbackupd/serverCA.pem
DataDirectory = /var/lib/bbackupd/
NotifyScript = /etc/boxbackup/bbackupd/NotifySysadmin.sh
AutomaticBackup = no
UpdateStoreInterval = 0
MinimumFileAge = 0
MaxUploadWait = 0
FileTrackingSizeThreshold = 65535
DiffingUploadSizeThreshold = 8192
MaximumDiffingTime = 120
ExtendedLogging = yes
CommandSocket = /var/lib/bbackupd/bbackupd.sock
Server
{
    PidFile = /var/lib/bbackupd/bbackupd.pid
}
BackupLocations
{
    home-
    {
        Path = /home/
        ExcludeFilesRegex = .(mp3|MP3|ogg|OGG|avi|AVI)
        ExcludeDir = /home/sean/mount
        ExcludeDir = /home/sean/smb4k
    }
    etc-
    {
        Path = /etc/
    }
}
---end---

My server configuration looks like the following (with comments removed):

---/etc/boxbackup/bbstored.conf---
RaidFileConf = /etc/boxbackup/raidfile.conf
AccountDatabase = /etc/boxbackup/bbstored/accounts.txt
TimeBetweenHousekeeping = 900
Server
{
    PidFile = /var/run/bbstored.pid
    User = bbstored
    ListenAddresses = inet:frodo
    CertificateFile = /etc/boxbackup/bbstored/frodo-cert.pem
    PrivateKeyFile = /etc/boxbackup/bbstored/frodo-key.pem
    TrustedCAsFile = /etc/boxbackup/bbstored/clientCA.pem
}
---end---

Any suggestions would be greatly appreciated!

Thanks!.

-Sean


--
A man is never drunk if he can lie on the floor without holding on. - Joe E. Lewis ------=_Part_1424_22004529.1202997633464--