From boxbackup at fluffy.co.uk Tue Oct 2 18:52:35 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 2 Oct 2007 18:52:35 +0100 (BST) Subject: [Box Backup] chris-merge - 2GB+ - Get OK - Compare fails? In-Reply-To: <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> References: <46DBC799.6060406@kontrapunkt.com> <46DBC978.6020401@kontrapunkt.com> <46E4F4BB.9040603@kontrapunkt.com> <46E5000F.4000206@kontrapunkt.com> <46E506F2.1000804@kontrapunkt.com> <46EA4D21.7090107@kontrapunkt.com> <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> Message-ID: Hi Tobias, On Wed, 26 Sep 2007, Tobias Balle-Petersen wrote: > All the errors are of this format: > > file A has different contents to store file file B > > The compare ends with: > [ 0 (of 8) differences probably due to file modifications after the last > upload ] > Differences: 8 (0 dirs excluded, 0 files excluded, 2 files not checked) > Logging off... > Exception: Connection Protocol_ObjTooBig (7/42) Are all the files which failed to compare properly over 2GB, or only the first one? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Oct 3 03:06:43 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Tue, 2 Oct 2007 19:06:43 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X Message-ID: <34800.26845.qm@web60623.mail.yahoo.com> Hi all, A new situation for me. A new client is in a Mac environment (info from the client): main machine OS 10.4.10 Dual 2 GHz Power PC G5 server OS 10.4.8 400 MHz Power PC G4 I'm not a Mac guy at all, so I'm not sure I even know where to begin. I guess I have to set up a build environment (yikes! how?!), and then follow: http://bbdev.fluffy.co.uk/trac/wiki/SourceCodeRepository ? Any and all help appreciated! Thanks! Pete From boxbackup at fluffy.co.uk Wed Oct 3 05:50:45 2007 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Tue, 02 Oct 2007 21:50:45 -0700 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <34800.26845.qm@web60623.mail.yahoo.com> References: <34800.26845.qm@web60623.mail.yahoo.com> Message-ID: <47031FA5.8050306@reedtz.com> On 10/2/07 7:06 PM, Peter Jalajas, TebucoSafe Backups wrote: > Hi all, > > A new situation for me. A new client is in a Mac environment (info from the client): > > main machine > OS 10.4.10 > Dual 2 GHz Power PC G5 > > server > OS 10.4.8 > 400 MHz Power PC G4 > > I'm not a Mac guy at all, so I'm not sure I even know where to begin. I guess I have to set up a > build environment (yikes! how?!), and then follow: > http://bbdev.fluffy.co.uk/trac/wiki/SourceCodeRepository ? > > Any and all help appreciated! > I'll be happy to send you the G5 10.4.10 parcel for client and server, but I don't have a G4 machine anymore. As far as a development environment goes, Apple's XCode Dev Kit is free to download if you sign up at Apple's developer site (http://www.apple.com/developer). The ADC Online membership is free. Let me know if you want the G5 stuff, and what version of Box. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Wed Oct 3 09:49:40 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 3 Oct 2007 09:49:40 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <47031FA5.8050306@reedtz.com> References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> Message-ID: Hi Pete and Per, On Tue, 2 Oct 2007, Per Reedtz Thomsen wrote: > On 10/2/07 7:06 PM, Peter Jalajas, TebucoSafe Backups wrote: >> >> A new situation for me. A new client is in a Mac environment (info from >> the client): ... >> I'm not a Mac guy at all, so I'm not sure I even know where to begin. I >> guess I have to set up a >> build environment (yikes! how?!), and then follow: >> http://bbdev.fluffy.co.uk/trac/wiki/SourceCodeRepository ? >> >> Any and all help appreciated! >> > I'll be happy to send you the G5 10.4.10 parcel for client and server, > but I don't have a G4 machine anymore. As far as a development > environment goes, Apple's XCode Dev Kit is free to download if you sign > up at Apple's developer site (http://www.apple.com/developer). The ADC > Online membership is free. Pete, does your client have Fink installed, or would they mind installing it? Per, do you know much about Fink? Would you be interested in helping to get Box into Fink's repository? I don't have a Mac, so I'd be flying blind, but I'd like to get it done. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Oct 3 10:03:36 2007 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 03 Oct 2007 11:03:36 +0200 Subject: [Box Backup] chris-merge - 2GB+ - Get OK - Compare fails? In-Reply-To: References: <46DBC799.6060406@kontrapunkt.com> <46DBC978.6020401@kontrapunkt.com> <46E4F4BB.9040603@kontrapunkt.com> <46E5000F.4000206@kontrapunkt.com> <46E506F2.1000804@kontrapunkt.com> <46EA4D21.7090107@kontrapunkt.com> <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> Message-ID: <47035AE8.6000208@kontrapunkt.com> Hello Chris. None of them ar bigger than 2GB. Tobias Chris Wilson wrote: > Hi Tobias, > > On Wed, 26 Sep 2007, Tobias Balle-Petersen wrote: > >> All the errors are of this format: >> >> file A has different contents to store file file B >> >> The compare ends with: >> [ 0 (of 8) differences probably due to file modifications after the >> last upload ] >> Differences: 8 (0 dirs excluded, 0 files excluded, 2 files not checked) >> Logging off... >> Exception: Connection Protocol_ObjTooBig (7/42) > > Are all the files which failed to compare properly over 2GB, or only the > first one? > > Cheers, Chris. From boxbackup at fluffy.co.uk Wed Oct 3 12:02:09 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 3 Oct 2007 12:02:09 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> Message-ID: <107702FD-52C9-4642-9C22-66406360EC53@mbrown.co.uk> Hi Chris, > Pete, does your client have Fink installed, or would they mind > installing it? > > Per, do you know much about Fink? Would you be interested in > helping to get Box into Fink's repository? I don't have a Mac, so > I'd be flying blind, but I'd like to get it done. I own a mac, I would be happy to assist here ... I have an intel based Mac (MacBook Pro) .. so I guess a universal binary would be the way to go (for both Intel / PPC) Let me know if this is something you wish me to help with, I have played with fink a while back for installing the odd app ... Regards Matt Brown From boxbackup at fluffy.co.uk Wed Oct 3 12:09:16 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 3 Oct 2007 12:09:16 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <107702FD-52C9-4642-9C22-66406360EC53@mbrown.co.uk> References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> <107702FD-52C9-4642-9C22-66406360EC53@mbrown.co.uk> Message-ID: <023A0C28-A436-45EE-BED8-33FA6B931CB0@mbrown.co.uk> > Hi Chris, > > > >> Pete, does your client have Fink installed, or would they mind >> installing it? >> >> Per, do you know much about Fink? Would you be interested in >> helping to get Box into Fink's repository? I don't have a Mac, so >> I'd be flying blind, but I'd like to get it done. > > > I own a mac, I would be happy to assist here ... I have an intel > based Mac (MacBook Pro) .. so I guess a universal binary would be > the way to go (for both Intel / PPC) > > Let me know if this is something you wish me to help with, I have > played with fink a while back for installing the odd app ... > > Regards > > Matt Brown Oh and I have an old 12" G4 Mac kicking around too, which would be also handy for testing on :-) From boxbackup at fluffy.co.uk Wed Oct 3 16:45:54 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Wed, 3 Oct 2007 08:45:54 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <47031FA5.8050306@reedtz.com> Message-ID: <426656.95816.qm@web60619.mail.yahoo.com> Great, Per, thanks, please do send me the G5 client parcel, preferrably for one of Chris' very recent versions, I think, if appropriate and possible. Any mac installation instructions you could provide would be welcome. The client says we can reach out from the G5 to the G4 box to pull files for backing up to my box backup server (on Amazon EC2!). I don't think I'll need the development environs now, true? Thanks again! Pete From boxbackup at fluffy.co.uk Wed Oct 3 18:47:44 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 3 Oct 2007 18:47:44 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <023A0C28-A436-45EE-BED8-33FA6B931CB0@mbrown.co.uk> References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> <107702FD-52C9-4642-9C22-66406360EC53@mbrown.co.uk> <023A0C28-A436-45EE-BED8-33FA6B931CB0@mbrown.co.uk> Message-ID: Hi Matt, On Wed, 3 Oct 2007, Matt Brown wrote: >> > Pete, does your client have Fink installed, or would they mind >> > installing it? >> > >> > Per, do you know much about Fink? Would you be interested in helping >> > to get Box into Fink's repository? I don't have a Mac, so I'd be >> > flying blind, but I'd like to get it done. >> >> I own a mac, I would be happy to assist here ... I have an intel based >> Mac (MacBook Pro) .. so I guess a universal binary would be the way to >> go (for both Intel / PPC) I think Fink builds packages from source, doesn't it? (I don't know for sure but I'm about to get in touch with them to ask about how we can do this). > Oh and I have an old 12" G4 Mac kicking around too, which would be also > handy for testing on :-) If there's some way that you could set me up with VNC access to it then that would be perfect :-) Thanks in advance. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Oct 3 18:53:57 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 3 Oct 2007 18:53:57 +0100 (BST) Subject: [Box Backup] chris-merge - 2GB+ - Get OK - Compare fails? In-Reply-To: <47035AE8.6000208@kontrapunkt.com> References: <46DBC799.6060406@kontrapunkt.com> <46DBC978.6020401@kontrapunkt.com> <46E4F4BB.9040603@kontrapunkt.com> <46E5000F.4000206@kontrapunkt.com> <46E506F2.1000804@kontrapunkt.com> <46EA4D21.7090107@kontrapunkt.com> <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> <47035AE8.6000208@kontrapunkt.com> Message-ID: Hi Tobias, Would you mind not top-posting? It makes our discussion difficult to read for other list members who haven't read previous messages in detail or who are new to the list. Thank you. I've fixed it here: On Wed, 3 Oct 2007, Tobias Balle-Petersen wrote: > Chris Wilson wrote: >> On Wed, 26 Sep 2007, Tobias Balle-Petersen wrote: >> >>> All the errors are of this format: >>> >>> file A has different contents to store file file B >>> >>> The compare ends with: >>> [ 0 (of 8) differences probably due to file modifications after the >>> last upload ] >>> Differences: 8 (0 dirs excluded, 0 files excluded, 2 files not checked) >>> Logging off... >>> Exception: Connection Protocol_ObjTooBig (7/42) >> >> Are all the files which failed to compare properly over 2GB, or only the >> first one? > > None of them ar bigger than 2GB. OK, that is strange and very worrying. I would very much appreciate your help in fixing this. First off, please could you remind me what versions of Box Backup you are running on the client and the server? Secondly, please could you make a copy of one of the files that's failing to compare, in the same directory, and check that the copy is backed up, and then run a compare and see if the copy fails to compare too? That should establish whether the problem is on the client side, or with an old client which generated files which now can't be patched. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Oct 3 21:43:11 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 3 Oct 2007 21:43:11 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> <107702FD-52C9-4642-9C22-66406360EC53@mbrown.co.uk> <023A0C28-A436-45EE-BED8-33FA6B931CB0@mbrown.co.uk> Message-ID: <61109C04-0400-4982-9528-526537565CC9@mbrown.co.uk> Hi Chris, > I think Fink builds packages from source, doesn't it? (I don't know > for sure but I'm about to get in touch with them to ask about how > we can do this). > >> Oh and I have an old 12" G4 Mac kicking around too, which would be >> also handy for testing on :-) > > If there's some way that you could set me up with VNC access to it > then that would be perfect :-) Thanks in advance. Just sent you a message off list, give me a call. Regards Matt Brown From boxbackup at fluffy.co.uk Thu Oct 4 08:09:21 2007 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Thu, 04 Oct 2007 00:09:21 -0700 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> Message-ID: <470491A1.5020005@reedtz.com> On 10/3/07 1:49 AM, Chris Wilson wrote: > Hi Pete and Per, > > On Tue, 2 Oct 2007, Per Reedtz Thomsen wrote: > >> On 10/2/07 7:06 PM, Peter Jalajas, TebucoSafe Backups wrote: >>> >>> A new situation for me. A new client is in a Mac environment (info >>> from >>> the client): > ... >>> I'm not a Mac guy at all, so I'm not sure I even know where to >>> begin. I >>> guess I have to set up a >>> build environment (yikes! how?!), and then follow: >>> http://bbdev.fluffy.co.uk/trac/wiki/SourceCodeRepository ? >>> >>> Any and all help appreciated! >>> >> I'll be happy to send you the G5 10.4.10 parcel for client and >> server, but I don't have a G4 machine anymore. As far as a >> development environment goes, Apple's XCode Dev Kit is free to >> download if you sign up at Apple's developer site >> (http://www.apple.com/developer). The ADC Online membership is free. > > Pete, does your client have Fink installed, or would they mind > installing it? > > Per, do you know much about Fink? Would you be interested in helping > to get Box into Fink's repository? I don't have a Mac, so I'd be > flying blind, but I'd like to get it done. I'd love to help with that. Not sure what Matt is doing, so just want to coordinate. I also have the ability to make Universal Binaries.. First problem though: I am not sure what to pull from SVN. I pulled the tip of chris/general and trunk, but both do not pass runtest.pl... Not sure if I got the wrong code, or if there is some other problem. Here is the summary of test results for the trunk build. The chris/general branch has the same problems: common: FAILED: 1 tests failed (first at :0) crypto: FAILED: 1 tests failed (first at :0) compress: PASSED raidfile: FAILED: 1 tests failed (first at :0) basicserver: FAILED: 1 tests failed (first at :0) backupstore: FAILED: 3 tests failed (first at ../../lib/common/Test.h:157) backupstorefix: FAILED: 2 tests failed (first at ../../lib/common/Test.h:157) backupstorepatch: FAILED: 2 tests failed (first at ../../lib/common/Test.h:278) backupdiff: FAILED: 1 tests failed (first at :0) bbackupd: FAILED: 3 tests failed (first at ../../lib/common/Test.h:157) Let me know if you want the full config.log, and test output, and I'll send it. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Thu Oct 4 10:57:08 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Thu, 4 Oct 2007 10:57:08 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <470491A1.5020005@reedtz.com> References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> <470491A1.5020005@reedtz.com> Message-ID: <8FCC6110-649C-4700-A0AD-FD8DB553D883@mbrown.co.uk> Hi Per, > I'd love to help with that. Not sure what Matt is doing, so just > want to coordinate. I also have the ability to make Universal > Binaries.. > I have given Chris remote shell/GUI access to a G4 Mac I have here, and if he requires it he can have access to my Intel MacBook Pro also :-) Regards Matt Brown From boxbackup at fluffy.co.uk Thu Oct 4 17:27:50 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 4 Oct 2007 17:27:50 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <470491A1.5020005@reedtz.com> References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> <470491A1.5020005@reedtz.com> Message-ID: Hi Per, On Thu, 4 Oct 2007, Per Reedtz Thomsen wrote: >> Per, do you know much about Fink? Would you be interested in helping to >> get Box into Fink's repository? I don't have a Mac, so I'd be flying >> blind, but I'd like to get it done. > > I'd love to help with that. Not sure what Matt is doing, so just want to > coordinate. I also have the ability to make Universal Binaries.. > > First problem though: I am not sure what to pull from SVN. I pulled the > tip of chris/general and trunk, but both do not pass runtest.pl... Not > sure if I got the wrong code, or if there is some other problem. Better to use trunk for now. chris/merge will go away when I've merged the last few troublesome changes. Thanks to Matt, I now have access to a real Mac, and I (re)discovered these problems while testing yesterday. The first one is that there's a single file descriptor leak from syslog(). Definitely not harmful, but annoying because it breaks our tests. I'm working on that now. I think that there are other problems too. Will look into them when I have fixed the fd problem. If you could work on creating a package anyway, that would be great. Once I've fixed these problems we can run with it and start distributing straight away. A Fink person offered to help with the package on IRC. I'd really like to get a working DMG too, as that seems to be the most natural package format on MacOS X, right? And one which doesn't interfere with the rest of your system unlike Fink or DarwinPorts. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Oct 4 19:46:36 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Thu, 4 Oct 2007 11:46:36 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: Message-ID: <545160.87834.qm@web60625.mail.yahoo.com> Hi all, Wow! I am humbled by the incredibly thorough and quick response to my request. I love this project! Thank you all who responded! A question comes up with these pre-built Mac and Windows packages. That is, since we end-users are not building the client parcels ourselves, and relying on the generosity and trust of others (Chris, Per, etc) to build them for us, we thus create a bit of a security gap over which we must take a leap of faith and trust. Does anyone have any suggestions for a reasonable way to manage that security risk, to shrink that security gap? A client may ask someday. Thanks, Pete From boxbackup at fluffy.co.uk Thu Oct 4 22:28:56 2007 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Thu, 04 Oct 2007 14:28:56 -0700 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <34800.26845.qm@web60623.mail.yahoo.com> <47031FA5.8050306@reedtz.com> Message-ID: <47055B18.2000804@reedtz.com> All, On a Macintosh note, I have a launchd config file for bbackupd, that makes bbackupd launch on boot. Not quite sure where this file should be checked in? contrib? It should obviously be a part of a Macintosh installation package, along with a bbstored companion... Where should I check it in? Or does it require a review? Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Fri Oct 5 00:17:20 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 5 Oct 2007 00:17:20 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <545160.87834.qm@web60625.mail.yahoo.com> References: <545160.87834.qm@web60625.mail.yahoo.com> Message-ID: Hi Pete, On Thu, 4 Oct 2007, Peter Jalajas, TebucoSafe Backups wrote: > Wow! I am humbled by the incredibly thorough and quick response to my > request. I love this project! Thank you all who responded! > > A question comes up with these pre-built Mac and Windows packages. > That is, since we end-users are not building the client parcels > ourselves, and relying on the generosity and trust of others (Chris, > Per, etc) to build them for us, we thus create a bit of a security gap > over which we must take a leap of faith and trust. Does anyone have any > suggestions for a reasonable way to manage that security risk, to shrink > that security gap? A client may ask someday. I'm not sure that downloading and installing a binary package is any different to downloading, compiling and installing a source package, UNLESS you take the time to verify the sources by hand. Assuming that you don't (and I don't, even though I know I should :-) then it comes down to taking packages (binary or source) only from those who you trust. Leading to the question, "who do you trust"? Which is one that I'd be very interested to hear your answers on :-) E.g. would you trust a package (binary or source) from Fink? From Debian (Reinhard Tartler)? From Per Thomsen? From me? From Ben? What would your customers trust? Would you trust a source or binary package based on trunk less than an official release with known bugs and a well-known MD5sum? Would you trust a Microsoft patch with a license agreement that allows them to install arbitrary software on your machine at a later date? Would you still trust Microsoft after they installed that arbitrary software without telling you? Would you trust a (source or binary) release more if it had a published MD5 checksum? If it had a PGP signature? Can you actually verify stuff against my PGP key? Do you trust that it was really me who uploaded that key? Would you trust my emails more if I PGP-signed them? In my view it comes down to an element of unknown risk (of bugs more than trojans) from running newer, less tested code, versus an element of (better) known risk running older code with known bugs (which might also be trojaned). And also the reputation of the code authors, the maintainers, the packagers and the distributors. Personally I put near-absolute trust in my distro maintainers and the packages that they choose to release (and whether those packages are destined to run with root privileges or not) because I feel they they have more to lose by being publicly humiliated than I do by running their code. As far as Box is concerned, I trust James to keep our Subversion server safe, and I trust Subversion to send me commit logs, which I read to ensure that nothing happens to the Box source without me knowing about it and understanding what it does (as best this bear of little brain can understand anything). So what do "security gap", faith and trust mean for you? I guess we need better definitions of those terms. At least I do before I can even try to assuage your doubts. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sat Oct 6 14:14:37 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sat, 6 Oct 2007 14:14:37 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <545160.87834.qm@web60625.mail.yahoo.com> References: <545160.87834.qm@web60625.mail.yahoo.com> Message-ID: Hi all, On Thu, 4 Oct 2007, Peter Jalajas, TebucoSafe Backups wrote: > A question comes up with these pre-built Mac and Windows packages. > That is, since we end-users are not building the client parcels > ourselves, and relying on the generosity and trust of others (Chris, > Per, etc) to build them for us, we thus create a bit of a security gap > over which we must take a leap of faith and trust. Does anyone have any > suggestions for a reasonable way to manage that security risk, to shrink > that security gap? A client may ask someday. One measure that should increase our project security would be to enable SSL on our Trac and Subversion server, to reduce the risk of password theft. James, would that be hard for you to do? I did it the other day for our company Subversion server and it "just worked" with the standard Apache SSL configuration. Perhaps we could even group together to pay for a real SSL certificate (blech!) or get an OpenCA certificate? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sat Oct 6 15:52:38 2007 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Sat, 6 Oct 2007 16:52:38 +0200 Subject: [Box Backup] Help installing client on Mac OS X References: <545160.87834.qm@web60625.mail.yahoo.com> Message-ID: <006801c80828$8b2c4ef0$e901050a@stenorvaio> > Perhaps we could even group together to pay for a real SSL certificate > (blech!) or get an OpenCA certificate? I (or rather my company) would be happy to sponsor a "real" SSL certificate, if needed. It would feel great to be able to give something back to the project! Regards, Stefan From boxbackup at fluffy.co.uk Sun Oct 7 00:21:42 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sun, 7 Oct 2007 00:21:42 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <545160.87834.qm@web60625.mail.yahoo.com> Message-ID: <20071006232142.GD31832@netinertia.co.uk> On Sat, Oct 06, 2007 at 02:14:37PM +0100, Chris Wilson wrote: > One measure that should increase our project security would be to enable SSL > on our Trac and Subversion server, to reduce the risk of password theft. > > James, would that be hard for you to do? I did it the other day for our > company Subversion server and it "just worked" with the standard Apache SSL > configuration. I do have SSL enabled, although I need a new certificate as it was self-signed (I only use it for admin stuff) and has expired. The other issue is that SSL doesn't support vhosts (which bbdev is) so I would have to ask for an additional IP from my colo provider (shouldn't be a problem). > Perhaps we could even group together to pay for a real SSL certificate > (blech!) or get an OpenCA certificate? I've pondered cacert before but never really looked into it. James From boxbackup at fluffy.co.uk Sun Oct 7 10:21:46 2007 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Sun, 07 Oct 2007 10:21:46 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <20071006232142.GD31832@netinertia.co.uk> References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> Message-ID: <1191748906.20762.8.camel@avenin.ebourne.me.uk> On Sun, 2007-10-07 at 00:21 +0100, James O'Gorman wrote: > I've pondered cacert before but never really looked into it. I've been using cacert for years. I like it a lot and it works really well for me. I don't believe that certificates should artificially cost a whole load of money any more than software should. Computer security shouldn't be a privilege. I personally trust a CA cert certificate more than I would from a number of the commercial authorities (Verisign is one for starters). The two downsides of CA cert is that with the default "membership" certificates are only valid for 6 months, which means 2 minutes spent renewing it twice a year. And more importantly for a project like box, most of the major browsers still don't have it as a default root cert. On the other hand, if someone is prepared to pay for a commercial cert as already offered, I certainly don't mind. Cheers, Martin. From boxbackup at fluffy.co.uk Sun Oct 7 11:01:47 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 7 Oct 2007 11:01:47 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <20071006232142.GD31832@netinertia.co.uk> References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> Message-ID: Hi James, On Sun, 7 Oct 2007, James O'Gorman wrote: > On Sat, Oct 06, 2007 at 02:14:37PM +0100, Chris Wilson wrote: >> One measure that should increase our project security would be to enable SSL >> on our Trac and Subversion server, to reduce the risk of password theft. >> >> James, would that be hard for you to do? I did it the other day for our >> company Subversion server and it "just worked" with the standard Apache SSL >> configuration. > > I do have SSL enabled, although I need a new certificate as it was > self-signed (I only use it for admin stuff) and has expired. Doesn't seem to be working for me: https://bbdev.fluffy.co.uk/ The connection was interrupted The connection to bbdev.fluffy.co.uk was interrupted while the page was loading. > The other issue is that SSL doesn't support vhosts (which bbdev is) so I > would have to ask for an additional IP from my colo provider (shouldn't > be a problem). Good point, if we want a real cert (for a self-signed one it hardly matters if it's for a different vhost). >> Perhaps we could even group together to pay for a real SSL certificate >> (blech!) or get an OpenCA certificate? > > I've pondered cacert before but never really looked into it. I've never used it either, but it didn't look hard. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun Oct 7 12:19:58 2007 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 7 Oct 2007 12:19:58 +0100 Subject: [Box Backup] Re: Help installing client on Mac OS X (Chris Wilson) Message-ID: Chris Wilson wrote: >> A question comes up with these pre-built Mac and Windows packages. >> That is, since we end-users are not building the client parcels >> ourselves, and relying on the generosity and trust of others (Chris, >> Per, etc) to build them for us, we thus create a bit of a security >> gap >> over which we must take a leap of faith and trust. Does anyone >> have any >> suggestions for a reasonable way to manage that security risk, to >> shrink >> that security gap? A client may ask someday. >> > > One measure that should increase our project security would be to > enable > SSL on our Trac and Subversion server, to reduce the risk of password > theft Perhaps we should also start using the boxbackup.org domain I've got but never used and take the opportunity to get a better hosting arrangment for the web site and things? This is not just a fluffy.co.uk project any more. Ben From boxbackup at fluffy.co.uk Sun Oct 7 15:40:20 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sun, 7 Oct 2007 15:40:20 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> Message-ID: <20071007144020.GE31832@netinertia.co.uk> On Sun, Oct 07, 2007 at 11:01:47AM +0100, Chris Wilson wrote: > Doesn't seem to be working for me: That's not what I meant :-) I have SSL enabled for certain things, not everything. As I said, I only use it for my own admin bits at the minute. > Good point, if we want a real cert (for a self-signed one it hardly matters > if it's for a different vhost). I'll ask my provider for some new IPs anyway. I need at least two more for something else coming up, so I'll stick this in too. Unless we want to go IPv6 only, in which case I have a /64 :-) James From boxbackup at fluffy.co.uk Sun Oct 7 15:48:46 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sun, 7 Oct 2007 15:48:46 +0100 Subject: [Box Backup] Re: Help installing client on Mac OS X (Chris Wilson) In-Reply-To: References: Message-ID: <20071007144845.GF31832@netinertia.co.uk> On Sun, Oct 07, 2007 at 12:19:58PM +0100, Ben Summers wrote: > Perhaps we should also start using the boxbackup.org domain I've got but > never used and take the opportunity to get a better hosting arrangment for > the web site and things? This is not just a fluffy.co.uk project any more. I don't mind offering to host the website, although I don't have any mailing list software installed at the moment so wouldn't be able to take the lists right now. James From boxbackup at fluffy.co.uk Sun Oct 7 15:48:54 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 7 Oct 2007 15:48:54 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <20071007144020.GE31832@netinertia.co.uk> References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> <20071007144020.GE31832@netinertia.co.uk> Message-ID: Hi James, On Sun, 7 Oct 2007, James O'Gorman wrote: > Unless we want to go IPv6 only, in which case I have a /64 :-) Sorry, I'm not even sure how I would use that, as my provider is IPv4 only so I guess I'd have to tunnel to somewhere? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun Oct 7 15:53:23 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sun, 7 Oct 2007 15:53:23 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> <20071007144020.GE31832@netinertia.co.uk> Message-ID: <20071007145322.GG31832@netinertia.co.uk> On Sun, Oct 07, 2007 at 03:48:54PM +0100, Chris Wilson wrote: > Sorry, I'm not even sure how I would use that, as my provider is IPv4 only > so I guess I'd have to tunnel to somewhere? It was a bit of a flippant remark, tbh. As much as some people would love to "flip the switch" and go to v6, it's not going to happen until there's more ISP support. You can get a tunnel from Sixxs though - Goscomb (my colo provider) are the tunnel brokers for the UK. Going back on topic, I've just remembered a site a friend told me about a while ago who do cheap SSL certificates which seem to be fully valid Thawte certs. I don't mind providing the RapidSSL certificate myself if everyone agrees it looks okay. http://www.trustico.co.uk/products/rapidssl/rapidssl.html James From boxbackup at fluffy.co.uk Sun Oct 7 16:22:44 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 7 Oct 2007 16:22:44 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <20071007145322.GG31832@netinertia.co.uk> References: <545160.87834.qm@web60625.mail.yahoo.com> <20071006232142.GD31832@netinertia.co.uk> <20071007144020.GE31832@netinertia.co.uk> <20071007145322.GG31832@netinertia.co.uk> Message-ID: Hi James, On Sun, 7 Oct 2007, James O'Gorman wrote: > Going back on topic, I've just remembered a site a friend told me about > a while ago who do cheap SSL certificates which seem to be fully valid > Thawte certs. I don't mind providing the RapidSSL certificate myself if > everyone agrees it looks okay. > > http://www.trustico.co.uk/products/rapidssl/rapidssl.html Looks good (and cheap!) to me - I think it would be great and very kind if you got one for us :-) Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon Oct 8 08:03:04 2007 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Mon, 08 Oct 2007 09:03:04 +0200 Subject: [Box Backup] Re: Help installing client on Mac OS X (Chris Wilson) In-Reply-To: <20071007144845.GF31832@netinertia.co.uk> References: <20071007144845.GF31832@netinertia.co.uk> Message-ID: <4709D628.4030704@infoflexconnect.se> > I don't mind offering to host the website, although I don't have any > mailing list software installed at the moment so wouldn't be able to > take the lists right now. > Maybe Google Groups would be an option for the mailing list? The interface and search options available there may be an improvement... Regards, Stefan From boxbackup at fluffy.co.uk Mon Oct 8 17:38:00 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Mon, 8 Oct 2007 09:38:00 -0700 (PDT) Subject: [Box Backup] noatime for this? Message-ID: <179695.73482.qm@web60615.mail.yahoo.com> Hi all, Instigated by news of Con Kolivas' departure from the Linux kernel team, I am exploring ways to speed up my perceived desktop performance (currently on Ubuntu, but open to other suggestions), and somehow noticed this recent article: http://rudd-o.com/archives/2007/10/02/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that/ which suggests adding ",noatime" to disk mount options, and which eventually led to this one: http://lwn.net/Articles/244829/ which duly warns that some applications need noatime. Does Box Backup client or server need noatime or any of the other options mentioned in that last article? (Any other desktop perceived-performance suggestions that don't negatively impact Box Backup would be greatly appreciated.) Thanks, Pete From boxbackup at fluffy.co.uk Mon Oct 8 18:55:30 2007 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Mon, 08 Oct 2007 18:55:30 +0100 Subject: [Box Backup] noatime for this? In-Reply-To: <179695.73482.qm@web60615.mail.yahoo.com> References: <179695.73482.qm@web60615.mail.yahoo.com> Message-ID: <1191866130.12391.21.camel@avenin.ebourne.me.uk> On Mon, 2007-10-08 at 09:38 -0700, Peter Jalajas, TebucoSafe Backups wrote: > Instigated by news of Con Kolivas' departure from the Linux kernel team Unfortunately he was never really part of the "team", which was a lot of the problem. > , I am exploring ways to > speed up my perceived desktop performance (currently on Ubuntu, but open to other suggestions), > and somehow noticed this recent article: > http://rudd-o.com/archives/2007/10/02/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that/ > which suggests adding ",noatime" to disk mount options, and which eventually led to this one: > http://lwn.net/Articles/244829/ > which duly warns that some applications need noatime. > > Does Box Backup client or server need noatime or any of the other options mentioned in that last > article? noatime does not adversely affect Box Backup. I've run with noatime set on all my filesystems on all my machines since back in the late 90s. Quite frankly I think you'd have to be mad not to, and I find it disappointing that more distros don't use it as a default, since for 99% of users it's the right thing. Cheers, Martin. From boxbackup at fluffy.co.uk Mon Oct 8 19:14:26 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Mon, 8 Oct 2007 11:14:26 -0700 (PDT) Subject: [Box Backup] noatime for this? In-Reply-To: <1191866130.12391.21.camel@avenin.ebourne.me.uk> Message-ID: <146120.26052.qm@web60614.mail.yahoo.com> --- Martin Ebourne wrote: > On Mon, 2007-10-08 at 09:38 -0700, Peter Jalajas, TebucoSafe Backups > wrote: > > Instigated by news of Con Kolivas' departure from the Linux kernel team > > Unfortunately he was never really part of the "team", which was a lot of > the problem. Ah, yes, thanks, Martin. > > , I am exploring ways to > > speed up my perceived desktop performance (currently on Ubuntu, but open to other > suggestions), > > and somehow noticed this recent article: > > > http://rudd-o.com/archives/2007/10/02/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that/ > > which suggests adding ",noatime" to disk mount options, and which eventually led to this one: Actually, a typo on my part: that rudd-o.com article above doesn't mention noatime, but the lwn.net one below does. Sorry for any confusion. > > http://lwn.net/Articles/244829/ > > which duly warns that some applications need noatime. > > > > Does Box Backup client or server need noatime or any of the other options mentioned in that > last > > article? > > noatime does not adversely affect Box Backup. I've run with noatime set > on all my filesystems on all my machines since back in the late 90s. > Quite frankly I think you'd have to be mad not to, and I find it > disappointing that more distros don't use it as a default, since for 99% > of users it's the right thing. Awesome, great to hear, thanks again. Yes, the noatime default is a very interesting issue (as are the swappiness and cache defaults). Pete From boxbackup at fluffy.co.uk Mon Oct 8 22:10:06 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Mon, 8 Oct 2007 14:10:06 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: Message-ID: <703521.18052.qm@web60619.mail.yahoo.com> Hi Chris, Great points all, as always! So implementing a strategy of security as a risk-management onion, and knowing that nothing is ever perfectly secure, the Box Backup project could/should take reasonably aggressive measures to use the available convenient processes that help ensure (assure?) the security of the source code and binaries. Maybe you all do already, and I just haven't bother to track it all down. So, yes, having widely disseminated sha1sums of everything goes a very long way towards that end with very little pain, I think. Signatures go yet again a bit further, but with a little more pain. (Insert other heavier lifting here.) You hosting a keysigning party in Cambs is probably too aggressive and inconvenient, providing too little bang for the buck, as we say here. ;^) In a previous life as a Release Engineering manager for an enterprise security product, I appended to our software build process an immediate automatic wide-dissemination of md5sums (md5 was state of the art then, as I believe sha1 is now) of all individual source files, binaries, tarballs, etc, so that anyone (developers, testers, sales, app engineers, customers) had the _ability_ to _very conveniently_ _know_ that they had the correct files. For sure, that information was mostly ignored, but when it was needed, it proved very much worth the little additional effort expended. I can't recall if I also published the md5sums of the build tools, too, but that would add another layer to the security onion. What steps do package builders typically use these days to lock down their build environments to ensure that they aren't sending out malware? Derived from an interesting page re ISO 9000 http://c2.com/cgi/wiki?WikiPagesAboutSayWhatYouDo : * Say What You Do (Have Documented Security Procedures) * Do What You Say (Follow the Procedures) * Record What You Did (Widely Publish Security-Relevant Records to a Separate Google-Crawled Mailing List, perhaps via http://www.itconsult.co.uk/stamper.htm) (I wonder if a shared Amazon EC2 server, or a public build site like SourceForge, could be put to good use in this regard?) I didn't win the lottery for the Led Zeppelin benefit concert at O2, so we'll have to postpone the keysigning party. Thanks again, Chris, and everyone else! Pete --- Chris Wilson wrote: > I'm not sure that downloading and installing a binary package is any > different to downloading, compiling and installing a source package, > UNLESS you take the time to verify the sources by hand. > > Assuming that you don't (and I don't, even though I know I should :-) then > it comes down to taking packages (binary or source) only from those who > you trust. Leading to the question, "who do you trust"? Which is one that > I'd be very interested to hear your answers on :-) > > E.g. would you trust a package (binary or source) from Fink? From Debian > (Reinhard Tartler)? From Per Thomsen? From me? From Ben? What would your > customers trust? > > Would you trust a source or binary package based on trunk less than an > official release with known bugs and a well-known MD5sum? > > Would you trust a Microsoft patch with a license agreement that allows > them to install arbitrary software on your machine at a later date? Would > you still trust Microsoft after they installed that arbitrary software > without telling you? > > Would you trust a (source or binary) release more if it had a published > MD5 checksum? If it had a PGP signature? Can you actually verify stuff > against my PGP key? Do you trust that it was really me who uploaded that > key? Would you trust my emails more if I PGP-signed them? > > In my view it comes down to an element of unknown risk (of bugs more than > trojans) from running newer, less tested code, versus an element of > (better) known risk running older code with known bugs (which might also > be trojaned). And also the reputation of the code authors, the > maintainers, the packagers and the distributors. > > Personally I put near-absolute trust in my distro maintainers and the > packages that they choose to release (and whether those packages are > destined to run with root privileges or not) because I feel they they have > more to lose by being publicly humiliated than I do by running their code. > > As far as Box is concerned, I trust James to keep our Subversion server > safe, and I trust Subversion to send me commit logs, which I read to > ensure that nothing happens to the Box source without me knowing about it > and understanding what it does (as best this bear of little brain can > understand anything). > > So what do "security gap", faith and trust mean for you? I guess we need > better definitions of those terms. At least I do before I can even try to > assuage your doubts. > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > From boxbackup at fluffy.co.uk Tue Oct 9 07:41:05 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 9 Oct 2007 07:41:05 +0100 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <703521.18052.qm@web60619.mail.yahoo.com> References: <703521.18052.qm@web60619.mail.yahoo.com> Message-ID: <20071009064105.GA6936@netinertia.co.uk> On Mon, Oct 08, 2007 at 02:10:06PM -0700, Peter Jalajas, TebucoSafe Backups wrote: > So implementing a strategy of security as a risk-management onion, and knowing that nothing is > ever perfectly secure, the Box Backup project could/should take reasonably aggressive measures to > use the available convenient processes that help ensure (assure?) the security of the source code > and binaries. Maybe you all do already, and I just haven't bother to track it all down. Apart from anything, I have the Box Backup sources stored within Box Backup :-) The entire svn and trac data is part of my Box store, so I'd say that's pretty safe. If, god forbid, this server was compromised, we have fully-encrypted backups. I think Ben takes offsite backups too. > So, yes, having widely disseminated sha1sums of everything goes a very long way towards that end > with very little pain, I think. Signatures go yet again a bit further, but with a little more > pain. (Insert other heavier lifting here.) You hosting a keysigning party in Cambs is probably > too aggressive and inconvenient, providing too little bang for the buck, as we say here. ;^) > > In a previous life as a Release Engineering manager for an enterprise security product, I appended > to our software build process an immediate automatic wide-dissemination of md5sums (md5 was state > of the art then, as I believe sha1 is now) of all individual source files, binaries, tarballs, Most people/projects are skipping SHA-1 and using SHA-256 now. (FreeBSD used to just do md5sums of third-party distfiles in ports, now it does md5sum and sha256.) James From boxbackup at fluffy.co.uk Tue Oct 9 18:34:34 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Tue, 9 Oct 2007 10:34:34 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <20071009064105.GA6936@netinertia.co.uk> Message-ID: <374017.59039.qm@web60617.mail.yahoo.com> X-Stamper-To: boxbackup at fluffy.co.uk Hi James, --- James O'Gorman wrote: > On Mon, Oct 08, 2007 at 02:10:06PM -0700, Peter Jalajas, TebucoSafe Backups wrote: > > So implementing a strategy of security as a risk-management onion, and knowing that nothing is > > ever perfectly secure, the Box Backup project could/should take reasonably aggressive measures > to > > use the available convenient processes that help ensure (assure?) the security of the source > code > > and binaries. Maybe you all do already, and I just haven't bother to track it all down. > > Apart from anything, I have the Box Backup sources stored within Box > Backup :-) The entire svn and trac data is part of my Box store, so I'd > say that's pretty safe. If, god forbid, this server was compromised, we > have fully-encrypted backups. I think Ben takes offsite backups too. Yes, definitely. That is an "availability" factor, where we could recover if we were compromised, _and noticed it_. What I was thinking, but couldn't yet verbalize, was something like this, maybe a code "chain of custody" kind of thing. 1. The security architecture of Box Backup was analyzed and the results published and openly discussed. For example, one section might say, "Box Backup contains no crypto code and instead uses freely available open-source external crypto libraries including openssl installed on the host system. You can find the code using the openssl libraries in these Box Backup source code files..." 2. The source code files are written, and at various milestones automatically hashed (and maybe signed) and the hashes published as widely as conveniently. This could be just a single hash of the milestone tarball. The user could simply Google the hash of his copy of the tarball, and should be able to see that same hash on multiple servers (bbdev.fluffy.co.uk, google, my server, others' servers, http://www.itconsult.co.uk/stamper.htm?, http://copyclaim.com?, etc.), providing reasonably good verification, helping us detect if Box Backup has been compromised. (Signing the tarball would maybe provide better security, but maybe at a bit more cost to everyone, I think worth it, but I'll let those that have to do it decide.) 3. For packaged binaries, the builders could then also perform and publish some kind of build-environment verification (source code, compiler tools, config files, output binaries, etc) to help us end-users trust the package. (http://sourceforge.net/projects/tripwire/ comes to mind.) This part, I know, is a lot of work, and of limited security value, but some reasonably aggressive level of it should be done, IMHO, and over time, the security value of it all should increase, I'm guessing (right now we have nothing). To paraphrase Chris, whom should the potential Box Backup users out there trust? We can help more end-users trust us, and Box Backup, by providing some more security-related information. I'm open to edification and suggestions. > > So, yes, having widely disseminated sha1sums of everything goes a very long way towards that > end > > with very little pain, I think. Signatures go yet again a bit further, but with a little more > > pain. (Insert other heavier lifting here.) You hosting a keysigning party in Cambs is > probably > > too aggressive and inconvenient, providing too little bang for the buck, as we say here. ;^) > > > > In a previous life as a Release Engineering manager for an enterprise security product, I > appended > > to our software build process an immediate automatic wide-dissemination of md5sums (md5 was > state > > of the art then, as I believe sha1 is now) of all individual source files, binaries, tarballs, > > Most people/projects are skipping SHA-1 and using SHA-256 now. (FreeBSD > used to just do md5sums of third-party distfiles in ports, now it does > md5sum and sha256.) Wow, I just did a bunch of googling on the hash matter. http://en.wikipedia.org/wiki/Cryptographic_hash . RIPEMD-160, SHA-512, Whirlpool, Tiger, egads. Based on what I've read this morning, sha1 should be fine for our purposes, but again, I'll leave that to the folks that have to do that work. Again, say what we do, do what we say, document it, and let the end-user decide. For the record, for example, to start the process, my fairly up-to-date and default Ubuntu has: pj at b:~/download$ dpkg -l | grep openssl ii openssl 0.9.8c-4ubuntu0.1 Secure Socket Layer (SSL) binary and related which yields, presuming rmd160 is the strongest hash that I have already installed: pj at b:~/download$ openssl rmd160 boxbackup* RIPEMD160(boxbackup-0.10_plus_chris_merge_1792.tgz)= e78f0288c8eb6ff14da7bc2d283a33e2c885f019 RIPEMD160(boxbackup-chris_general_1280-backup-client-mingw32.zip)= 58cec843115dc309bbf3d078deca0ea8a3e6a215 RIPEMD160(boxbackup-chris_general_1516-backup-client-mingw32.zip)= fec1062d9cddafe0220d86bc1e82d357b3b8c3bf RIPEMD160(boxbackup-chris_general_1569-backup-client-mingw32.zip)= 0cdae189f69974a9612433af8e62cb5e99c57207 RIPEMD160(boxbackup-chris_general_1857-backup-client-mingw32.zip)= bb541e1ac1b9b0500261093f439ed48f07d1908a What I'm saying is that eventually Chris' 1857 build with a rmd160 of bb541e1ac1b9b0500261093f439ed48f07d1908a should acquire a reputation, good or bad. But with a very commonly available and simple command, we all know that we are talking about the exact same package. Google will soon pick up that number and spread it around. Phew. Thanks again, James, Pete From boxbackup at fluffy.co.uk Tue Oct 9 18:37:36 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Tue, 9 Oct 2007 10:37:36 -0700 (PDT) Subject: [Box Backup] Testing Stamper Message-ID: <793094.29182.qm@web60622.mail.yahoo.com> -----BEGIN PGP SIGNED MESSAGE----- ######################################################## # # This is a proof of posting certificate from # stamper.itconsult.co.uk certifying that a user # claiming to be:- # pjalajas at tebucosafe.com # requested that this message be sent to:- # boxbackup at fluffy.co.uk # # This certificate was issued at 17:40 (GMT) # on Tuesday 09 October 2007 with reference 0464180 # # CAUTION: while the message may well be from the sender # indicated in the "From:" header, the sender # has NOT been authenticated by this service # # For information about the Stamper service see # http://www.itconsult.co.uk/stamper.htm # ######################################################## This is a test. This is only a test. To see what happens when using various versions of the Stamper system sent to a mailing list. pj at b:~/download$ openssl dgst -rmd160 boxbackup* RIPEMD160(boxbackup-0.10_plus_chris_merge_1792.tgz)= e78f0288c8eb6ff14da7bc2d283a33e2c885f019 RIPEMD160(boxbackup-chris_general_1280-backup-client-mingw32.zip)= 58cec843115dc309bbf3d078deca0ea8a3e6a215 RIPEMD160(boxbackup-chris_general_1516-backup-client-mingw32.zip)= fec1062d9cddafe0220d86bc1e82d357b3b8c3bf RIPEMD160(boxbackup-chris_general_1569-backup-client-mingw32.zip)= 0cdae189f69974a9612433af8e62cb5e99c57207 RIPEMD160(boxbackup-chris_general_1857-backup-client-mingw32.zip)= bb541e1ac1b9b0500261093f439ed48f07d1908a -----BEGIN PGP SIGNATURE----- Version: 2.6.3i Charset: noconv Comment: Stamper Reference Id: 0464180 iQEVAgUBRwu89oGVnbVwth+BAQHXqAf9GVcyoEvTmZyZesndkJUY/44b8qEgI5Uj 6jD4wE1zW3QtLShbEi/4DWBOKZbxtCvXQTO7TgP0RXKT7PMrU0eN1skFezLJ+IuQ q4Fqy9+tNJKpejwPtzDoH5Hore3W5Pgb/AvpxvKwJBgpX7vY0mlI/W/+GHVjdJV4 K1dfnMlY6QEziHnpgZalJRgDCvc6CeRONLXWO9r6RPel1w2m5TTaw6MJizMC5bqZ DGEtvl1y4XIUK+xmabDeil1qmw7BB9kZ38F4/vPL2qyu9wIENSYb1buhVl0d08io AvXU29HuOIX6qRwcFnwE3PPYzv47n/QQ6xY28hEldfjKTMtW03pjNg== =PQrt -----END PGP SIGNATURE----- From boxbackup at fluffy.co.uk Tue Oct 9 22:00:56 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Tue, 9 Oct 2007 14:00:56 -0700 (PDT) Subject: [Box Backup] Testing Hashes and Signatures Message-ID: <242660.852.qm@web60619.mail.yahoo.com> Sorry to waste bandwidth. Trying GunPG this time. It has SHA256, SHA512, and RIPEMD160. My email client will probably break the signature. At the bottom is an ascii-armored version. Maybe this will help someone someday... Suggestions welcome. (Maybe it would just be easier for the package maintainers to sign the packages...) Pete pj at b:~/download$ for x in SHA256 SHA512 RIPEMD160 ; do echo ${x} ; gpg --print-md ${x} boxbackup* ; echo "" ; done | gpg --clearsign -au 813F2332 You need a passphrase to unlock the secret key for user: "E.W. Peter Jalajas <...>" 3200-bit RSA key, ID 1C23B5F0, created 2003-09-23 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 SHA256 boxbackup-0.10_plus_chris_merge_1792.tgz: B5A71427 7DAAE276 FEBB7509 3F39C608 DD4391E8 5F95AB6A 383A7647 3D624E1E boxbackup-chris_general_1280-backup-client-mingw32.zip: 0A4443DC BEAC87AD 89DDEA5F 066B681F D8FDCAB1 2D4FBAF0 0DB08480 534B204E boxbackup-chris_general_1516-backup-client-mingw32.zip: BEF7693F D69C7A41 2E9A07DC 9C19A646 930BDE67 51AA7C0F 7560071D E0C8972E boxbackup-chris_general_1569-backup-client-mingw32.zip: 07BE1CF0 0B03F7CD 0F2B5030 5049EDBF B51B4098 149CB3E9 5D4BD5D6 1B7C2943 boxbackup-chris_general_1857-backup-client-mingw32.zip: 01708674 9DA2FBE3 29524394 7D7BBFE5 B6FF9267 724FDB00 6E7BEE0B 510F648A boxbackup-chris_general_784-backup-client-mingw32.zip: 44E8951E E362FEF4 B41ADB79 27721D07 3D2C1E6C 3BC08879 38F01F13 9F55929D SHA512 boxbackup-0.10_plus_chris_merge_1792.tgz: 365B23AB 247D262C 1CCE7503 E39A0B12 7C912CD0 240D4EB7 BAD2AD9F 119A7A5F 124BF1B9 506D0458 5392A74B 92B2037C C19E0540 F0A9745E 438F9532 DF42E46D boxbackup-chris_general_1280-backup-client-mingw32.zip: DD78C524 71A2A33D AFB9E772 83395B38 3F2A1971 1C9EC2E1 DF2EACF8 D5A27285 A6FA5645 8BDECB7B 2B947A09 950AA47F FB541D8B 8A92BF1F 77235900 1BA592FB boxbackup-chris_general_1516-backup-client-mingw32.zip: 67A7174E B43D3845 285D7E4C A0E8B72C 45656F53 CB8D95D1 2C68607C 20F34114 E49369F1 6F4B1BCA 27A47A1A 217E16E7 56895CCC BFBE8B89 7DFF7202 96620888 boxbackup-chris_general_1569-backup-client-mingw32.zip: 82784F9F 828FB8BF 48EAD1F7 DCDF0646 AC54614C 8ABE0C7F BFA0738E B3BD8AD2 736ECA42 5E07F5B6 0E259AED 7D6F4666 AE4E7750 EF5655C0 669C4EC3 51A8915C boxbackup-chris_general_1857-backup-client-mingw32.zip: BF25BAC5 B97FB4E8 E7BB64E2 27345C2B 32612CE8 26F8FEEB 77463D82 56791153 7B5BBDAB 3567A91A 4505FE1F B22609BF 43CB8F81 5EEC0031 89B746D3 84174261 boxbackup-chris_general_784-backup-client-mingw32.zip: 391D2425 3F47CBE7 FDF9210C D65C6EBA D70F5C02 6FFF44E0 0383A00B 7B674486 70645716 0459E2B5 5A66190A F25A3BCA 49F0502E 3421EE13 32428704 03D80264 RIPEMD160 boxbackup-0.10_plus_chris_merge_1792.tgz: E78F 0288 C8EB 6FF1 4DA7 BC2D 283A 33E2 C885 F019 boxbackup-chris_general_1280-backup-client-mingw32.zip: 58CE C843 115D C309 BBF3 D078 DECA 0EA8 A3E6 A215 boxbackup-chris_general_1516-backup-client-mingw32.zip: FEC1 062D 9CDD AFE0 220D 86BC 1E82 D357 B3B8 C3BF boxbackup-chris_general_1569-backup-client-mingw32.zip: 0CDA E189 F699 74A9 6124 33AF 8E62 CB5E 99C5 7207 boxbackup-chris_general_1857-backup-client-mingw32.zip: BB54 1E1A C1B9 B050 0261 093F 439E D48F 07D1 908A boxbackup-chris_general_784-backup-client-mingw32.zip: 2FF7 E735 D6C4 7120 3BB1 51D7 79C3 9350 1B4C 9FA0 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iQGlAwUBRwvoT7OW7D0cI7XwAQIHzQx+K0RSRR5Yb7aeNVea/FollE4ofGXKn8Eu AwtpQoICbXnl7dYPCLs3G4hQfQwkTsiq49E3LkbXp5gOBpoqnO8Po/9KoCUlGyt0 5wCYw2ufRCUxmJcp3EohjIM7tiXq4evRUCjSkL0bfcb3JpVVoKMQyA0KlADg0wPE tDN42XtCkG4ASBDM6pvfDd0n3uKB5yCL4i3iadE+HYhZG9F3lIMBtcItAAGQTXsT EfuyhuBJx3ywLmwsQHN4zMpySmKETYVne2W5mbJ6srZTisc5NnRoI1Qdqmoqn53k XHAOTVCJ7w+E2ZaNuPWyafX6XpJ7qS6KBqFdV2TsGRXLy+MKXicUnU7GWe1UDxlQ M0U+KJCsiXwqNLt7UVQobXU28tnsOcrFosH1gWKmOQOILL2Ga1pe8LVh965JxL7H l5lxXDOj1H75Ax3fsuyP5+Hd8NFuxeJczoe0ae1o+iNLh8vlks2r7ANDw/Mj4BAh f2b0aA9CGzqRElAz2PTqbXClTHKzmiLcPtQBcQFAn1hZvixfjpGbxg== =Cctm -----END PGP SIGNATURE----- pj at b:~/download$ for x in SHA256 SHA512 RIPEMD160 ; do echo ${x} ; gpg --print-md ${x} boxbackup* ; echo "" ; done | gpg -sau 813F2332 You need a passphrase to unlock the secret key for user: "E.W. Peter Jalajas <...>" 3200-bit RSA key, ID 1C23B5F0, created 2003-09-23 -----BEGIN PGP MESSAGE----- Version: GnuPG v1.4.6 (GNU/Linux) owGllnuo1/UZxz1Ko36mK5owmLjHNSPEy+d+KVZ+ns9Fi7ygUyRnB5Uzj2YmRyWV NtnFzS5gk8M2KRMrF0SDmnXS6ZJFbGMJ+2NSyoYTTDZ1G+6WbSu2929jl3864Om/ w/n++D7P835e7/fzfWzcmFGjew5+87efmXjjy5d63ri4ctTssRe/tGhOUtZ1Vt6/ ZeWKVfdu3jBdzJCid8O6zRt7V/UPrNnYe1/fwOq+XumjmrFp9bZbqMM2eWmUJ19S qso7apXZWxFJNx2zE4FKMTrKGsi2aBO7RDro5J3xpItTpsr6fyX/XWh13/q+gRXr eqUKYvp/nqxb07d+0/T71qxf/YBWM7at2YAGRDLG6JKJa8rBp0IhllKTbSScYxdk oxJayYklqWIapyZIFBbBBEFWG1bCDNOAlW7YBrg276JGFRezTwZVakzCo6WYZUzO OIpacKnOk5Up+SwaeeuE8LJQFTlEr4ZrwMXhFfBcZe4OxUI3nwuJptgKjemEibVw I7aSjYiBpImZdY1ki+FiiyPJPqto9Ac3EKwfvgHpRXDeUCxJNa6aVLQKOzegwjO3 aolda1FBAa9Mg/iCXEXfVTA0Ec2ZkD6wAR/MsPWNqSFaWalqp1pthtjIVNhHUt4r WUQXNJVldZk0ZxECHunQhGxSU2zWRhVLpwP8rVRXgr92lpVOTMr4opzKJHOuwF+j GVDAUpHPUapcBH4jiqnsiVNRqcRGEnz4LqpSGW6SYwcrc0UYC7PoqJI3TFGBUO0z gaYqrBHURIre2EoGQ0SrFZVmVDWujNxHpfiQsTXyMqmkdaHUOFboR0HraFkHOFol Gb3EkLFmVSXqKtiuweM2Ka+CpeRass7YDgUgn9lDG47GJyRCtCIl4xs1tkaWwBQS pmuwKOpoG4GF5IRtNB65IZ1HJHlTQYEuOhhL6Kv4ajIlUQN7bMlYZ12zmjKHEm2B abMLTkBlJZo2UhqqJmoXm+yQa4Yl5wSc0H6S+EP6KkEwWQf0ckb8APzAIQL51rwS iqJzCqiFkTs7KKDfwElQoXGAj02oqcjmqeTSRDdaUrbGScwWEiNLIC43pI8OmF9z CUCNvHY1J6NAVxW+WXYkqrIx1YJ2MZ1zeFE12LYVVBvEsRkORaCZmnU3tUKUNo88 Irgpy2iVOPrG8CvB/exMVdBUG5sVk1YOPsEj5VpotTKgME6XoKCyj1JiXZ4tc0nc IY3/pYhVGCtsq0CIlXIidkXS2GoLEtPWLISWOAmMVxVNwYAM1Blx1uCQFWWUhReM zwwCWkGwSZGR/za7yomKFw36KXDTGsIJydw9eAJZ5xk5aYIjj+VZL12HYPZYkddk k3MyikTQKukubiY2YYWqpI2StSKrNGoHLwzeWIJQznQ6C+9YUOcW6cSV5Fb1AfdR hUA5QGjXzjdJpiRPxFkVOEbjSmusJweYGkkZRx4tNuSK9xiNvLOFskYU4CpoIiQz ogNsgscUKOkKDpW0Izd/q1ni8GOEmEs3w6C+UqIQBcdI5wqaira+aw4Mr7l9iMOb S6Iq4fnmIoxvUiQgbAjKJZi2OsjHiOkYQT4ywX8IAyEx0T14zzgTxOAC+3OSSHS/ PXBsKxXTXapHmEVxxef0tv/VUggw+FNbIJ27B0EJnE18PiEICi54RCREbbtZjdyJ iJtO56GeA2Ou+tc35H8/Kkfv+f212xeuffXCsnk9u5ds+d1VPR977/A9/f3j9h9e NPiWm/DC6S1Lr3/ypkv7v3Fkt9o3ecaxznc/cmLg/bcP7p145w1nln71Zw/t/tGJ uadHnTr31N5xy3fcu3HqP76/aNLpr3+i7yerZv/xW48cvLR86nWvTvj4b8YO3f/0 0c/ftK3nmp+364+PPvnmnhUzP7d7Z+/hPee3D149/6+X6d2/P/v4rS9NW7zllSPH +n/649T5zsnLdu/aBxZOCxem7Z+3+Jnj5+65Y+Ufdr1xYOdrE05vnHK+99rbzjy3 /vE3dx498trld+Ysb29dNy49POaH7ujD26YNnHlwYMymHXumXnh+7V2Xj12Ys/WW X3xq/GNzp7x99urTM28+pxcPnpp54PyG8rVfzt988m+dPX39unfi0OvnaOjAiQcn vbBd/uWTvz609tanji/buuuLH51wzYufXXDDqYN/fmLyrJN3L90y7/kXdz97bvxd h5asmfflHWeHbv7B4Jmv7H1GXJz0nBrUT75T7nx09KPvL/nVlHefSGHy8Qt10ux9 Yxfe+N4+vl3dPjiwbMf8VxaMf33V1m9v+nQeGrPr0PcGZ33hpR0Lp499ZNbZP/0T =nKrK -----END PGP MESSAGE----- From boxbackup at fluffy.co.uk Thu Oct 11 17:41:44 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Thu, 11 Oct 2007 09:41:44 -0700 (PDT) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <426656.95816.qm@web60619.mail.yahoo.com> Message-ID: <42253.10277.qm@web60618.mail.yahoo.com> Hi all, Notwithstanding all my distracting chatter about the security of using such a thing...have we decided on a good client parcel for Mac G5 that I should use? Thanks for all your help! Pete From boxbackup at fluffy.co.uk Thu Oct 11 19:08:54 2007 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Thu, 11 Oct 2007 11:08:54 -0700 Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <42253.10277.qm@web60618.mail.yahoo.com> References: <42253.10277.qm@web60618.mail.yahoo.com> Message-ID: <470E66B6.9090203@reedtz.com> On 10/11/07 9:41 AM, Peter Jalajas, TebucoSafe Backups wrote: > Hi all, > > Notwithstanding all my distracting chatter about the security of using such a thing...have we > decided on a good client parcel for Mac G5 that I should use? > Chris: I don't know how far you got with the test failures on OS X? Once they are cleared up, I can build a G5 version... Also (with a bit of studying) I think I could build a universal binary... Let me know if that would be useful? Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Fri Oct 12 00:32:03 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 12 Oct 2007 00:32:03 +0100 (BST) Subject: [Box Backup] Help installing client on Mac OS X In-Reply-To: <470E66B6.9090203@reedtz.com> References: <42253.10277.qm@web60618.mail.yahoo.com> <470E66B6.9090203@reedtz.com> Message-ID: Hi Per and Pete, On Thu, 11 Oct 2007, Per Reedtz Thomsen wrote: > On 10/11/07 9:41 AM, Peter Jalajas, TebucoSafe Backups wrote: >> Hi all, >> >> Notwithstanding all my distracting chatter about the security of using >> such a thing...have we >> decided on a good client parcel for Mac G5 that I should use? >> > Chris: I don't know how far you got with the test failures on OS X? Once they > are cleared up, I can build a G5 version... Also (with a bit of studying) I > think I could build a universal binary... Not fixed yet, sorry. Been too busy. Can you give me another week or so? If you need to get started you can use the current code, I'll keep looking into the test failures and give you an update as soon as I can. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Fri Oct 12 15:56:41 2007 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Fri, 12 Oct 2007 16:56:41 +0200 Subject: [Box Backup] chris-merge - 2GB+ - Get OK - Compare fails? In-Reply-To: References: <46DBC799.6060406@kontrapunkt.com> <46DBC978.6020401@kontrapunkt.com> <46E4F4BB.9040603@kontrapunkt.com> <46E5000F.4000206@kontrapunkt.com> <46E506F2.1000804@kontrapunkt.com> <46EA4D21.7090107@kontrapunkt.com> <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> <47035AE8.6000208@kontrapunkt.com> Message-ID: <470F8B29.2070007@kontrapunkt.com> Hello Chris. I'm not familiar with the term top-posting, so I might unintentially be doing i again now. Sorry. The problem with the compare seems to be a with the MS Office files themselves (I only get errors on MS-files). I can restore the files from backup, and they have the same content as the original files. Despite this, the restored and original file have different MD5 sigs. I have found by testing, that the MD5 sig of office files change everytime they are opened even though they have not been changed by the user and saved back. So... No problem it seems. Thanks, Tobias Chris Wilson wrote: > Hi Tobias, > > Would you mind not top-posting? It makes our discussion difficult to > read for other list members who haven't read previous messages in detail > or who are new to the list. Thank you. I've fixed it here: > > On Wed, 3 Oct 2007, Tobias Balle-Petersen wrote: >> Chris Wilson wrote: >>> On Wed, 26 Sep 2007, Tobias Balle-Petersen wrote: >>> >>>> All the errors are of this format: >>>> >>>> file A has different contents to store file file B >>>> >>>> The compare ends with: >>>> [ 0 (of 8) differences probably due to file modifications after the >>>> last upload ] >>>> Differences: 8 (0 dirs excluded, 0 files excluded, 2 files not checked) >>>> Logging off... >>>> Exception: Connection Protocol_ObjTooBig (7/42) >>> >>> Are all the files which failed to compare properly over 2GB, or only the >>> first one? >> >> None of them ar bigger than 2GB. > > OK, that is strange and very worrying. I would very much appreciate your > help in fixing this. > > First off, please could you remind me what versions of Box Backup you > are running on the client and the server? > > Secondly, please could you make a copy of one of the files that's > failing to compare, in the same directory, and check that the copy is > backed up, and then run a compare and see if the copy fails to compare > too? That should establish whether the problem is on the client side, or > with an old client which generated files which now can't be patched. > > Cheers, Chris. From boxbackup at fluffy.co.uk Fri Oct 12 19:23:10 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 12 Oct 2007 19:23:10 +0100 (BST) Subject: [Box Backup] chris-merge - 2GB+ - Get OK - Compare fails? In-Reply-To: <470F8B29.2070007@kontrapunkt.com> References: <46DBC799.6060406@kontrapunkt.com> <46DBC978.6020401@kontrapunkt.com> <46E4F4BB.9040603@kontrapunkt.com> <46E5000F.4000206@kontrapunkt.com> <46E506F2.1000804@kontrapunkt.com> <46EA4D21.7090107@kontrapunkt.com> <170FF7F0-C92B-46E6-8BF6-154951CBDFEA@kontrapunkt.com> <47035AE8.6000208@kontrapunkt.com> <470F8B29.2070007@kontrapunkt.com> Message-ID: Hi Tobias, On Fri, 12 Oct 2007, Tobias Balle-Petersen wrote: > I'm not familiar with the term top-posting, so I might unintentially be doing i again now. Sorry. http://en.wikipedia.org/wiki/Top-posting#Top-posting > The problem with the compare seems to be a with the MS Office files > themselves (I only get errors on MS-files). > > I can restore the files from backup, and they have the same content as > the original files. Despite this, the restored and original file have > different MD5 sigs. I have found by testing, that the MD5 sig of office > files change everytime they are opened even though they have not been > changed by the user and saved back. OK, that's been seen by other users too. See: http://bbdev.fluffy.co.uk/trac/wiki/FAQ#what-does-the-has-different-contents-to-store-file-log-message-mean > So... No problem it seems. OK, that's great, thanks for letting me know :-) Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon Oct 15 18:20:03 2007 From: boxbackup at fluffy.co.uk (Peter Jalajas, TebucoSafe Backups) Date: Mon, 15 Oct 2007 10:20:03 -0700 (PDT) Subject: [Box Backup] Windows archive attribute? "BackupIfModifiedSince"? Configurable by BackupLocation? Message-ID: <728870.43960.qm@web60612.mail.yahoo.com> Hi all, I was wondering whether Box Backup pays any attention to the Windows attribute flags? Here's why. Say a client has a very large pile of files that includes some "currently" active files that she wants to backup with Box Backup on a regular basis, and some other "inactive" files that she does NOT want to backup with Box Backup. But using Excludes just wouldn't work. She _could_ backup all of their (inactive) files to some other media, and then mark all of her files as having been backed up by removing the (possibly counter-intuitive) Archive attribute, and then it would be neat if Box Backup could then look for files that have the Archive flag set (marking it for to be backed up) and back them up. And now that I've typed all that, maybe a better, easier, more portable way could be a new "BackupIfModifiedSince" configuration setting that could be set to some date in the past, so that files modified before that date would be ignored by Box Backup, but those modified afterwards would be backed up by Box Backup. She could backup her gajillion files modified before a certain date to other media (for example, DVD), and then use Box Backup to only protect her more recent files. (I guess it would work very similar to a "BackupIfSizeIs" setting (a similar feature request is already under "Don't backup files bigger than X kbytes").) Hmm, maybe it'd also be useful if it (and maybe some other settings, too) could be set differently under each BackupLocation. Suggestions welcome. Thanks, Pete From boxbackup at fluffy.co.uk Wed Oct 17 10:32:35 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 17 Oct 2007 10:32:35 +0100 Subject: [Box Backup] Trunk_1871 Message-ID: Hi Chris, I have this morning checked out 1871 and tried to compile and test, however the tests failed. FAILED: 2 tests failed (first at ../../lib/server/ServerControl.h:136) -------- common: PASSED crypto: PASSED compress: PASSED raidfile: PASSED basicserver: PASSED backupstore: PASSED backupstorefix: PASSED backupstorepatch: PASSED backupdiff: PASSED bbackupd: FAILED: 2 tests failed (first at ../../lib/server/ ServerControl.h:136) make: *** [release/common/test] Error 1 Just thought you might like to know... btw any joy with the Mac ? Regards Matt Brown matt at mbrown.co.uk From boxbackup at fluffy.co.uk Wed Oct 17 10:35:45 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 17 Oct 2007 10:35:45 +0100 (BST) Subject: [Box Backup] Trunk_1871 In-Reply-To: References: Message-ID: Hi Matt, On Wed, 17 Oct 2007, Matt Brown wrote: > I have this morning checked out 1871 and tried to compile and test, however > the tests failed. > > FAILED: 2 tests failed (first at ../../lib/server/ServerControl.h:136) > -------- > common: PASSED > crypto: PASSED > compress: PASSED > raidfile: PASSED > basicserver: PASSED > backupstore: PASSED > backupstorefix: PASSED > backupstorepatch: PASSED > backupdiff: PASSED > bbackupd: FAILED: 2 tests failed (first at > ../../lib/server/ServerControl.h:136) > make: *** [release/common/test] Error 1 > > Just thought you might like to know... What platform, Mac? Do you have a bbackupd or bbstored already running? Could you try running just the bbackupd test with ./runtest bbackupd? Could you look out for "FAILURE:" lines in the test output and paste the surrounding lines? > btw any joy with the Mac ? A little, still debugging a couple of issues but i think that it basically works even though the tests don't pass. Thanks for lending it to me! Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Oct 17 10:44:58 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 17 Oct 2007 10:44:58 +0100 Subject: [Box Backup] Trunk_1871 In-Reply-To: References: Message-ID: Hi Chris, >> I have this morning checked out 1871 and tried to compile and >> test, however the tests failed. >> >> FAILED: 2 tests failed (first at ../../lib/server/ServerControl.h: >> 136) >> -------- >> common: PASSED >> crypto: PASSED >> compress: PASSED >> raidfile: PASSED >> basicserver: PASSED >> backupstore: PASSED >> backupstorefix: PASSED >> backupstorepatch: PASSED >> backupdiff: PASSED >> bbackupd: FAILED: 2 tests failed (first at ../../lib/server/ >> ServerControl.h:136) >> make: *** [release/common/test] Error 1 >> >> Just thought you might like to know... > > What platform, Mac? Do you have a bbackupd or bbstored already > running? Could you try running just the bbackupd test with ./ > runtest bbackupd? Could you look out for "FAILURE:" lines in the > test output and paste the surrounding lines? > No, this is a Ubuntu Dapper box, no clients running (I use this box for building) - I shall re-run and paste the info. >> btw any joy with the Mac ? > > A little, still debugging a couple of issues but i think that it > basically works even though the tests don't pass. Thanks for > lending it to me! > No probs, I was just checking you could connect to it (it had fallen asleep) :-) and you are of course most welcome. Matt From boxbackup at fluffy.co.uk Wed Oct 17 10:59:40 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 17 Oct 2007 10:59:40 +0100 Subject: [Box Backup] Trunk_1871 In-Reply-To: References: Message-ID: Hi Chris, >> bbackupd: FAILED: 2 tests failed (first at ../../lib/server/ >> ServerControl.h:136) >> make: *** [release/common/test] Error 1 >> >> Just thought you might like to know... > > What platform, Mac? Do you have a bbackupd or bbstored already > running? Could you try running just the bbackupd test with ./ > runtest bbackupd? Could you look out for "FAILURE:" lines in the > test output and paste the surrounding lines? Re-ran and all passed this time ?? this was the output I got for the previous tests: NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors 2003-2007 Waiting for backup daemon to start: . done. waiting: ............ done. NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors 2003-2007 Waiting for backup daemon to start: . done. waiting: ............ NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors 2003-2007 NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors 2003-2007 FAILURE: Condition [killed] failed at ../../lib/server/ServerControl.h (136) FAILURE: Condition [KillServer(pid)] failed at testbbackupd.cpp(698) Matt From boxbackup at fluffy.co.uk Wed Oct 17 13:21:16 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 17 Oct 2007 13:21:16 +0100 (BST) Subject: [Box Backup] Trunk_1871 In-Reply-To: References: Message-ID: Hi Matt, On Wed, 17 Oct 2007, Matt Brown wrote: > Re-ran and all passed this time ?? > > this was the output I got for the previous tests: > > NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors > 2003-2007 > Waiting for backup daemon to start: . done. > waiting: ............ > done. > NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors > 2003-2007 > Waiting for backup daemon to start: . done. > waiting: ............ > NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors > 2003-2007 > NOTICE: Box Backup Client vtrunk_1871, (c) Ben Summers and contributors > 2003-2007 > FAILURE: Condition [killed] failed at ../../lib/server/ServerControl.h(136) > FAILURE: Condition [KillServer(pid)] failed at testbbackupd.cpp(698) Not sure what happened there, sorry. Perhaps we didn't wait long enough after killing the old process before starting the new one, and it hadn't actually shut down yet, so the new one failed to start. I've seen this occasionally but it's not reproducible and the fact that the tests pass mean that Box is OK. Moral of the story: if at first you don't succeed, try, try again :-) Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Oct 18 21:52:29 2007 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Thu, 18 Oct 2007 21:52:29 +0100 Subject: [Box Backup] Re: [Box Backup-dev] Re:BoxReport.pl (was ChrisMerge_1828) In-Reply-To: References: <743781.65915.qm@web60615.mail.yahoo.com> <1ACB8805-3754-44CB-AFFD-F36E62DF520A@mbrown.co.uk> <46F008C0.8070805@logical-progress.com> <7E797529-84DB-466B-B05C-33D8E89BBE4A@mbrown.co.uk> Message-ID: Hi Chris, >> The script can be run by hand with or without command line >> parameters (will default to the last found backup in the log >> file). Script does require ExtendedLogging = yes to get the data >> it needs. > > You might want to look into using LogAllFileAccess messages instead > of ExtendedLogging. ExtendedLogging might be slower (it's on the > network path) and LogAllFileAccess should include full paths in > each message. It's also more readable, IMHO. Hopefully the output > is not difficult to parse. > >> -- Chris I might try altering the code for the backup-client to >> call boxreport.pl if it finds ReportScript = /usr/local/bin/ >> boxreport.pl (or similar) in bbackupd.conf (if I can work out how >> to do it :-) that way I can get the the report to generate and >> mail right after the backup has completed rather than having to >> wait for cron etc. > > There should be no need for that, now that we have backup-start and > backup-finish hooks for NotifyScript. When NotifyScript is called > for backup-start, it could record the current time. When called for > backup-finish, it could generate a report from the backup-start > time to the current time. Following on from this, is it possible to send an interrupt signal to bbackupd ? i.e something like bbackupctl -q stop or bbackupctl -q interrupt ? The reasoning behind it is we only allow clients to backup between a certain window each day, more for there benefit so it does not saturate their line and cause us to get "My internet connection is really slow etc" type calls. At the mo I simply run a cron at two certain times a day, and issue a boxbackup-client stop which in turn just kills the process and isnt very clean.... its not very often a backup takes a huge amount of time, until someone decides to offload their entire collection of MP3, Docs, Pron etc somewhere on the mount point :-) Then NotifySysAdmin could get a handle like backup-stopped or backup- interrupted ? The report can then take this action into account and also means we can keep the daemon running without having to keep starting and stopping each day. Your thoughts ? Matt From boxbackup at fluffy.co.uk Fri Oct 19 09:06:20 2007 From: boxbackup at fluffy.co.uk (Lesinigo Luca) Date: Fri, 19 Oct 2007 10:06:20 +0200 Subject: [Box Backup] Feedback and issues with boxbackup-0.10 on OpenSolaris/ZFS Message-ID: <807A909F-4446-4EC2-96A9-6424BF69FE72@lm-net.it> Hello there. Hope this is the right place for this stuff. I'm used to run boxbackup-0.10 in linux, but I'm moving it on a solaris box. Solaris express community edition, svn_67, on good old 32bit x86. 1) I more or less followed http://bbdev.fluffy.co.uk/trac/wiki/ CompilingOnSolaris and it compiled well, but "make install-backup- server" did not work. You can read the exact commands I used at http://www.semlug.net/moin/OpenSolaris/BoxBackup - sorry for the Italian comments, but the commands should be pretty clear. "make install-backup-server" gives me [1], repeated once for every binary in the server parcel. 2) I manually installed boxbackup server on the system (as you can read on the semlug.net wiki) and it seems to be working without problems on my SXCE install. It's happily running inside a solaris container living on a ZFS filesystem, hope this feedback sounds good to you as it does to me :) 2b) zfs with compression=on (uses lzjb, a really light algorithm, much faster than gzip) reports a 1.07 compression factor on a 50GB boxbackup store. Maybe this means you can squeeze something more off the bbackup compression? 3) I kept getting errors like "while housekeeping account 00040001, exception Common OSFileError (Error accessing a file. Check permissions.) (1/9) -- aborting" [from bbstored] and "Exception: RaidFile RaidFileDoesntExist (2/11)" [from bbstoreaccounts info], when the real problem was that I put the 'account directories' directly under the raidfile directory (like $dir/$accountsdir), while bbstored expected $dir/backup/$accountsdir. Mine was just a stupid error transferring the data from a server to another, but the error messages weren't useful at all, they were misleading! 4) This is more of a feature request, but having a solaris smf(5) manifest for boxbackup would be great. I tried to look into it but it proved to be a more challenging task than what I expected, having never wrote a manifest. I must say I'm more or less a beginner with solaris. I hope this mail is useful to you. I'll keep my subscription to the m.l. for a while so I'll read any reply to this. If boxbackup-0.10 on solaris proves to be stable as it seems to be, I'll convert my production servers (I offer an internet boxbackup server service to my customers) to solaris on ZFS. Meanwhile, thanks for boxbackup and keep up with the good work! :) [1] partial output of make install-backup-server: (cd parcels/boxbackup-0.10-backup-server-solaris2.11; ./install- backup-server) find: stat() error /opt/boxbackup/bin: No such file or directory find: cycle detected for /lib/32/ find: cycle detected for /lib/secure/32/ find: cycle detected for /usr/lib/locale/common/32/ find: cycle detected for /usr/lib/locale/common/LC_CTYPE/32/ find: cycle detected for /usr/lib/locale/common/LO_LTYPE/32/ find: cycle detected for /usr/lib/secure/32/ find: cycle detected for /usr/lib/32/ find: cycle detected for /usr/lib/lwp/32/ find: cycle detected for /usr/lib/link_audit/32/ find: cannot follow symbolic link /usr/lib/lu/lu_activate_sync: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lu_init_sync_start: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lu_init_sync_stop: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lucreatezone: No such file or directory find: cannot follow symbolic link /usr/lib/lu/ludevices: No such file or directory find: cannot follow symbolic link /usr/lib/lu/ludo: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lufslist: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lumount: No such file or directory find: cannot follow symbolic link /usr/lib/lu/luprintf: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lurootdev: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lustatus: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lusync: No such file or directory find: cannot follow symbolic link /usr/lib/lu/luumount: No such file or directory [....................] -- Lesinigo Luca LM Networks S.r.l. From boxbackup at fluffy.co.uk Fri Oct 19 11:18:05 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Fri, 19 Oct 2007 11:18:05 +0100 Subject: [Box Backup] Feedback and issues with boxbackup-0.10 on OpenSolaris/ZFS In-Reply-To: <807A909F-4446-4EC2-96A9-6424BF69FE72@lm-net.it> References: <807A909F-4446-4EC2-96A9-6424BF69FE72@lm-net.it> Message-ID: <20071019101804.GL55134@netinertia.co.uk> On Fri, Oct 19, 2007 at 10:06:20AM +0200, Lesinigo Luca wrote: > Hello there. Hope this is the right place for this stuff. I'm used to run > boxbackup-0.10 in linux, but I'm moving it on a solaris box. Solaris express > community edition, svn_67, on good old 32bit x86. > > 1) I more or less followed > http://bbdev.fluffy.co.uk/trac/wiki/CompilingOnSolaris and it compiled well, > but "make install-backup-server" did not work. You can read the exact > commands I used at http://www.semlug.net/moin/OpenSolaris/BoxBackup - sorry > for the Italian comments, but the commands should be pretty clear. "make > install-backup-server" gives me [1], repeated once for every binary in the > server parcel. The "install-backup-server" script just calls the install command for each file, so it's possible it's a bug in install in OpenSolaris. Looking at your output it does look like this may be the case, so you might need to open a ticket on http://bugs.opensolaris.org/ . > 2) I manually installed boxbackup server on the system (as you can read on > the semlug.net wiki) and it seems to be working without problems on my SXCE > install. It's happily running inside a solaris container living on a ZFS > filesystem, hope this feedback sounds good to you as it does to me :) Sounds good :-) > 4) This is more of a feature request, but having a solaris smf(5) manifest > for boxbackup would be great. I tried to look into it but it proved to be a > more challenging task than what I expected, having never wrote a manifest. I > must say I'm more or less a beginner with solaris. I might have a look at this this weekend if I get a chance. I'll commit it straight to trunk if I get it working. James From boxbackup at fluffy.co.uk Fri Oct 19 15:26:22 2007 From: boxbackup at fluffy.co.uk (Wout Mertens) Date: Fri, 19 Oct 2007 16:26:22 +0200 Subject: [Box Backup] Hash collisions and boxbackup Message-ID: <5681F7A7-FA4A-480A-80ED-1523B75102E2@cisco.com> Hey all, I thought you might be interested in the odds of having hash collisions. http://www.backupcentral.com/content/view/145/47/ Basically, if you use SHA-1 and assume a flat distribution of hash values and 4KB blocks, you need 95 Exabyte of data before you should worry... Wout. From boxbackup at fluffy.co.uk Sun Oct 21 16:04:58 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 21 Oct 2007 16:04:58 +0100 (BST) Subject: [Box Backup] Interrupting backups in progress In-Reply-To: References: <743781.65915.qm@web60615.mail.yahoo.com> <1ACB8805-3754-44CB-AFFD-F36E62DF520A@mbrown.co.uk> <46F008C0.8070805@logical-progress.com> <7E797529-84DB-466B-B05C-33D8E89BBE4A@mbrown.co.uk> Message-ID: Hi Matt, On Thu, 18 Oct 2007, Matt Brown wrote: >> > -- Chris I might try altering the code for the backup-client to call >> > boxreport.pl if it finds ReportScript = /usr/local/bin/boxreport.pl (or >> > similar) in bbackupd.conf (if I can work out how to do it :-) that way I >> > can get the the report to generate and mail right after the backup has >> > completed rather than having to wait for cron etc. >> >> There should be no need for that, now that we have backup-start and >> backup-finish hooks for NotifyScript. When NotifyScript is called for >> backup-start, it could record the current time. When called for >> backup-finish, it could generate a report from the backup-start time to the >> current time. > > Following on from this, is it possible to send an interrupt signal to > bbackupd ? i.e something like bbackupctl -q stop or bbackupctl -q interrupt > ? You can send a "reload" command with bbackupctl, that should stop any backup currently in progress. > The reasoning behind it is we only allow clients to backup between a > certain window each day, more for there benefit so it does not saturate > their line and cause us to get "My internet connection is really slow > etc" type calls. At the mo I simply run a cron at two certain times a > day, and issue a boxbackup-client stop which in turn just kills the > process and isnt very clean.... its not very often a backup takes a huge > amount of time, until someone decides to offload their entire collection > of MP3, Docs, Pron etc somewhere on the mount point :-) But it's much better to be able to sync during the day without disruption. Perhaps a router with QoS support might help? Or perhaps we could add rate limiting and schedules to bbackupd? Currently, reload and terminate will not update the StoreObjectInfoFile, so regularly using them will significantly reduce the network efficiency of Box. I'm not sure that it's actually necessary to terminate uncleanly rather than cleanly in this case. Any thoughts, anyone? > Then NotifySysAdmin could get a handle like backup-stopped or > backup-interrupted ? The report can then take this action into account > and also means we can keep the daemon running without having to keep > starting and stopping each day. At the moment it will get backup-stopped, we could add an interrupt command and a backup-interrupted message. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun Oct 21 16:52:56 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 21 Oct 2007 16:52:56 +0100 (BST) Subject: [Box Backup] Feedback and issues with boxbackup-0.10 on OpenSolaris/ZFS In-Reply-To: <807A909F-4446-4EC2-96A9-6424BF69FE72@lm-net.it> References: <807A909F-4446-4EC2-96A9-6424BF69FE72@lm-net.it> Message-ID: Hi Lesinigo, On Fri, 19 Oct 2007, Lesinigo Luca wrote: > Hello there. Hope this is the right place for this stuff. I'm used to run > boxbackup-0.10 in linux, but I'm moving it on a solaris box. Solaris express > community edition, svn_67, on good old 32bit x86. > > 1) I more or less followed > http://bbdev.fluffy.co.uk/trac/wiki/CompilingOnSolaris and it compiled well, > but "make install-backup-server" did not work. You can read the exact > commands I used at http://www.semlug.net/moin/OpenSolaris/BoxBackup - sorry > for the Italian comments, but the commands should be pretty clear. "make > install-backup-server" gives me [1], repeated once for every binary in the > server parcel. ... > [1] partial output of make install-backup-server: > (cd parcels/boxbackup-0.10-backup-server-solaris2.11; > ./install-backup-server) > find: stat() error /opt/boxbackup/bin: No such file or directory > find: cycle detected for /lib/32/ > find: cycle detected for /lib/secure/32/ ... > find: cannot follow symbolic link /usr/lib/lu/lu_activate_sync: No such file > find: or directory > find: cannot follow symbolic link /usr/lib/lu/lu_init_sync_start: No such > find: file or directory I'm afraid I don't know what that means, but the install script should only reference the install program, which should be a Unix standard command and should have nothing to do with "find". Perhaps you have a broken install script in your PATH? > 2) I manually installed boxbackup server on the system (as you can read > on the semlug.net wiki) and it seems to be working without problems > on my SXCE install. It's happily running inside a solaris container > living on a ZFS filesystem, hope this feedback sounds good to you as > it does to me :) Yes, that's excellent. You might like to disable Box Backup's own feature that keeps old version of files, and use ZFS to do that instead (by being able to take the repository back in time). > 2b) zfs with compression=on (uses lzjb, a really light algorithm, much > faster than gzip) reports a 1.07 compression factor on a 50GB boxbackup > store. Maybe this means you can squeeze something more off the bbackup > compression? We deliberately compress quite small blocks in order to make it possible to use an rsync-like algorithm on them. Perhaps we could do additional compression on the server, but for 7% gain it doesn't sound that useful. > 3) I kept getting errors like "while housekeeping account 00040001, > exception Common OSFileError (Error accessing a file. Check > permissions.) (1/9) -- aborting" [from bbstored] and "Exception: > RaidFile RaidFileDoesntExist (2/11)" [from bbstoreaccounts info], > when the real problem was that I put the 'account directories' > directly under the raidfile directory (like $dir/$accountsdir), while > bbstored expected $dir/backup/$accountsdir. Mine was just a stupid > error transferring the data from a server to another, but the error > messages weren't useful at all, they were misleading! The error messages have been improved a lot in the current development version, which I hope we will release soon. Did you get an exception logged just before that, and if so what was the file and line number? You could help us a lot by testing the latest trunk rather than using 0.10 which is very old. Have you read about the known problems with 0.10 at [http://bbdev.fluffy.co.uk/trac/wiki/Installation]? > 4) This is more of a feature request, but having a solaris smf(5) > manifest for boxbackup would be great. I tried to look into it but it > proved to be a more challenging task than what I expected, having > never wrote a manifest. I must say I'm more or less a beginner with > solaris. Sorry, I know less about SMF files than you do, I have no idea how I would do it or what I would want one for. If you do write one, please tell us about it and we'll add it to the distribution of you agree. > I hope this mail is useful to you. I'll keep my subscription to the m.l. > for a while so I'll read any reply to this. If boxbackup-0.10 on solaris > proves to be stable as it seems to be, I'll convert my production > servers (I offer an internet boxbackup server service to my customers) > to solaris on ZFS. > > Meanwhile, thanks for boxbackup and keep up with the good work! :) Cheers, Chis. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun Oct 21 19:06:54 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sun, 21 Oct 2007 19:06:54 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting Message-ID: <20071021180654.GO55134@netinertia.co.uk> Hi everyone, We touched in this briefly recently, and I've had a short conversation with Ben about it and we thought we'd throw this open to the community. Currently the website, listserver and development server are all hosted by different people at different places. While this isn't necessarily a bad thing, they're under different domains which can make it tricky for people to remember where the different things are :-) Ben said that he has boxbackup.org available for use, which we could move everything under. The key question is - how would we want to manage this? I have no problem extending my hosting to cover everything, although if anyone else wants to chip in obviously that's welcome too. I've proposed that I could host it all under a FreeBSD Jail (similar to a Solaris Zone, for those who are familiar) so that it could all be easily be manageable, and I could give the key developers admin access to this jail so that it's not just me or Ben being relied on. So, comments, anyone? James From boxbackup at fluffy.co.uk Sun Oct 21 20:41:43 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 21 Oct 2007 20:41:43 +0100 (BST) Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <20071021180654.GO55134@netinertia.co.uk> References: <20071021180654.GO55134@netinertia.co.uk> Message-ID: Hi James and all, On Sun, 21 Oct 2007, James O'Gorman wrote: > We touched in this briefly recently, and I've had a short conversation > with Ben about it and we thought we'd throw this open to the community. > > Currently the website, listserver and development server are all hosted > by different people at different places. While this isn't necessarily a > bad thing, they're under different domains which can make it tricky for > people to remember where the different things are :-) > > Ben said that he has boxbackup.org available for use, which we could > move everything under. The key question is - how would we want to manage > this? I have no problem extending my hosting to cover everything, > although if anyone else wants to chip in obviously that's welcome too. > > I've proposed that I could host it all under a FreeBSD Jail (similar to > a Solaris Zone, for those who are familiar) so that it could all be > easily be manageable, and I could give the key developers admin access > to this jail so that it's not just me or Ben being relied on. > > So, comments, anyone? +1 from me. Especially to be able to update the "official" Box Backup website :-) I'd be happy to chip in whatever I can. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun Oct 21 21:27:33 2007 From: boxbackup at fluffy.co.uk (Dave Bamford) Date: Sun, 21 Oct 2007 21:27:33 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <20071021180654.GO55134@netinertia.co.uk> References: <20071021180654.GO55134@netinertia.co.uk> Message-ID: <471BB635.8000803@logical-progress.com> Sounds good to me. Dave Bamford James O'Gorman wrote: > Hi everyone, > > We touched in this briefly recently, and I've had a short conversation > with Ben about it and we thought we'd throw this open to the community. > > Currently the website, listserver and development server are all hosted > by different people at different places. While this isn't necessarily a > bad thing, they're under different domains which can make it tricky for > people to remember where the different things are :-) > > Ben said that he has boxbackup.org available for use, which we could > move everything under. The key question is - how would we want to manage > this? I have no problem extending my hosting to cover everything, > although if anyone else wants to chip in obviously that's welcome too. > > I've proposed that I could host it all under a FreeBSD Jail (similar to > a Solaris Zone, for those who are familiar) so that it could all be > easily be manageable, and I could give the key developers admin access > to this jail so that it's not just me or Ben being relied on. > > So, comments, anyone? > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > From boxbackup at fluffy.co.uk Sun Oct 21 22:31:53 2007 From: boxbackup at fluffy.co.uk (Mikael Syska) Date: Sun, 21 Oct 2007 23:31:53 +0200 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <471BB635.8000803@logical-progress.com> References: <20071021180654.GO55134@netinertia.co.uk> <471BB635.8000803@logical-progress.com> Message-ID: <471BC549.4070700@syska.dk> Hi, Great ... I allways use google to find the site .. so this seems like a step in the right direction. // ouT Dave Bamford wrote: > Sounds good to me. > > Dave Bamford > > > > James O'Gorman wrote: >> Hi everyone, >> >> We touched in this briefly recently, and I've had a short conversation >> with Ben about it and we thought we'd throw this open to the community. >> >> Currently the website, listserver and development server are all hosted >> by different people at different places. While this isn't necessarily a >> bad thing, they're under different domains which can make it tricky for >> people to remember where the different things are :-) >> >> Ben said that he has boxbackup.org available for use, which we could >> move everything under. The key question is - how would we want to manage >> this? I have no problem extending my hosting to cover everything, >> although if anyone else wants to chip in obviously that's welcome too. >> >> I've proposed that I could host it all under a FreeBSD Jail (similar to >> a Solaris Zone, for those who are familiar) so that it could all be >> easily be manageable, and I could give the key developers admin access >> to this jail so that it's not just me or Ben being relied on. >> >> So, comments, anyone? >> >> James >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Mon Oct 22 05:56:17 2007 From: boxbackup at fluffy.co.uk (Per Reedtz Thomsen) Date: Sun, 21 Oct 2007 21:56:17 -0700 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <20071021180654.GO55134@netinertia.co.uk> References: <20071021180654.GO55134@netinertia.co.uk> Message-ID: <471C2D71.8040504@reedtz.com> On 10/21/07 11:06 AM, James O'Gorman wrote: > Hi everyone, > > We touched in this briefly recently, and I've had a short conversation > with Ben about it and we thought we'd throw this open to the community. > > Currently the website, listserver and development server are all hosted > by different people at different places. While this isn't necessarily a > bad thing, they're under different domains which can make it tricky for > people to remember where the different things are :-) > > Ben said that he has boxbackup.org available for use, which we could > move everything under. The key question is - how would we want to manage > this? I have no problem extending my hosting to cover everything, > although if anyone else wants to chip in obviously that's welcome too. > > I've proposed that I could host it all under a FreeBSD Jail (similar to > a Solaris Zone, for those who are familiar) so that it could all be > easily be manageable, and I could give the key developers admin access > to this jail so that it's not just me or Ben being relied on. > > So, comments, anyone? > Sounds great to me. I'll be happy to chip in a bit too. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Mon Oct 22 08:12:26 2007 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Mon, 22 Oct 2007 08:12:26 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <20071021180654.GO55134@netinertia.co.uk> References: <20071021180654.GO55134@netinertia.co.uk> Message-ID: <471C4D5A.4090708@hickinbottom.com> Sounds good to me. It would be nice to get that all in place prior to the 0.11 release as it's possible that will generate a bit more interest and new users (given the better support for Windows clients). Stuart James O'Gorman wrote: > Hi everyone, > > We touched in this briefly recently, and I've had a short conversation > with Ben about it and we thought we'd throw this open to the community. > > Currently the website, listserver and development server are all hosted > by different people at different places. While this isn't necessarily a > bad thing, they're under different domains which can make it tricky for > people to remember where the different things are :-) > > Ben said that he has boxbackup.org available for use, which we could > move everything under. The key question is - how would we want to manage > this? I have no problem extending my hosting to cover everything, > although if anyone else wants to chip in obviously that's welcome too. > > I've proposed that I could host it all under a FreeBSD Jail (similar to > a Solaris Zone, for those who are familiar) so that it could all be > easily be manageable, and I could give the key developers admin access > to this jail so that it's not just me or Ben being relied on. > > So, comments, anyone? > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > From boxbackup at fluffy.co.uk Mon Oct 22 22:08:26 2007 From: boxbackup at fluffy.co.uk (Peter Porter) Date: Mon, 22 Oct 2007 15:08:26 -0600 Subject: [Box Backup] ExtendedLogging in Windows build In-Reply-To: <20071019110002.2349.96965.Mailman@love.warhead.org.uk> References: <20071019110002.2349.96965.Mailman@love.warhead.org.uk> Message-ID: <471D114A.7010805@brightavenue.com> I'm getting an error accessing a file from one of my BoxBackup clients running Win32 build "vchris_general_568": Exception caught (Common OSFileError (Error accessing a file. Check permissions.) 1/9), reset state and waiting to retry... I've enabled ExtendedLogging to figure out what file it's hanging up on, but I don't see any extended logging. Any advice? Where does the ExtendedLogging go on Win32 builds? I'm also curious why it hangs up on one file and (seemingly) fails to continue backing up the rest of the definition. Thanks! Peter From boxbackup at fluffy.co.uk Tue Oct 23 00:13:52 2007 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 23 Oct 2007 00:13:52 +0100 (BST) Subject: [Box Backup] ExtendedLogging in Windows build In-Reply-To: <471D114A.7010805@brightavenue.com> References: <20071019110002.2349.96965.Mailman@love.warhead.org.uk> <471D114A.7010805@brightavenue.com> Message-ID: Hi Peter, On Mon, 22 Oct 2007, Peter Porter wrote: > I'm getting an error accessing a file from one of my BoxBackup clients > running Win32 build "vchris_general_568": That build is extremely old and I don't support it any more. Please could you try the latest one? > Exception caught (Common OSFileError (Error accessing a file. Check > permissions.) 1/9), reset state and waiting to retry... > > I've enabled ExtendedLogging to figure out what file it's hanging up on, but > I don't see any extended logging. ExtendedLogging may not help you much here, it's a protocol log rather than a file access log. If the latest build doesn't give you better error messages, please report whatever it does tell you (especially the file and line number of the exception report) and try setting LogAllFileAccess = yes and run bbackupd with the -V option. > Any advice? Where does the > ExtendedLogging go on Win32 builds? To the console, if you run it from the console, and to the Windows event log in any case (which you can view with Event Viewer). But detailed logging is suppressed in newer builds unless you run with the -V option. > I'm also curious why it hangs up on one file and (seemingly) fails to > continue backing up the rest of the definition. That was a kind of bug (or paranoid behaviour) in older versions, now hopefully fixed. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue Oct 23 12:05:45 2007 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 23 Oct 2007 12:05:45 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting Message-ID: James O'Gorman wrote: > I've proposed that I could host it all under a FreeBSD Jail > (similar to > a Solaris Zone, for those who are familiar) so that it could all be > easily be manageable, and I could give the key developers admin access > to this jail so that it's not just me or Ben being relied on. > > So, comments, anyone? Sounds good to me, although it will reduce the lovely google rating I have on fluffy.co.uk. :-( I'll work out the details with James offlist. Ben From boxbackup at fluffy.co.uk Tue Oct 23 20:07:29 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Tue, 23 Oct 2007 20:07:29 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: References: Message-ID: <20071023190728.GD61624@netinertia.co.uk> On Tue, Oct 23, 2007 at 12:05:45PM +0100, Ben Summers wrote: > Sounds good to me, although it will reduce the lovely google rating I have > on fluffy.co.uk. :-( Righto, I'm getting there! Ben has put the website into svn, so this is now checked out on the new server and the website is running happily. www.boxbackup.org (and .com) should now be up and working (seems to be working from my end). This is currently the same as www.fluffy.co.uk/boxbackup but now it's in svn anyone can modify it. Some more decisions need to be made now (this is mainly for the developers): How do we want to work the svn and trac sites? I can either create subdomains (e.g. svn.boxbackup.org / trac.boxbackup.org) or keep it similar to the current setup (have /svn and /trac as aliases on the www site). My inclination is to go with the subdomain option, but am easy either way. I won't move the repository just yet so everything will be exactly the same for the time being. James From boxbackup at fluffy.co.uk Wed Oct 24 08:13:41 2007 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Wed, 24 Oct 2007 08:13:41 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <20071023190728.GD61624@netinertia.co.uk> References: <20071023190728.GD61624@netinertia.co.uk> Message-ID: <471EF0A5.9080006@hickinbottom.com> I'm happy with subdomains, too. Presumably www.boxbackup.org and boxbackup.org will be equivalent and also get you to the Trac site without needing the explicit Trac subdomain to be specified? Stuart James O'Gorman wrote: > On Tue, Oct 23, 2007 at 12:05:45PM +0100, Ben Summers wrote: > >> Sounds good to me, although it will reduce the lovely google rating I have >> on fluffy.co.uk. :-( >> > > Righto, I'm getting there! > > Ben has put the website into svn, so this is now checked out on the new > server and the website is running happily. > > www.boxbackup.org (and .com) should now be up and working (seems to be > working from my end). This is currently the same as > www.fluffy.co.uk/boxbackup but now it's in svn anyone can modify it. > > Some more decisions need to be made now (this is mainly for the > developers): > > How do we want to work the svn and trac sites? I can either create > subdomains (e.g. svn.boxbackup.org / trac.boxbackup.org) or keep it > similar to the current setup (have /svn and /trac as aliases on the www > site). > > My inclination is to go with the subdomain option, but am easy either > way. > > I won't move the repository just yet so everything will be exactly the > same for the time being. > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > From boxbackup at fluffy.co.uk Wed Oct 24 08:50:56 2007 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Wed, 24 Oct 2007 08:50:56 +0100 Subject: [Box Backup] RFC: Migration of domain and hosting In-Reply-To: <471EF0A5.9080006@hickinbottom.com> References: <20071023190728.GD61624@netinertia.co.uk> <471EF0A5.9080006@hickinbottom.com> Message-ID: <20071024075056.GH61624@netinertia.co.uk> On Wed, Oct 24, 2007 at 08:13:41AM +0100, Stuart Hickinbottom wrote: > I'm happy with subdomains, too. Presumably www.boxbackup.org and > boxbackup.org will be equivalent and also get you to the Trac site > without needing the explicit Trac subdomain to be specified? As I said, www.boxbackup.org is currently the original website. If people want to modify it, they can as it's now in subversion. I'm not sure about the merits of having the main website as the Trac site though? James From boxbackup at fluffy.co.uk Mon Oct 29 14:59:03 2007 From: boxbackup at fluffy.co.uk (Kenny Millington) Date: Mon, 29 Oct 2007 14:59:03 +0000 Subject: [Box Backup] BoxBackup Reporting Message-ID: <1193669943.17919.29.camel@helios> Hi, I'm new to the list so first of all I'd like to say Hi to everyone. :-) I've written a (python) script based on the perl script Matt Brown posted to the list sometime ago for producing (e-mail) reports from BoxBackup logs. Alot of credit for this work goes to Matt for both the original idea and for providing a reasonable amount of testing of this script. The script (bbreporter.py) is being released under the GNU GPL v3 (or above) and is available from the following link:- http://www.3ait.co.uk/misc/bbreporter.py It's been tested on both Debian Sarge and Ubuntu Dapper. I don't believe it requires any python libraries not included by default. To use this script you need to make use of the BoxBackup config option "LogAllFileAccess = yes" and run the daemon with the -v command line switch. Running "bbreporter.py --help" should get you started (once you have log files to parse). The script is reasonably well commented where necessary so it shouldn't be too difficult to see how things work. (Infact I think I waffle on a bit too much about log rotation in the comments but I decided to leave it in. :-)) As far as log rotation goes... I didn't go for any of the options in the script comments, I elected to do the local6.* -/var/log/box syslog rule and add a local6.none to the /var/log/syslog line so that the boxbackup log messages don't clutter the syslog (especially now LogAllFileAccess is on). Right that's it, I hope it works for everyone and people find it useful. Patches are ofcourse welcome. If anyone has any questions I'll certainly do my best to answer them. -- Kenny Millington Systems Developer kenny.millington at 3ait.co.uk 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ CoReg: 3866698 VATReg: 771388600 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. From boxbackup at boxbackup.org Tue Oct 2 21:33:23 2007 From: boxbackup at boxbackup.org (Lesinigo Luca) Date: Tue, 2 Oct 2007 22:33:23 +0200 Subject: [Box Backup] Feedback and issues with boxbackup-0.10 on OpenSolaris/ZFS Message-ID: --Apple-Mail-5-449122644 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello there. Hope this is the right place for this stuff. I'm used to run boxbackup-0.10 in linux, but I'm moving it on a solaris box. Solaris express community edition, svn_67, on good old 32bit x86. 1) I more or less followed http://bbdev.fluffy.co.uk/trac/wiki/ CompilingOnSolaris and it compiled well, but "make install-backup- server" did not work. You can read the exact commands I used at http://www.semlug.net/moin/OpenSolaris/BoxBackup - sorry for the Italian comments, but the commands should be pretty clear. "make install-backup-server" gives me [1], repeated once for every binary in the server parcel. 2) I manually installed boxbackup server on the system (as you can read on the semlug.net wiki) and it seems to be working without problems on my SXCE install. It's happily running inside a solaris container living on a ZFS filesystem, hope this feedback sounds good to you as it does to me :) 2b) zfs with compression=on (uses lzjb, a really light algorithm, much faster than gzip) reports a 1.07 compression factor on a 50GB boxbackup store. Maybe this means you can squeeze something more off the bbackup compression? 3) I kept getting errors like "while housekeeping account 00040001, exception Common OSFileError (Error accessing a file. Check permissions.) (1/9) -- aborting" [from bbstored] and "Exception: RaidFile RaidFileDoesntExist (2/11)" [from bbstoreaccounts info], when the real problem was that I put the 'account directories' directly under the raidfile directory (like $dir/$accountsdir), while bbstored expected $dir/backup/$accountsdir. Mine was just a stupid error transferring the data from a server to another, but the error messages weren't useful at all, they were misleading! 4) This is more of a feature request, but having a solaris smf(5) manifest for boxbackup would be great. I tried to look into it but it proved to be a more challenging task than what I expected, having never wrote a manifest. I must say I'm more or less a beginner with solaris. I hope this mail is useful to you. I'll keep my subscription to the m.l. for a while so I'll read any reply to this. If boxbackup-0.10 on solaris proves to be stable as it seems to be, I'll convert my production servers (I offer an internet boxbackup server service to my customers) to solaris on ZFS. Meanwhile, thanks for boxbackup and keep up with the good work! :) [1] partial output of make install-backup-server: (cd parcels/boxbackup-0.10-backup-server-solaris2.11; ./install- backup-server) find: stat() error /opt/boxbackup/bin: No such file or directory find: cycle detected for /lib/32/ find: cycle detected for /lib/secure/32/ find: cycle detected for /usr/lib/locale/common/32/ find: cycle detected for /usr/lib/locale/common/LC_CTYPE/32/ find: cycle detected for /usr/lib/locale/common/LO_LTYPE/32/ find: cycle detected for /usr/lib/secure/32/ find: cycle detected for /usr/lib/32/ find: cycle detected for /usr/lib/lwp/32/ find: cycle detected for /usr/lib/link_audit/32/ find: cannot follow symbolic link /usr/lib/lu/lu_activate_sync: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lu_init_sync_start: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lu_init_sync_stop: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lucreatezone: No such file or directory find: cannot follow symbolic link /usr/lib/lu/ludevices: No such file or directory find: cannot follow symbolic link /usr/lib/lu/ludo: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lufslist: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lumount: No such file or directory find: cannot follow symbolic link /usr/lib/lu/luprintf: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lurootdev: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lustatus: No such file or directory find: cannot follow symbolic link /usr/lib/lu/lusync: No such file or directory find: cannot follow symbolic link /usr/lib/lu/luumount: No such file or directory [....................] -- Lesinigo Luca LM Networks S.r.l. --Apple-Mail-5-449122644 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello there. Hope this is the right place for this stuff.=A0I'm used to = run boxbackup-0.10 in linux, but I'm moving it on a solaris box. Solaris = express community edition, svn_67, on good old 32bit = x86.

1) I more or less followed=A0http://bbd= ev.fluffy.co.uk/trac/wiki/CompilingOnSolaris and it compiled well, = but "make install-backup-server" did not work.=A0You can read the exact = commands I used at=A0http://www.semlu= g.net/moin/OpenSolaris/BoxBackup - sorry for the Italian comments, = but the commands should be pretty clear. "make install-backup-server" = gives me [1], repeated once for every binary in the server = parcel.

2) I = manually installed boxbackup server on the system (as you can read on = the semlug.net wiki) and it seems to be working without problems on my = SXCE install. It's happily running inside a solaris container living on = a ZFS filesystem, hope this feedback sounds good to you as it does to me = :)

2b) zfs = with compression=3Don (uses lzjb, a really light algorithm, much faster = than gzip) reports a 1.07 compression factor on a 50GB boxbackup store. = Maybe this means you can squeeze something more off the bbackup = compression?

3) I kept getting errors = like "while housekeeping account 00040001, exception Common OSFileError = (Error accessing a file. Check permissions.) (1/9) -- aborting" [from = bbstored] and "Exception: RaidFile RaidFileDoesntExist (2/11)" [from = bbstoreaccounts info], when the real problem was that I put the 'account = directories' directly under the raidfile directory (like = $dir/$accountsdir), while bbstored expected = $dir/backup/$accountsdir.
Mine was just a stupid error = transferring the data from a server to another, but the error messages = weren't useful at all, they were misleading!

4) This is more of a = feature request, but having a solaris smf(5) manifest for boxbackup = would be great. I tried to look into it but it proved to be a more = challenging task than what I expected, having never wrote a manifest. I = must say I'm more or less a beginner with = solaris.

I hope this mail is useful to you. = I'll keep my subscription to the m.l. for a while so I'll read any reply = to this.
If boxbackup-0.10 on solaris proves to be stable as = it seems to be, I'll convert my production servers (I offer an internet = boxbackup server service to my customers) to solaris on = ZFS.

Meanwhile, thanks for = boxbackup and keep up with the good work! :)

[1] partial output of make = install-backup-server:
(cd = parcels/boxbackup-0.10-backup-server-solaris2.11; = ./install-backup-server)
find: stat() error = /opt/boxbackup/bin: No such file or directory
find: cycle = detected for /lib/32/
find: cycle detected for = /lib/secure/32/
find: cycle detected for = /usr/lib/locale/common/32/
find: cycle detected for = /usr/lib/locale/common/LC_CTYPE/32/
find: cycle detected for = /usr/lib/locale/common/LO_LTYPE/32/
find: cycle detected for = /usr/lib/secure/32/
find: cycle detected for = /usr/lib/32/
find: cycle detected for = /usr/lib/lwp/32/
find: cycle detected for = /usr/lib/link_audit/32/
find: cannot follow symbolic link = /usr/lib/lu/lu_activate_sync: No such file or directory
find: = cannot follow symbolic link /usr/lib/lu/lu_init_sync_start: No such file = or directory
find: cannot follow symbolic link = /usr/lib/lu/lu_init_sync_stop: No such file or directory
find: = cannot follow symbolic link /usr/lib/lu/lucreatezone: No such file or = directory
find: cannot follow symbolic link = /usr/lib/lu/ludevices: No such file or directory
find: cannot = follow symbolic link /usr/lib/lu/ludo: No such file or = directory
find: cannot follow symbolic link = /usr/lib/lu/lufslist: No such file or directory
find: cannot = follow symbolic link /usr/lib/lu/lumount: No such file or = directory
find: cannot follow symbolic link = /usr/lib/lu/luprintf: No such file or directory
find: cannot = follow symbolic link /usr/lib/lu/lurootdev: No such file or = directory
find: cannot follow symbolic link = /usr/lib/lu/lustatus: No such file or directory
find: cannot = follow symbolic link /usr/lib/lu/lusync: No such file or = directory
find: cannot follow symbolic link = /usr/lib/lu/luumount: No such file or = directory
[....................]

On 10/3/07, Chris Wilson <chris at qwirx.com> wrote:
Hi Pete and Per,

On Tue, 2 Oct 2007, Per Reedtz Thomsen wrote:

> On 10/2/07 7:06 PM, Peter Jalajas, TebucoSafe Backups wrote:
>>
>>  A new situation for me.  A new client is in a Mac environment (info from
>>  the client):
...
>>  I'm not a Mac guy at all, so I'm not sure I even know where to begin.  I
>>  guess I have to set up a
>>  build environment (yikes!  how?!), and then follow:
>>  http://bbdev.fluffy.co.uk/trac/wiki/SourceCodeRepository ?
>>
>>  Any and all help appreciated!
>>
> I'll be happy to send you the G5 10.4.10 parcel for client and server,
> but I don't have a G4 machine anymore. As far as a development
> environment goes, Apple's XCode Dev Kit is free to download if you sign
> up at Apple's developer site ( http://www.apple.com/developer).  The ADC
> Online membership is free.

Pete, does your client have Fink installed, or would they mind installing
it?

Per, do you know much about Fink? Would you be interested in helping to
get Box into Fink's repository? I don't have a Mac, so I'd be flying
blind, but I'd like to get it done.

Cheers, Chris.
--
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |
_______________________________________________
boxbackup mailing list
boxbackup at fluffy.co.uk
http://lists.warhead.org.uk/mailman/listinfo/boxbackup

------=_Part_22665_33205174.1191412519677--