From boxbackup at fluffy.co.uk Mon May 1 00:26:58 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Mon, 01 May 2006 01:26:58 +0200 Subject: [Box Backup] How to fix compare errors??? more info In-Reply-To: References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445528FF.50507@homann.se> Message-ID: <445547C2.7090908@homann.se> Chris Wilson wrote: > Hi Magnus, > >> This is what I got: >> homann at crux:~$ md5sum bigfile* >> f1c9645dbc14efddc7d8a322685f26eb bigfile >> f1c9645dbc14efddc7d8a322685f26eb bigfile.sparse >> homann at crux:~$ > > Is this on the same partition where you have gps.pdf? No, it wasn't actually. But I tried it on that disk too, and got the same result. >Can you try to > identify how gps.pdf differs from its copy? > od -Ax -tx4 gps.pdf > gps.pdf.od > od -Ax -tx4 gps-back2.pdf > gps.pdf.od.2 > diff -u gps.pdf.od gps.pdf.od.2 Hmm.... I took two od:s from the same gps.pdf. I then did a diff, which gave different results (see below for exact copy of my console). Not good at all. homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ od -Ax -tx4 gps.pdf > gps.pdf.od.1 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ od -Ax -tx4 gps.pdf > gps.pdf.od.2 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ diff -u gps.pdf.od.1 gps.pdf.od.2 --- gps.pdf.od.1 2006-05-01 01:18:15.000000000 +0200 +++ gps.pdf.od.2 2006-05-01 01:18:21.000000000 +0200 @@ -1189680,7 +1189680,7 @@ 1229160 c29e5512 5f60c1e8 1f45af34 78df8588 1229170 479aeaa9 dcbdb1fd cebb196f b5ca0335 1229180 1b24b0de 09273a09 9b6fb3c9 4ce19601 -1229190 7da8d2ad b9166fc6 4694ff62 789e2647 +1229190 7da8d2ad b9166fc6 4694ff62 789u2647 12291a0 e78e3552 9e15d79d 003e8e2c 35d3c0cd 12291b0 54b22f25 854b0678 ca639db4 c18b2e78 12291c0 eaf74679 a7bf37ba 2f92393a 36d6899e homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ diff -u gps.pdf.od.1 gps.pdf.od.2 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ diff -u gps.pdf.od.1 gps.pdf.od.2 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ diff -u gps.pdf.od.1 gps.pdf.od.2 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ diff -u gps.pdf.od.1 gps.pdf.od.2 --- gps.pdf.od.1 2006-05-01 01:18:15.000000000 +0200 +++ gps.pdf.od.2 2006-05-01 01:18:21.000000000 +0200 @@ -1262757,7 +1262757,7 @@ 13468b0 bfc1c1bf c1bdc2c1 c1c1bfc1 c3c1c1bf 13468c0 c2c3c1c1 c0c2c3c0 c5c2c4c3 c5bebdbf 13468d0 d9ddc2c1 d9dce0d6 88cfd6d9 b4b87d83 -13468e0 a9r5bba9 b9a9b5b9 b6b6b0b4 a9b7b6ae +13468e0 a9b5bba9 b9a9b5b9 b6b6b0b4 a9b7b6ae 13468f0 b8a5b8b5 b5b9a9b6 b6b3bcac beafb1b9 1346900 b6b4aab8 a9bcb3a0 58edecde 00009165 1346910 e6dbd72c dbd4dad9 dddcd7dc d6dbdad8 homann at crux:/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf$ So, three out of 5 attempts of running diff found no errors, but the other two did... >> This is .10, built by myself. All tests passed. > > Are you sure you didn't fiddle with the output after pasting it into the > email? You mean with the double path component? Completely sure. It appears I have issues with my files, which isn't really boxbackups fault. Please note that I currenlty have both the original file and the backup on the same partition, so either could be corrupt it seems. Any hints for debugging this error? :-) From boxbackup at fluffy.co.uk Mon May 1 00:39:09 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 30 Apr 2006 23:39:09 +0000 (GMT) Subject: [Box Backup] How to fix compare errors??? more info In-Reply-To: <445547C2.7090908@homann.se> References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445528FF.50507@homann.se> <445547C2.7090908@homann.se> Message-ID: Hi Magnus, On Mon, 1 May 2006, Magnus Homann wrote: > -1229190 7da8d2ad b9166fc6 4694ff62 789e2647 > +1229190 7da8d2ad b9166fc6 4694ff62 789u2647 "u" is not even a hex digit. But "e" and "u" differ by one bit, 0x10. I suspect you have faulty memory, or your system is overclocked. Try running memtest on your machine overnight and see what happens. > -13468e0 a9r5bba9 b9a9b5b9 b6b6b0b4 a9b7b6ae > +13468e0 a9b5bba9 b9a9b5b9 b6b6b0b4 a9b7b6ae Again, "r" is not a hex digit, but differs from "b" by 0x10. Until you can reliably create two ODs of the same file with no differences between them, please don't expect Box Backup to work reliably on your system :-) Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon May 1 08:16:43 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Mon, 01 May 2006 09:16:43 +0200 Subject: [Box Backup] How to fix compare errors??? more info In-Reply-To: References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445528FF.50507@homann.se> <445547C2.7090908@homann.se> Message-ID: <4455B5DB.1020700@homann.se> > Until you can reliably create two ODs of the same file with no > differences between them, please don't expect Box Backup to work > reliably on your system :-) Of course not, but a way to forcibly re-upload a without having to 'touch' it is valuable. Preferrable I would like a way to tell bbackupd to check the files the same way that compare -aq does, and upload the files that differ. Currently, if the first storing of the file in the backup fails in a way that is undetected, box does not seem to re-check and re-try. Not even by manual intervention. From boxbackup at fluffy.co.uk Mon May 1 21:15:00 2006 From: boxbackup at fluffy.co.uk (Wendell MacKenzie) Date: Mon, 01 May 2006 16:15:00 -0400 Subject: [Box Backup] config problem Message-ID: <44566C44.2060000@sympatico.ca> Hi: Just got my smarmy box-backup setup installed and having problems getting the client to talk to the server. I've setup my server, created the certificate store, created my backup account, signed my server certificate, etc. The bbstored launches fine. Then I created my client ID, signed it and am now attempting to launch the bbackupd client backup utility. It's failing with the following: May 1 16:08:30 macduff bbackupd[25097]: Starting daemon (config: /etc/box/bbackupd.conf) (version 0.10) May 1 16:08:31 macduff bbackupd[25097]: Beginning scan of local files May 1 16:08:31 macduff bbackupd[25097]: Opening connection to server macduff... May 1 16:08:31 macduff bbstored[24902]: Incoming connection from 10.61.16.126 port 62225 (handling in child 25098) May 1 16:08:31 macduff bbstored[25098]: SSL err during Accept: error:1409441B:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error May 1 16:08:31 macduff bbackupd[25097]: SSL err during Connect: error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01 May 1 16:08:31 macduff bbackupd[25097]: SSL err during Connect: error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed May 1 16:08:31 macduff bbackupd[25097]: SSL err during Connect: error:0D089006:asn1 encoding routines:ASN1_verify:EVP lib May 1 16:08:31 macduff bbackupd[25097]: SSL err during Connect: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed May 1 16:08:31 macduff bbackupd[25097]: Exception caught (Connection TLSHandshakeFailed 7/30), reset state and waiting to retry... May 1 16:08:31 macduff bbstored[25098]: in server child, exception Connection TLSHandshakeFailed (7/30) -- terminating child May 1 16:08:41 macduff bbackupd[25097]: File statistics: total file size uploaded 0, bytes already on server 0, encoded size 0 This is the relevant key area in my bbackup.conf file: StoreHostname = macduff AccountNumber = 0x388 KeysFile = /etc/box/bbackupd/388-FileEncKeys.raw CertificateFile = /etc/box/bbstored/ca/clients/388-cert.pem PrivateKeyFile = /etc/box/bbackupd/388-key.pem TrustedCAsFile = /etc/box/bbstored/ca/roots/serverCA.pem Platform is SUSE 10.0 on Intel Pentium. Any ideas?? What'd I do wrong? Thx. Wendell From boxbackup at fluffy.co.uk Mon May 1 22:45:09 2006 From: boxbackup at fluffy.co.uk (Wendell MacKenzie) Date: Mon, 01 May 2006 17:45:09 -0400 Subject: [Box Backup] config question Message-ID: <44568165.1000201@sympatico.ca> Hi: Managed to straighten out my key issue from previous posting. Now the bbstored daemon and the bbackupd client are running and talking together harmoniously. I've setup my config in a snapshot fashion to enable "on-demand" backup of my filesystems. When I instruct the backup to begin, the bbackupd daemon is logging a shedload of messages into /var/log/message about files and directories that are being backed up (presumably).... question is: where to? My /etc/box/bbackup.conf has the parm: DataDirectory = /mnt/backup/wendell specified. Does this not mean to backup the files/directories to this as a target directory? If so, why am I not seeing the target structure being assembled. I let it run for 15 mins or so... Am I missing something basic here? thx. Wendell From boxbackup at fluffy.co.uk Mon May 1 23:05:47 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 1 May 2006 22:05:47 +0000 (GMT) Subject: [Box Backup] config question In-Reply-To: <44568165.1000201@sympatico.ca> References: <44568165.1000201@sympatico.ca> Message-ID: Hi Wendell, > Managed to straighten out my key issue from previous posting. What was causing it? Those key errors were weird, I've never seen anything like that. > When I instruct the backup to begin, the bbackupd daemon is logging a > shedload of messages into /var/log/message about files > and directories that are being backed up (presumably).... > > question is: where to? My /etc/box/bbackup.conf has the parm: > > DataDirectory = /mnt/backup/wendell > > specified. Does this not mean to backup the files/directories to > this as a target directory? If so, why am I not seeing the target > structure being assembled. I let it run for 15 mins or so... No, the data directory is for client state. The location where your backups are stored is controlled by bbstored, and specified in raidfile.conf. The names are encrypted, so you won't recognise your directory and file structure in there, but the amount of data should be similar, and should match what "bbstoreaccounts" tells you. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 2 01:39:18 2006 From: boxbackup at fluffy.co.uk (Wendell MacKenzie) Date: Mon, 01 May 2006 20:39:18 -0400 Subject: [Box Backup] config question In-Reply-To: References: <44568165.1000201@sympatico.ca> Message-ID: <4456AA36.9040509@sympatico.ca> Chris Wilson wrote: > Hi Wendell, > >> Managed to straighten out my key issue from previous posting. > > What was causing it? Those key errors were weird, I've never seen > anything like that. WML (wendell mackenzie logic :) I didn't follow the instructions 2 the T...I guess. > >> When I instruct the backup to begin, the bbackupd daemon is logging a >> shedload of messages into /var/log/message about files >> and directories that are being backed up (presumably).... >> >> question is: where to? My /etc/box/bbackup.conf has the parm: >> >> DataDirectory = /mnt/backup/wendell >> >> specified. Does this not mean to backup the files/directories to >> this as a target directory? If so, why am I not seeing the target >> structure being assembled. I let it run for 15 mins or so... > > No, the data directory is for client state. The location where your > backups are stored is controlled by bbstored, and specified in > raidfile.conf. The names are encrypted, so you won't recognise your > directory and file structure in there, but the amount of data should > be similar, and should match what "bbstoreaccounts" tells you. got it. Built boxi tonight...any plans for rpm'ing it? > > Cheers, Chris. From boxbackup at fluffy.co.uk Tue May 2 10:05:04 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 2 May 2006 09:05:04 +0000 (GMT) Subject: [Box Backup] config question In-Reply-To: <4456AA36.9040509@sympatico.ca> References: <44568165.1000201@sympatico.ca> <4456AA36.9040509@sympatico.ca> Message-ID: Hi Wendell, > got it. Built boxi tonight...any plans for rpm'ing it? Yes, definitely, but I'm nearing the end of a complete rewrite at the moment, and I want to get that out of the way first. I'll address packaging before the next release, which I hope will be a 1.0 alpha version. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 2 23:03:22 2006 From: boxbackup at fluffy.co.uk (Alex Howansky) Date: Tue, 2 May 2006 17:03:22 -0500 (CDT) Subject: [Box Backup] windows service Message-ID: I'm running Chris' latest Windows binary, but I can't get it to start as a service. Running bbackupd.exe -i returns: Failed to create Box Backup service: error 1078 There is a "Box Backup" entry in the services manager list, presumably from the previous 0.09 version I was running. It's pointing to the right exe, but when I click the start button, I get: Could not start the Box Backup service on Local Computer. Error 1053: The service did not respond to the start or control request in a timely fashion. If I run bbackupd.exe from the command prompt and let it just sit there in the foreground and do its thing, it works fine. Any ideas? -- Alex Howansky Wankwood Associates http://wankwood.com/ From boxbackup at fluffy.co.uk Tue May 2 23:12:06 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 2 May 2006 22:12:06 +0000 (GMT) Subject: [Box Backup] windows service In-Reply-To: References: Message-ID: Hi Alex, > I'm running Chris' latest Windows binary, but I can't get it to start as a > service. Running bbackupd.exe -i returns: > > Failed to create Box Backup service: error 1078 Error 1078 means "duplicate service name". You already have a service registered with the name "Box Backup". Please delete it with the "-r" option before installing a new one. > Could not start the Box Backup service on Local Computer. > Error 1053: The service did not respond to the start or control request in a > timely fashion. The command-line parameters have probably changed since that version. Please remove the old one and install the new one, and see if that helps. Hope this helps. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 3 09:07:40 2006 From: boxbackup at fluffy.co.uk (zzbug@hjollum.com) Date: Wed, 03 May 2006 10:07:40 +0200 Subject: [Box Backup] Latest "safe" win32 box backup client Message-ID: <445864CC.1050406@hjollum.com> Hi I am currently running the windows box backup client v. 0.09f. Is this the latest "safe" client version? If not, which should I be using, and where do I obatin it? I dont have any experience in compiling from source code, so I would prefer a compiled version. best regards and thanks in advance, Jari From boxbackup at fluffy.co.uk Wed May 3 09:32:01 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 3 May 2006 08:32:01 +0000 (GMT) Subject: [Box Backup] Latest "safe" win32 box backup client In-Reply-To: <445864CC.1050406@hjollum.com> References: <445864CC.1050406@hjollum.com> Message-ID: Hi Jari, > I am currently running the windows box backup client v. 0.09f. Is this > the latest "safe" client version? I would consider the new 0.10 versions of the client, where the Windows code has been merged into the main tree, to be at least as safe if not safer. However, you will have to upgrade the server to 0.10 as well to use them. If you try the new client, please let us know if you have any problems. > If not, which should I be using, and where do I obatin it? I dont have > any experience in compiling from source code, so I would prefer a > compiled version. You can find the new client here: http://boxbackup.hostworks.ca/index.php/0.10#Windows_native_client_status Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 2 12:20:39 2006 From: boxbackup at fluffy.co.uk (Bjorn Andersson) Date: Tue, 2 May 2006 13:20:39 +0200 Subject: [Box Backup] Bad file descriptor Message-ID: <20060502132039.17acebe3@emotpin.laeskback.com> Hello I'm running boxbackup v0.10 on my server with hw-raid (not using built in raid in bbackup). Box is OpenBSD 3.8, normal p4 with a megaraid (500gb partition). Now and then something happends to some of my accounts. The daily insecurity report contans these: find: /home/backup/backup/00001001/info.rfw: Bad file descriptor and of course the logs 'exception BackupStore CouldNotLoadStoreInfo (4/12)'. With one exception it is always info.rfw (on different accounts each time). And most of the time it seams to be 2 accounts at a time. 'bbstoreaccount check 1001 fix' fixes the problem, but i dont find it to be a very good solution. A fsck of the disc does find the errors and fix them, but then i have to run check fix anyway to rebuild the info.rfw. All accounts is way under their limits. Is this anything you've ever seen before? Should i blame ffs, or maybe ami(4)? Regards // Bjorn From boxbackup at fluffy.co.uk Wed May 3 20:16:05 2006 From: boxbackup at fluffy.co.uk (Dave Bamford) Date: Wed, 03 May 2006 20:16:05 +0100 Subject: [Box Backup] windows service In-Reply-To: References: Message-ID: <44590175.80405@logical-progress.com> Chris Wilson wrote: > Hi Alex, > >> I'm running Chris' latest Windows binary, but I can't get it to start >> as a >> service. Running bbackupd.exe -i returns: >> >> Failed to create Box Backup service: error 1078 > > > Error 1078 means "duplicate service name". You already have a service > registered with the name "Box Backup". Please delete it with the "-r" > option before installing a new one. > Ah.. I wish I knew about the "-r" option today. I had the same problem and had to delete the service by removing the registry keys and rebooting. Don't you just love windows... Dave Bamford. From boxbackup at fluffy.co.uk Wed May 3 21:37:54 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Wed, 03 May 2006 22:37:54 +0200 Subject: [Box Backup] Copy a backu store? Message-ID: <445914A2.2050005@homann.se> I have used a local store (same disk) during testing. Is it possible to just copy this store to remote backup store and continue running from that computer? Is it more efficient than doing a new store remotely? rsync? scp? other? Magnus From boxbackup at fluffy.co.uk Wed May 3 21:30:14 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 3 May 2006 21:30:14 +0100 Subject: [Box Backup] Copy a backup store? In-Reply-To: <445914A2.2050005@homann.se> References: <445914A2.2050005@homann.se> Message-ID: <60B5BA92-7871-48B9-B666-31AF40585310@fluffy.co.uk> On 3 May 2006, at 21:37, Magnus Homann wrote: > I have used a local store (same disk) during testing. Is it > possible to just copy this store to remote backup store and > continue running from that computer? Yes, if you keep the account number. > Is it more efficient than doing a new store remotely? Yes (as long as it's not using userland raid) > > rsync? scp? other? Whatever you normally use to transfer files. Ben From boxbackup at fluffy.co.uk Wed May 3 22:04:20 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 3 May 2006 21:04:20 +0000 (GMT) Subject: [Box Backup] windows service In-Reply-To: <44590175.80405@logical-progress.com> References: <44590175.80405@logical-progress.com> Message-ID: Hi Dave, > I wish I knew about the "-r" option today. I had the same problem and > had to delete the service by removing the registry keys and rebooting. > Don't you just love windows... Thanks for pointing that out. I've added a note to the relevant wiki page, where the -i option is described. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 4 18:37:31 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 04 May 2006 19:37:31 +0200 Subject: [Box Backup] How to fix compare errors??? normal? In-Reply-To: References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> Message-ID: <445A3BDB.3070904@homann.se> Chris Wilson wrote: >> Local file >> '/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf/gps.pdf' >> has different contents to store file >> '/persistent-homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf'. > > This is also weird. The two paths should be the same (below the location > root), but they're not. What version of Box Backup is this? And did you > build it yourself? So, have anyone seen this double path behaviour? I still get it... From boxbackup at fluffy.co.uk Thu May 4 18:37:47 2006 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Thu, 4 May 2006 19:37:47 +0200 Subject: [Box Backup] How to fix compare errors??? normal? References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445A3BDB.3070904@homann.se> Message-ID: <009201c66fa1$75158440$0101050a@stenorhem> >>> Local file >>> '/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf/gps.pdf' >>> has different contents to store file >>> '/persistent-homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf'. >> >> This is also weird. The two paths should be the same (below the location >> root), but they're not. What version of Box Backup is this? And did you >> build it yourself? > > So, have anyone seen this double path behaviour? I still get it... Yep, I get it as well, for some files... query > compare -aq WARNING: Quick compare used -- file attributes are not checked. Local file '/home/cerberus/parser.log/parser.log' has different contents to store file '/home/cerberus/parser.log'. Local file '/home/danbra/.Xauthority/.Xauthority' has different contents to store file '/home/danbra/.Xauthority'. Stefan From boxbackup at fluffy.co.uk Thu May 4 19:21:17 2006 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Thu, 4 May 2006 20:21:17 +0200 Subject: [Box Backup] How to fix compare errors??? normal? References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445A3BDB.3070904@homann.se> Message-ID: <00d201c66fa7$88cfd020$0101050a@stenorhem> >>> Local file >>> '/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf/gps.pdf' >>> has different contents to store file >>> '/persistent-homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf'. >> >> This is also weird. The two paths should be the same (below the location >> root), but they're not. What version of Box Backup is this? And did you >> build it yourself? > > So, have anyone seen this double path behaviour? I still get it... Just had a look in the source... it would seem it is a "cosmetic bug" in BackupQueries.cpp. printf("Local file '%s" DIRECTORY_SEPARATOR "%s' has different contents " "to store file '%s/%s'.\n", localName.c_str(), i->first.c_str(), storeName.c_str(), i->first.c_str()); localName contains the filename including the full path and then the file name is added an extra time when the warning is displayed. So I would say this issue could safely be ignored until fixed. Regards, Stefan From boxbackup at fluffy.co.uk Thu May 4 20:10:11 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 04 May 2006 21:10:11 +0200 Subject: [Box Backup] How to fix compare errors??? normal? In-Reply-To: <00d201c66fa7$88cfd020$0101050a@stenorhem> References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445A3BDB.3070904@homann.se> <00d201c66fa7$88cfd020$0101050a@stenorhem> Message-ID: <445A5193.4070509@homann.se> Stefan Norlin wrote: >>>> Local file >>>> '/persistent/homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf/gps.pdf' >>>> has different contents to store file >>>> '/persistent-homann/gps-3.1.0w-academic-x86-linux/doc/gps/pdf/gps.pdf'. >>> >>> This is also weird. The two paths should be the same (below the >>> location root), but they're not. What version of Box Backup is this? >>> And did you build it yourself? >> >> So, have anyone seen this double path behaviour? I still get it... > > Just had a look in the source... it would seem it is a "cosmetic bug" > in BackupQueries.cpp. Thanks, Stefan! As I thought, but I was a bit surprised Chris hadn't noticed previously. From boxbackup at fluffy.co.uk Thu May 4 23:27:10 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 4 May 2006 22:27:10 +0000 (GMT) Subject: [Box Backup] How to fix compare errors??? normal? In-Reply-To: <445A5193.4070509@homann.se> References: <44548C66.60609@homann.se> <4454CBDA.3000803@homann.se> <445A3BDB.3070904@homann.se> <00d201c66fa7$88cfd020$0101050a@stenorhem> <445A5193.4070509@homann.se> Message-ID: Hi Magnus, On Thu, 4 May 2006, Magnus Homann wrote: > Stefan Norlin wrote: >> Just had a look in the source... it would seem it is a "cosmetic bug" >> in BackupQueries.cpp. You're absolutely right, thanks for finding it for me. > As I thought, but I was a bit surprised Chris hadn't noticed previously. I did have a look at the code when you reported the problem, suspecting a bug of this kind, but I didn't see the bug because the variable in question is redefined in that scope, with a different meaning. This is bad practice, and probably my fault, from when I was fixing the character encodings on Windows. I've checked in a fix into the SVN trunk. Please could you check it out and see if it fixes the problem with the wrong path being reported? It does not actually fix the underlying problem of store corruption that you've been seeing. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 7 15:03:49 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Sun, 07 May 2006 16:03:49 +0200 Subject: [Box Backup] Running bbstored as a user Message-ID: <445DFE45.3080503@homann.se> I'm trying to set upp bbstored from an ordinary Unix account (Fedora Core 4). I've modified the bbstored.conf and used that file path as a argument to bbstored. But what about loggin? I cannpt modify syslog or make a file in /var/log. Is there anpther way to see what's happening? Printouts on the screen, piped to a file? Thanks From boxbackup at fluffy.co.uk Sun May 7 23:55:07 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 7 May 2006 22:55:07 +0000 (GMT) Subject: [Box Backup] Running bbstored as a user In-Reply-To: <445DFE45.3080503@homann.se> References: <445DFE45.3080503@homann.se> Message-ID: Hi Magnus, > I'm trying to set upp bbstored from an ordinary Unix account (Fedora > Core 4). I've modified the bbstored.conf and used that file path as a > argument to bbstored. That should work, but be aware that if Box Backup hits an unreadable file, it may abort the backup run. It used to, and I think we fixed that problem in 0.10, but I'm not sure. > But what about loggin? I cannpt modify syslog or make a file in > /var/log. Is there anpther way to see what's happening? Printouts on the > screen, piped to a file? Sorry, no other options yet (except syslog). We are planning to write a logging system that would allow log messages to be directed wherever you want, including multiple destinations, but nobody has got around to it yet. You're welcome to volunteer :-) Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon May 8 06:49:47 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Mon, 08 May 2006 07:49:47 +0200 Subject: [Box Backup] Running bbstored as a user In-Reply-To: References: <445DFE45.3080503@homann.se> Message-ID: <445EDBFB.9090306@homann.se> Chris Wilson wrote: > Hi Magnus, > >> I'm trying to set upp bbstored from an ordinary Unix account (Fedora >> Core 4). I've modified the bbstored.conf and used that file path as a >> argument to bbstored. > > That should work, but be aware that if Box Backup hits an unreadable > file, it may abort the backup run. It used to, and I think we fixed that > problem in 0.10, but I'm not sure. Yes, but bbackupd will stil be run as root. >> But what about loggin? I cannpt modify syslog or make a file in >> /var/log. Is there anpther way to see what's happening? Printouts on >> the screen, piped to a file? > > Sorry, no other options yet (except syslog). We are planning to write a > logging system that would allow log messages to be directed wherever you > want, including multiple destinations, but nobody has got around to it > yet. You're welcome to volunteer :-) No the answer I hoped for! :-) From boxbackup at fluffy.co.uk Mon May 8 09:42:46 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Mon, 08 May 2006 10:42:46 +0200 Subject: [Box Backup] No. of volumes in RAID? In-Reply-To: <20060505110001.1685.83183.Mailman@love.warhead.org.uk> References: <20060505110001.1685.83183.Mailman@love.warhead.org.uk> Message-ID: <445F0486.30906@kontrapunkt.com> Hello. Can a RAID consist of more than 3 volumes? Thanks, Tobias Balle-Petersen From boxbackup at fluffy.co.uk Mon May 8 09:52:25 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 8 May 2006 08:52:25 +0000 (GMT) Subject: [Box Backup] Running bbstored as a user In-Reply-To: <445EDBFB.9090306@homann.se> References: <445DFE45.3080503@homann.se> <445EDBFB.9090306@homann.se> Message-ID: Hi Magnus, >> That should work, but be aware that if Box Backup hits an unreadable file, >> it may abort the backup run. It used to, and I think we fixed that problem >> in 0.10, but I'm not sure. > > Yes, but bbackupd will stil be run as root. By default, yes, but you can change that. >> Sorry, no other options yet (except syslog). We are planning to write a >> logging system that would allow log messages to be directed wherever you >> want, including multiple destinations, but nobody has got around to it yet. >> You're welcome to volunteer :-) > > No the answer I hoped for! :-) Sorry, but can't you get the system administrator to set up logging so that you have access to the Box logs? For example, direct local6.info to /var/log/box, and make that file world readable, or readable only by your user account. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon May 8 09:57:49 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 8 May 2006 08:57:49 +0000 (GMT) Subject: [Box Backup] No. of volumes in RAID? In-Reply-To: <445F0486.30906@kontrapunkt.com> References: <20060505110001.1685.83183.Mailman@love.warhead.org.uk> <445F0486.30906@kontrapunkt.com> Message-ID: Hi Tobias, On Mon, 8 May 2006, Tobias Balle-Petersen wrote: > Can a RAID consist of more than 3 volumes? In theory yes, but not in the current implementation of the built-in software raid in Box Backup. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon May 8 14:30:49 2006 From: boxbackup at fluffy.co.uk (Jari Hjoellum) Date: Mon, 08 May 2006 15:30:49 +0200 Subject: [Box Backup] Latest "safe" win32 box backup client Message-ID: <445F4809.10805@hjollum.com> Hi Thanks. Unfortunately the server is running version 0.09. In this case what would be the best version? thanks, Jari From boxbackup at fluffy.co.uk Mon May 8 15:48:00 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 8 May 2006 14:48:00 +0000 (GMT) Subject: [Box Backup] Latest "safe" win32 box backup client In-Reply-To: <445F4809.10805@hjollum.com> References: <445F4809.10805@hjollum.com> Message-ID: Hi Jari, > Thanks. Unfortunately the server is running version 0.09. In this case > what would be the best version? The client you are already using, Nick's version, is the most heavily tested version that still supports the 0.09 protocol. Quite a few of the development snapshots that I released in the run-up to 0.10 will actually work with 0.09 servers, as long as you don't try to use the SSL keepalive feature. However, none of them are heavily tested, and many have known bugs that were discussed on this list. Until you can upgrade the server, I recommend that you stick with Nick's client, the latest version of which I think is 0.09f. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 9 17:44:05 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Tue, 9 May 2006 19:44:05 +0300 Subject: [Box Backup] Client connecting to server problem Message-ID: <948a6d890605090944u79a72fa0jc2b617f58e38a11c@mail.gmail.com> Hello, I just installed/compiled Box Backup Everything went fine without any errors Here is my problem When I run the client I get these errors in the logs: May 9 09:38:07 secure1 bbackupd[21117]: Beginning scan of local files May 9 09:38:07 secure1 bbackupd[21117]: Opening connection to server secure2.serversurgeon.com... May 9 09:38:07 secure1 bbackupd[21117]: Exception caught (Connection SocketConnectError (Probably a network issue between client and server, bad hostname, or server not running.) 7/15), reset state and waiting to retry... I have no firewall, I have no NAT/router Can somebody please tell me on what port the server is running so I can check if its a server problem or a client one? In server conf I have: ListenAddresses =3D inet:domain.com In client conf I have: StoreHostname =3D domain.com Please help, Thanks From boxbackup at fluffy.co.uk Tue May 9 18:07:12 2006 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Tue, 9 May 2006 19:07:12 +0200 Subject: [Box Backup] Client connecting to server problem References: <948a6d890605090944u79a72fa0jc2b617f58e38a11c@mail.gmail.com> Message-ID: <000e01c6738b$03c490e0$0101050a@stenorhem> > Can somebody please tell me on what port the server is running so I > can check if its a server problem or a client one? Port is 2201. Stefan From boxbackup at fluffy.co.uk Wed May 10 10:04:35 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Wed, 10 May 2006 12:04:35 +0300 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100202k2f1bab3bidfed2c2f9bb7a6f7@mail.gmail.com> References: <948a6d890605100202k2f1bab3bidfed2c2f9bb7a6f7@mail.gmail.com> Message-ID: <948a6d890605100204y2a5b7dd3o965da644be640b50@mail.gmail.com> Hello, When trying to do: /usr/local/bin/bbackupquery "compare -a" quit It tells me the following: Local directory '/path/to/dir' does not exist, but store directory '/path-to-dir' does. And this is for every directory of the backup When I look into that directory I see that everything exists and is ok What is wrong then? Please help Thanks From boxbackup at fluffy.co.uk Wed May 10 10:15:30 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 10 May 2006 09:15:30 +0000 (GMT) Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100204y2a5b7dd3o965da644be640b50@mail.gmail.com> References: <948a6d890605100202k2f1bab3bidfed2c2f9bb7a6f7@mail.gmail.com> <948a6d890605100204y2a5b7dd3o965da644be640b50@mail.gmail.com> Message-ID: Hi, > When trying to do: /usr/local/bin/bbackupquery "compare -a" quit > It tells me the following: > Local directory '/path/to/dir' does not exist, but store directory > '/path-to-dir' does. > And this is for every directory of the backup > When I look into that directory I see that everything exists and is ok > What is wrong then?> I don't know, but perhaps I could help you better if you tell us what version of Box Backup you're using, on what platform, and post a copy of your configuration file, the output of "bbackupquery ls /path-to-dir", and the output of "ls /path/to/dir". Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 10 10:26:28 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Wed, 10 May 2006 12:26:28 +0300 Subject: [Box Backup] bbackupquery strange issue Message-ID: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> I am using Box Backup 0.10 Server on Linux CentOS 4.3 and client on Linux Fedora Core 2 Copy of configuration file: StoreHostname =3D secure2.serversurgeon.com AccountNumber =3D 0x75AB23C KeysFile =3D /etc/box/bbackupd/75AB23C-FileEncKeys.raw CertificateFile =3D /etc/box/bbackupd/75AB23C-cert.pem PrivateKeyFile =3D /etc/box/bbackupd/75AB23C-key.pem TrustedCAsFile =3D /etc/box/bbackupd/serverCA.pem DataDirectory =3D /var/bbackupd # This script is run whenever bbackupd encounters a problem which requires # the system administrator to assist: # 1) The store is full, and no more data can be uploaded. # 2) Some files or directories were not readable. # The default script emails the system administrator. NotifyScript =3D /etc/box/bbackupd/NotifySysadmin.sh # A scan of the local discs will be made once an hour (approximately). # To avoid cycles of load on the server, this time is randomly adjusted by a small # percentage as the daemon runs. UpdateStoreInterval =3D 3600 # A file must have been modified at least 6 hours ago before it will be uploaded. MinimumFileAge =3D 21600 # If a file is modified repeated, it won't be uploaded immediately in case it's modified again. # However, it should be uploaded eventually. This is how long we should wait after first noticing # a change. (1 day) MaxUploadWait =3D 86400 # Files above this size (in bytes) are tracked, and if they are renamed they will simply be # renamed on the server, rather than being uploaded again. (64k - 1) FileTrackingSizeThreshold =3D 65535 # The daemon does "changes only" uploads for files above this size (in byte= s). # Files less than it are uploaded whole without this extra processing. DiffingUploadSizeThreshold =3D 8192 # The limit on how much time is spent diffing files. Most files shouldn't take very long, # but if you have really big files you can use this to limit the time spent diffing them. # * Reduce if you are having problems with processor usage. # * Increase if you have large files, and think the upload of changes is too large and want # to spend more time searching for unchanged blocks. MaximumDiffingTime =3D 20 # Uncomment this line to see exactly what the daemon is going when it's connected to the server. # ExtendedLogging =3D yes # Use this to temporarily stop bbackupd from syncronising or connecting to the store. # This specifies a program or script script which is run just before each sync, and ideally # the full path to the interpreter. It will be run as the same user bbackupd is running as, # usually root. # The script prints either "now" or a number to STDOUT (and a terminating newline, no quotes). # If the result was "now", then the sync will happen. If it's a number, then the script will # be asked again in that number of seconds. # For example, you could use this on a laptop to only backup when on a specific network. # SyncAllowScript =3D /path/to/intepreter/or/exe script-name parameters etc # Where the command socket is created in the filesystem. CommandSocket =3D /var/run/bbackupd.sock # Uncomment the StoreObjectInfoFile to enable the experimental archiving # of the daemon's state (including client store marker and configuration) # between backup runs. This saves time and increases efficiency when # bbackupd is frequently stopped and started, since it removes the need # to rescan all directories on the remote server. However, it is new and # not yet heavily tested, so use with caution. # StoreObjectInfoFile =3D /var/bbackupd/bbackupd.state Server { PidFile =3D /var/run/bbackupd.pid } # # BackupLocations specifies which locations on disc should be backed up. Ea= ch # directory is in the format # # { # Path =3D /path/of/directory # (optional exclude directives) # } # # 'name' is derived from the Path by the config script, but should merely b= e # unique. # # The exclude directives are of the form # # [Exclude|AlwaysInclude][File|Dir][|sRegex] =3D regex or full pathna= me # # (The regex suffix is shown as 'sRegex' to make File or Dir plural) # # For example: # # ExcludeDir =3D /home/guest-user # ExcludeFilesRegex =3D *.(mp3|MP3)$ # AlwaysIncludeFile =3D /home/username/veryimportant.mp3 # # This excludes the directory /home/guest-user from the backup along with all mp3 # files, except one MP3 file in particular. # # In general, Exclude excludes a file or directory, unless the directory is # explicitly mentioned in a AlwaysInclude directive. # # If a directive ends in Regex, then it is a regular expression rather than= a # explicit full pathname. See # # man 7 re_format # # for the regex syntax on your platform. # BackupLocations { home-httpd-vhosts-serversurgeon.com { Path =3D /home/httpd/vhosts/serversurgeon.com } } query > ls home-httpd-vhosts-serversurgeon.com 00000003 f----- .bash_history 00000004 f----- var 00000005 f----- .viminfo 00000006 f----- httpdocs.joomla-site.tar.gz 00000007 f----- ppts-2.0alpha3.tar.gz 00000008 f----- perl-CGI-Lite-2.02-1.1.fc2.rf.noarch.rpm 00000009 -d---- httpsdocs 0000180a -d---- error_docs 00001810 -d---- cgi-bin 00001861 -d---- anon_ftp 00001867 -d---- bin 00001880 -d---- conf 00001888 -d---- pd 0000188a -d---- web_users 0000188b -d---- subdomains 00003eef -d---- private 00003ef1 -d---- lib 00003f0d -d---- usr 000048f6 -d---- etc 000048fb -d---- tmp 00004931 -d---- statistics 00004a6d -d---- vault_scripts 00004a70 -d---- account 00007b91 -d---- support 00007c6c -d---- .ssh 00007c6e -d---- httpdocs.new 00007cb6 -d---- httpdocs 00008319 -d---- ezmlm [root at secure1 root]# ls /home/httpd/vhosts/serversurgeon.com/ account error_docs httpdocs.new =20 ppts-2.0alpha3.tar.gz tmp anon_ftp etc httpsdocs =20 private usr bin ezmlm lib =20 statistics var cgi-bin httpdocs pd =20 subdomains vault_scripts conf httpdocs.joomla-site.tar.gz=20 perl-CGI-Lite-2.02-1.1.fc2.rf.noarch.rpm support =20 web_users Please help Thank you From boxbackup at fluffy.co.uk Wed May 10 10:32:56 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 10 May 2006 09:32:56 +0000 (GMT) Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> Message-ID: Hi, On Wed, 10 May 2006, o omida parasita wrote: > Please help > Thank you What user did you run "bbackupquery compare" as? Please could you post the first few lines of the output? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 10 10:40:37 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Wed, 10 May 2006 12:40:37 +0300 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> Message-ID: <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> [root at secure1 root]# sudo -u root /usr/local/bin/bbackupquery "compare -a" = quit Box Backup Query Tool v0.10, (c) Ben Summers and contributors 2003-2006 Using configuration file /etc/box/bbackupd.conf Connecting to store... Handshake with store... Login to store... Login complete. Type "help" for a list of commands. Local file '/home/httpd/vhosts/serversurgeon.com/.bash_history' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/.bash_history' does. Local file '/home/httpd/vhosts/serversurgeon.com/.viminfo' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/.viminfo' does. Local file '/home/httpd/vhosts/serversurgeon.com/httpdocs.joomla-site.tar.g= z' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/httpdocs.joomla-site.tar.gz' does. Local file '/home/httpd/vhosts/serversurgeon.com/perl-CGI-Lite-2.02-1.1.fc2= .rf.noarch.rpm' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/perl-CGI-Lite-2.02-1.1.fc2.rf.noarch.= rpm' does. Local file '/home/httpd/vhosts/serversurgeon.com/ppts-2.0alpha3.tar.gz' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/ppts-2.0alpha3.tar.gz' does. Local file '/home/httpd/vhosts/serversurgeon.com/var' does not exist, but store file '/home-httpd-vhosts-serversurgeon.com/var' does. Local directory '/home/httpd/vhosts/serversurgeon.com/.ssh' does not exist, but store directory '/home-httpd-vhosts-serversurgeon.com/.ssh' does. Local directory '/home/httpd/vhosts/serversurgeon.com/account' does not exist, but store directory '/home-httpd-vhosts-serversurgeon.com/account' does. Local directory '/home/httpd/vhosts/serversurgeon.com/anon_ftp' does not exist, but store directory '/home-httpd-vhosts-serversurgeon.com/anon_ftp' does. From boxbackup at fluffy.co.uk Wed May 10 11:04:31 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 10 May 2006 10:04:31 +0000 (GMT) Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323328-257195519-1147255471=:24435 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Hi, Did you compile Box Backup on the client yourself? Can you easily make changes to the source code? Can you apply patches? Could you try applying this patch to your Box Backup client, recompile and re-run "compare -a"? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | --8323328-257195519-1147255471=:24435 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=compare-debug.patch Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=compare-debug.patch SW5kZXg6IGJpbi9iYmFja3VwcXVlcnkvQmFja3VwUXVlcmllcy5jcHANCj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0NCi0tLSBiaW4vYmJhY2t1cHF1ZXJ5L0Jh Y2t1cFF1ZXJpZXMuY3BwCShyZXZpc2lvbiA1NjYpDQorKysgYmluL2JiYWNr dXBxdWVyeS9CYWNrdXBRdWVyaWVzLmNwcAkod29ya2luZyBjb3B5KQ0KQEAg LTEyOTksMjUgKzEyOTksMzggQEANCiAJCQlpZihTX0lTUkVHKHN0LnN0X21v ZGUpIHx8IFNfSVNMTksoc3Quc3RfbW9kZSkpDQogCQkJewkNCiAJCQkgICAg Ly8gRmlsZSBvciBzeW1ib2xpYyBsaW5rDQorCQkJICAgIHByaW50ZigiJXMg aXMgYSBmaWxlIG9yIGxpbmtcbiIsIGxvY2FsRGlyRW4tPmRfbmFtZSk7DQog CQkJICAgIGxvY2FsRmlsZXMuaW5zZXJ0KHN0ZDo6c3RyaW5nKGxvY2FsRGly RW4tPmRfbmFtZSkpOw0KIAkJCX0NCiAJCQllbHNlIGlmKFNfSVNESVIoc3Qu c3RfbW9kZSkpDQogCQkJew0KIAkJCSAgICAvLyBEaXJlY3RvcnkNCisJCQkg ICAgcHJpbnRmKCIlcyBpcyBhIGRpcmVjdG9yeVxuIiwgbG9jYWxEaXJFbi0+ ZF9uYW1lKTsNCiAJCQkgICAgbG9jYWxEaXJzLmluc2VydChzdGQ6OnN0cmlu Zyhsb2NhbERpckVuLT5kX25hbWUpKTsNCi0JCQl9CQkJDQorCQkJfQ0KKwkJ CWVsc2UNCisJCQl7DQorCQkJICAgIHByaW50ZigiJXMgaXMgdW5rbm93biAo JWQpXG4iLCBsb2NhbERpckVuLT5kX25hbWUsDQorCQkJCXN0LnN0X21vZGUp Ow0KKwkJCX0NCiAjZWxzZQ0KIAkJCS8vIEVudHJ5IC0tIGZpbGUgb3IgZGly Pw0KIAkJCWlmKGxvY2FsRGlyRW4tPmRfdHlwZSA9PSBEVF9SRUcgfHwgbG9j YWxEaXJFbi0+ZF90eXBlID09IERUX0xOSykNCiAJCQl7DQogCQkJCS8vIEZp bGUgb3Igc3ltYm9saWMgbGluaw0KKwkJCQlwcmludGYoIiVzIGlzIGEgZmls ZSBvciBsaW5rXG4iLCBsb2NhbERpckVuLT5kX25hbWUpOw0KIAkJCQlsb2Nh bEZpbGVzLmluc2VydChzdGQ6OnN0cmluZyhsb2NhbERpckVuLT5kX25hbWUp KTsNCiAJCQl9DQogCQkJZWxzZSBpZihsb2NhbERpckVuLT5kX3R5cGUgPT0g RFRfRElSKQ0KIAkJCXsNCiAJCQkJLy8gRGlyZWN0b3J5DQorCQkJCXByaW50 ZigiJXMgaXMgYSBkaXJlY3RvcnlcbiIsIGxvY2FsRGlyRW4tPmRfbmFtZSk7 DQogCQkJCWxvY2FsRGlycy5pbnNlcnQoc3RkOjpzdHJpbmcobG9jYWxEaXJF bi0+ZF9uYW1lKSk7DQogCQkJfQ0KKwkJCWVsc2UNCisJCQl7DQorCQkJCXBy aW50ZigiJXMgaXMgdW5rbm93biAoJWQpXG4iLCBsb2NhbERpckVuLT5kX3R5 cGUpOw0KKwkJCX0NCiAjZW5kaWYNCiAJCX0NCiAJCS8vIENsb3NlIGRpcmVj dG9yeQ0K --8323328-257195519-1147255471=:24435-- From boxbackup at fluffy.co.uk Wed May 10 11:22:37 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Wed, 10 May 2006 13:22:37 +0300 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> Message-ID: <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> I installed the client from the parcel that was compiled on the server box with CentOS 4.3. I am compiling now the client on the FC2 box to see maybe this is causing the issue. Yes I can install patches. What patch do you want me to apply? Where to get it from? Thanks From boxbackup at fluffy.co.uk Wed May 10 11:47:59 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Wed, 10 May 2006 13:47:59 +0300 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: <948a6d890605100347h8df775cmcd0002aceb1d369e@mail.gmail.com> I compiled the client on the FC2 box and ran=20 /usr/local/bin/bbackupquery "compare -a" quit It didn't throw up all the previous not existing dirs and it seems it is working now Thanks a lot for your help From boxbackup at fluffy.co.uk Wed May 10 13:31:25 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 10 May 2006 12:31:25 +0000 (GMT) Subject: [Box Backup] bbackupquery strange issue In-Reply-To: <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: Hi, > I installed the client from the parcel that was compiled on the server > box with CentOS 4.3. That's probably the problem. There are two different ways to get information about a file or directory. Box Backup picks one depending on whether it works or not on the machine where it was compiled. That way might or might not work on another machine. I guess that it does work on CentOS, and not on Fedora, which caused your CentOS build to fail on Fedora. Ben, perhaps if we detect HAVE_VALID_DIRENT_D_TYPE and compile with it enabled, then we should add a check that d_type is correct for the "." and ".." directories, and maybe for a known file as well? Or else just use stat() on all platforms? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 10 13:46:17 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 10 May 2006 13:46:17 +0100 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: On 10 May 2006, at 13:31, Chris Wilson wrote: > Hi, > >> I installed the client from the parcel that was compiled on the >> server >> box with CentOS 4.3. > > That's probably the problem. There are two different ways to get > information about a file or directory. Box Backup picks one > depending on whether it works or not on the machine where it was > compiled. That way might or might not work on another machine. I > guess that it does work on CentOS, and not on Fedora, which caused > your CentOS build to fail on Fedora. > > Ben, perhaps if we detect HAVE_VALID_DIRENT_D_TYPE and compile with > it enabled, then we should add a check that d_type is correct for > the "." and ".." directories, and maybe for a known file as well? Yes if it's going to break silently otherwise. Tracking Linux as a target is a nightmare. > Or else just use stat() on all platforms? Doesn't sound the nicest plan ever. Ben From boxbackup at fluffy.co.uk Wed May 10 14:43:32 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 10 May 2006 13:43:32 +0000 (GMT) Subject: [Box Backup] bbackupquery strange issue In-Reply-To: References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: Hi Ben, On Wed, 10 May 2006, Ben Summers wrote: >> Ben, perhaps if we detect HAVE_VALID_DIRENT_D_TYPE and compile with it >> enabled, then we should add a check that d_type is correct for the "." and >> ".." directories, and maybe for a known file as well? > > Yes if it's going to break silently otherwise. Tracking Linux as a target is > a nightmare. I guess you could say that "tracking *BSD is a nightmare" as well, because the diversity between Linuxes and the diversity between the BSDs (including Darwin) is probably similar :-) >> Or else just use stat() on all platforms? > > Doesn't sound the nicest plan ever. What's wrong with stat()? d_type is not in the POSIX standard so we shouldn't really use it for portable code (IMHO). Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 10 20:05:06 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Wed, 10 May 2006 20:05:06 +0100 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: <1147287906.3262.14.camel@avenin.ebourne.me.uk> On Wed, 2006-05-10 at 12:31 +0000, Chris Wilson wrote: > Hi, > > > I installed the client from the parcel that was compiled on the server > > box with CentOS 4.3. > > That's probably the problem. There are two different ways to get > information about a file or directory. Box Backup picks one depending on > whether it works or not on the machine where it was compiled. That way > might or might not work on another machine. I guess that it does work on > CentOS, and not on Fedora, which caused your CentOS build to fail on > Fedora. There's not many unixen where its safe to build a binary on a more recent version and run it on an older version. Almost always unsupported. Generally works the other way though. CentOS 4.3 is more recent than FC2 so that's the problem here. Just checked on FC5 and d_type is now working. Certainly wasn't a year or two ago in the older FCs. The onward march of progress! Have to say I don't really like the parcels concept. Apart from being totally unusual and non standard it encourages people to copy binaries around between non binary compatible systems, and then be surprised when it doesn't work. To have a good chance of it working right you can either build from source (wonderful stuff!) or use a proper package manager that understands binary compatibility and prevents foot shooting. Unfortunately parcels are neither. Furthermore the test suite should ideally be run on every different platform in use. > Ben, perhaps if we detect HAVE_VALID_DIRENT_D_TYPE and compile with it > enabled, then we should add a check that d_type is correct for the "." and > ".." directories, and maybe for a known file as well? Or else just use > stat() on all platforms? I'm generally of the opinion that we should only support binaries built on the same platform, otherwise we land ourselves in a whole load of trouble. However, in this case it does seem like a reasonable check and it is trivial. You only need to test one entry, doesn't matter what it is, just check that d_type is not DT_UNKNOWN. Cheers, Martin. From boxbackup at fluffy.co.uk Wed May 10 20:07:42 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Wed, 10 May 2006 20:07:42 +0100 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: <1147288062.3262.16.camel@avenin.ebourne.me.uk> On Wed, 2006-05-10 at 13:43 +0000, Chris Wilson wrote: > What's wrong with stat()? Apart from just being slow? That's an extra syscall for every file... > d_type is not in the POSIX standard so we > shouldn't really use it for portable code (IMHO). That's what configure is for. Support the lowest common denominator, but also don't cripple everyone else. Cheers, Martin. From boxbackup at fluffy.co.uk Thu May 11 17:04:47 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 11 May 2006 17:04:47 +0100 Subject: [Box Backup] bbackupquery strange issue In-Reply-To: References: <948a6d890605100226n53fe1eanee0dc264aaece6da@mail.gmail.com> <948a6d890605100240n18b7d649rd068c8ac7cc2bb97@mail.gmail.com> <948a6d890605100322q272f360fr1c729044f1e936a2@mail.gmail.com> Message-ID: <905F9E4A-E213-4986-B6CD-D46B7EAAB6FA@fluffy.co.uk> On 10 May 2006, at 14:43, Chris Wilson wrote: > Hi Ben, > > On Wed, 10 May 2006, Ben Summers wrote: > >>> Ben, perhaps if we detect HAVE_VALID_DIRENT_D_TYPE and compile >>> with it enabled, then we should add a check that d_type is >>> correct for the "." and ".." directories, and maybe for a known >>> file as well? >> >> Yes if it's going to break silently otherwise. Tracking Linux as a >> target is a nightmare. > > I guess you could say that "tracking *BSD is a nightmare" as well, > because the diversity between Linuxes and the diversity between the > BSDs (including Darwin) is probably similar :-) Not quite a fair comparison. Targeting "Linux" is targeting an effectively unknown number of platforms. Targeting BSDs is known number of platforms. The thing about every installation of Linux being a surprise is the thing. > >>> Or else just use stat() on all platforms? >> >> Doesn't sound the nicest plan ever. > > What's wrong with stat()? d_type is not in the POSIX standard so we > shouldn't really use it for portable code (IMHO). Efficiency. The point of configure is to allow us to use the good stuff where we can without having to drop to the lowest denominator. With this argument you'd have us all using select()... Ben From boxbackup at fluffy.co.uk Sat May 13 23:47:27 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sat, 13 May 2006 15:47:27 -0700 (PDT) Subject: [Box Backup] How to move store to new machine Message-ID: <20060513224727.10895.qmail@web60625.mail.yahoo.com> Hi all, I'm back... Sorry to go quiet for so long; just busy. I'm moving my store from an old ubuntu (debian) breezy machine with boxbackup 0.09 to a new ubuntu dapper with boxbackup 0.10 across the room. I've got about 8 clients out there in the wild, mostly boxwin0.09. About 9 GB and 85,000 files in the store. Details below. I'm about to change my router/firewall port 2201 forwarding from the old store machine to the new one... Can you all help me prevent hours of wasted time and bandwidth? The two error messages I'm concerned about are the rsync, for example: rsync: send_files failed to open "/root/.bu/backup//write.lock": Permission denied (13) and the bbstoreaccounts check, for example: Directory ID 496 has wrong size for object 82b Are they fatal? Will they cause a complete repeat backing up my entire 9GB of storage? How can I fix or work around them? I hope my treatise below helps others. I intend to clean it up and add it to the wiki and Per's docs if appropriate. Thanks! Pete -------------- (Please excuse my feeble redaction and the likely typo errors I caused.) I've kind of followed the FAQ: http://boxbackup.hostworks.ca/index.php/FAQ and bits and pieces from elsewhere (I'm too lazy to retrace my steps). On the new store, I installed boxbackup 0.10 essentially following: http://boxbackup.hostworks.ca/index.php/Compiling_and_Installing_on_Debian_GNU/Linux but using http://prdownloads.sourceforge.net/boxbackup/boxbackup-0.10.tgz?download (I flailed a bit doing apt-get installs to make readline work, but I think I recall that installing libreadline5-dev finally made it work.) My bbstored.conf: user at new:~$ cat /etc/box/bbstored.conf RaidFileConf = /etc/box/raidfile.conf #pj 20060406: RaidFileConf points to a single dir /media/bu AccountDatabase = /etc/box/bbstored/accounts.txt # Uncomment this line to see exactly what commands are being received from clients. ExtendedLogging = yes # scan all accounts for files which need deleting every 15 minutes. TimeBetweenHousekeeping = 900 Server { PidFile = /var/run/bbstored.pid User = _bbstored ListenAddresses = inet:0.0.0.0 CertificateFile = /etc/box/bbstored/bu1..com-cert.pem PrivateKeyFile = /etc/box/bbstored/bu1..com-key.pem TrustedCAsFile = /etc/box/bbstored/clientCA.pem } My /etc/box/raidfile.conf: disc0 { SetNumber = 0 BlockSize = 1024 Dir0 = /media/bu Dir1 = /media/bu Dir2 = /media/bu } (Note that I think the docs and mail archives are unclear as to doing the 3 Dir[0-2] lines like that when NOT using boxbackup's raid. Can someone confirm? It seems to work, so I think it's correct.) (I also couldn't find anything useful about how to choose a good block size. Would someone please comment extensively on that?) /media/bu is mounted on /dev/md0 which is 2 250GB harddrives raid 1 (mirrored): df -h /media/bu/ | grep md /dev/md0 230G 13G 206G 6% /media/raid cat /etc/mdadm.conf ARRAY /dev/md0 level=raid1 num-devices=2 I stopped any possible running bbstored with: sudo killall bbstored and did a: ps -ef | grep bbstored to make sure. I copied the accounts.txt and keys from the old /etc/box/bbstored/ to the new one using on the old store: for x in `sudo find /etc/box/bbstored -type f` ; do zip bbstored.zip ${x} ; done scp bbstored_old.zip user at new:~ and then on the new store, moved that zip file to /etc/box/bbstored/ and unzipped it (well not exactly, but something like that with a few key-file moves by hand to clean up the little mess I had just made). (For editing some files, I typically lately use something like the following to make a quick backup first: sudo cp -p --force --backup=t /etc/box/bbstored/accounts.txt /etc/box/bbstored/accounts.txt ; sudo vim /etc/box/bbstored/accounts.txt ) One both machines, to make sure I had the same files in /etc/box/bbstored/, I ran: for x in `sudo find /etc/box/bbstored -type f` ; do sudo sha1sum ${x} ; done and compared the output. I've been using the rsync command below approx daily over the past several weeks to keep the old and new stores in sync in prep for the move. See the write.lock errors that I got for every account. user at new:~$ sudo nice rsync -v -u -a --delete --rsh=ssh --stats user at old:/root/.bu /media/bu receiving file list ... done backup// backup//info.rfw backup//od7.rfw backup//odb.rfw rsync: send_files failed to open "/root/.bu/backup//write.lock": Permission denied (13) backup//14/ backup//14/o55.rfw Number of files: 85820 Number of files transferred: 14 Total file size: 9811359304 bytes Total transferred file size: 252040 bytes Literal data: 12277 bytes Matched data: 239763 bytes File list size: 1212024 File list generation time: 9.919 seconds File list transfer time: 0.000 seconds Total bytes sent: 2760 Total bytes received: 1227291 sent 2760 bytes received 1227291 bytes 54668.93 bytes/sec total size is 9811359304 speedup is 7976.38 rsync error: some files could not be transferred (code 23) at main.c(1173) I think the write.lock errors are because those files are owned by user _bbstored with -rw------- 1 _bbstored users 0 2006-05-13 18:19 write.lock and I'm too lazy and not smart enough to figure out if I should change those permissions to 644 or rsync as root (on ubuntu) or other fix, or if I need them on the new store anyway (maybe they get recreated? Please help!). To check the new store, I ran: for x in `sudo cat /etc/box/bbstored/accounts.txt | cut -d\: -f1` ; do sudo bbstoreaccounts check ${x} ; done and for each account I got (note the many "wrong size" errors): Check store account ID Phase 1, check objects... Directory ID 496 has wrong size for object 82b Phase 3, check root... Phase 4, fix unattached objects... Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... Store account checked, no errors found. I don't know if that "wrong size" error is important, and/or related to the write.lock error, or neither. When I get the two errors above resolved, I'll change my firewall to forward port 2201 to the new machine. And it should just work, right? (I'll eventually move my current boxbackup0.09 clients to 0.10, but I'm in no rush. However, for new Windows clients, I'll use http://boxbackup.hostworks.ca/index.php/0.10#Windows_native_client_status ) From boxbackup at fluffy.co.uk Sun May 14 04:50:21 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 14 May 2006 03:50:21 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060513224727.10895.qmail@web60625.mail.yahoo.com> References: <20060513224727.10895.qmail@web60625.mail.yahoo.com> Message-ID: Hi Pete, On Sat, 13 May 2006, E.W. Peter Jalajas wrote: > I'm moving my store from an old ubuntu (debian) breezy machine with > boxbackup 0.09 to a new ubuntu dapper with boxbackup 0.10 across the > room. I've got about 8 clients out there in the wild, mostly > boxwin0.09. About 9 GB and 85,000 files in the store. [...] > I don't know if that "wrong size" error is important, and/or related to > the write.lock error, or neither. Please could you try to find out if they're related? i.e. do you get an error on the write lock file, or corruption detected by bbackupstore check, only when the client is currently logged in and backing up during the rsync? (I hope so). > When I get the two errors above resolved, I'll change my firewall to > forward port 2201 to the new machine. And it should just work, right? > > (I'll eventually move my current boxbackup0.09 clients to 0.10, but I'm > in no rush. However, for new Windows clients, I'll use > http://boxbackup.hostworks.ca/index.php/0.10#Windows_native_client_status > ) Unfortunately not. You have to upgrade all the clients to 0.10 to work with a 0.10 server, or leave the server on 0.09. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 14 07:53:07 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sat, 13 May 2006 23:53:07 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060514065307.9491.qmail@web60615.mail.yahoo.com> Hi again and thanks, Chris. > Please could you try to find out if they're related? i.e. do you get > an > error on the write lock file, or corruption detected by bbackupstore > check, only when the client is currently logged in and backing up > during > the rsync? (I hope so). Each of my 8 accounts on the old store looks exactly like this: user at old:~$ date Sun May 14 00:57:05 EDT 2006 user at old:~$ sudo find /root/.bu/ -name write.lock -exec ls -l {} \; -rw------- 1 _bbstored users 0 2006-05-14 00:51 /root/.bu/backup//write.lock I did: user at old:~$ ps -ef | grep bbstored 1003 9304 1 0 May11 ? 00:00:00 /etc/init.d/bbstored 1003 9305 9304 0 May11 ? 00:16:23 /etc/init.d/bbstored user at old:~$ sudo /etc/init.d/bbstored stop Box Backup Store Server v0.09, (c) Ben Summers 2003, 2004 bbstored: exception Common OSFileOpenError (Can't open a file -- attempted to load a non-existant config file or bad file referenced within?) (1/2) {Note to self: add this to http://boxbackup.hostworks.ca/index.php/Feature_requests ) user at old:~$ sudo kill 9304 (sudo xargs kill < /var/run/bbackupd.pid # should work too) user at old:~$ ps -ef | grep bbstored user at old:~$ sudo find /root/.bu/ -name write.lock -exec ls -l {} \; -rw------- 1 _bbstored users 0 2006-05-14 00:51 /root/.bu/backup//write.lock user at old:~$sudo fuser /root/.bu/backup//write.lock user at old:~$sudo lsof /root/.bu/backup//write.lock Then on the new server: user at new:~$ ps -ef | grep bb user at new:~$ sudo nice rsync -v -u -a --delete --rsh=ssh --stats user at old:/root/.bu/backup /media/ raid/bu Password: Password: receiving file list ... done backup// backup//info.rfw backup//odf.rfw rsync: send_files failed to open "/root/.bu/backup//write.lock": Permission denied (13) with the rsync error repeated once for each of the 8 accounts. user at new:~$ sudo bbstored check Box Backup Store Server v0.10, (c) Ben Summers and contributors 2003-2006 bbstored: failed to start: failed to open configuration file: check (I didn't think that was right...) user at new:~$ sudo bbstoreaccounts check Check store account ID Phase 1, check objects... Directory ID 5e53 has wrong size for object 653b Phase 3, check root... Phase 4, fix unattached objects... Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... Store account checked, no errors found. So, answering your question, I think I get the error and corruption whether or not the client is logged in (there never is a client logged in on the new server, yet) or the rsync is running. Well, now thinking about it a bit more, I tried: user at new:~$ sudo find /media/bu/ | grep 5e53 user at new:~$ sudo find /media/bu/ | grep 653b (the Directory ID and object listed above) and found nothing. Does that make sense? user at new:~$ ls -al /media/bu/backup/ total 1920 drwxr-xr-x 103 _bbstored users 8192 2006-05-14 00:51 . drwxr-xr-x 10 _bbstored users 4096 2006-05-04 01:22 .. drwxr-xr-x 2 _bbstored users 8192 2006-05-12 05:12 01 drwxr-xr-x 2 _bbstored users 8192 2006-05-03 14:43 02 -rw-r--r-- 1 _bbstored users 88 2006-05-14 00:51 info.rfw -rw-r--r-- 1 _bbstored users 456 2006-02-06 19:08 o01.rfw -rw------- 1 _bbstored users 0 2006-05-14 01:16 write.lock Does that look right? Here's the same from the old server: user at old:~$ sudo ls -l /root/.bu/backup// | less total 1908 drwxr-xr-x 2 _bbstored users 8192 2006-05-12 05:12 01 drwxr-xr-x 2 _bbstored users 8192 2006-05-03 14:43 02 -rw-r--r-- 1 _bbstored users 88 2006-05-14 00:51 info.rfw -rw-r--r-- 1 _bbstored users 456 2006-02-06 19:08 o01.rfw -rw------- 1 _bbstored users 0 2006-05-14 00:51 write.lock Wait, how did the write.lock file get there in the first place, if it wasn't ever readable by the rsync from the new server?! And just to be sure: user at old:~$ sudo du -shc /root/.bu/backup/* 869M /root/.bu/backup/ 664M /root/.bu/backup/ 3.2G /root/.bu/backup/ 1.4G /root/.bu/backup/ 88M /root/.bu/backup/ 1.7G /root/.bu/backup/ 168M /root/.bu/backup/ 1.5G /root/.bu/backup/ 9.4G total and user at new:~$ sudo du -shc /media/bu/backup/* 869M /media/bu/backup/ 664M /media/bu/backup/ 3.2G /media/bu/backup/ 1.4G /media/bu/backup/ 88M /media/bu/backup/ 1.7G /media/bu/backup/ 168M /media/bu/backup/ 1.5G /media/bu/backup/ 9.4G total Should I try?: /usr/local/bin/bbstoreaccounts check fix from: http://boxbackup.hostworks.ca/index.php/Fixing_problems_with_corruption_on_the_server Hmm, I just noticed that my old store raidfile.conf has: BlockSize = 4096 and my new store raidfile.conf has: BlockSize = 1024 Could that be the/a problem? (See my other comment about asking for someone to doc how to choose a BlockSize. When is a high or low BlockSize preferred?) It is fairly painless for me to reconstruct the new store from scratch if we have to (I can reformat the harddrives and rebuild the array if we have to). But, I _really_ don't want to re-backup all the clients (I'm only getting about 4 hours per stored GB from the clients at the moment, so that would be about 40 hours. Copying from the old to the new server is so fast that I haven't measured it, yet.). By the way, where are the account limits stored? Are they in info.rfw? I'm too exhausted at the moment to freak out about the fact that I have to upgrade my clients. I had thought that the constraint was the other way around...stupid me, it's pretty clear at http://www.fluffy.co.uk/boxbackup/ but not mentioned on the wiki, I think. Any pointers to the easiest way to upgrade Windows clients from 0.09 to 0.10? Thanks again for all your time and effort, Chris! I really appreciate it. (Per, I'm poking around the doc stuff. It'll take me some time to setup my xml editing system.) Pete --- Chris Wilson wrote: > Hi Pete, > > On Sat, 13 May 2006, E.W. Peter Jalajas wrote: > > I don't know if that "wrong size" error is important, and/or > related to > > the write.lock error, or neither. > > Please could you try to find out if they're related? i.e. do you get > an > error on the write lock file, or corruption detected by bbackupstore > check, only when the client is currently logged in and backing up > during > the rsync? (I hope so). > > Unfortunately not. You have to upgrade all the clients to 0.10 to > work > with a 0.10 server, or leave the server on 0.09. From boxbackup at fluffy.co.uk Sun May 14 16:30:14 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 14 May 2006 15:30:14 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060514065307.9491.qmail@web60615.mail.yahoo.com> References: <20060514065307.9491.qmail@web60615.mail.yahoo.com> Message-ID: Hi Pete, > Hi again and thanks, Chris. No problem :-) > user at old:~$ ps -ef | grep bbstored > > > user at old:~$ sudo find /root/.bu/ -name write.lock -exec ls -l {} \; > -rw------- 1 _bbstored users 0 2006-05-14 00:51 > /root/.bu/backup//write.lock Pete, are you sure there were no clients logged into the store when you killed bbstored? It doesn't look like there were, but it might be a good idea to sanity check. I don't think the lock files should be there, but the fact that they are, probably means that the server crashed while all eight accounts were backing up. This probably means that you have corruption on the old server, and you should run "bbstoreaccounts check fix" on there, and then rsync to the new server again. > Well, now thinking about it a bit more, I tried: > user at new:~$ sudo find /media/bu/ | grep 5e53 > user at new:~$ sudo find /media/bu/ | grep 653b > (the Directory ID and object listed above) and found nothing. > Does that make sense? Yes, because the filename is not the full ID. I think these files would be named something like "53/o5e.rfw" and "3b/o65.rfw", but I'm not sure. > -rw------- 1 _bbstored users 0 2006-05-14 01:16 write.lock > > Does that look right? Apart from the write.lock, yes. > Wait, how did the write.lock file get there in the first place, if it > wasn't ever readable by the rsync from the new server?! I guess that rsync created it even though it wasn't able to read the contents. But yes, it surprises me too. > Should I try?: > /usr/local/bin/bbstoreaccounts check fix > from: > http://boxbackup.hostworks.ca/index.php/Fixing_problems_with_corruption_on_the_server Yes, on the old server. > Hmm, I just noticed that my old store raidfile.conf has: > BlockSize = 4096 > and my new store raidfile.conf has: > BlockSize = 1024 I don't think it makes any difference without RAID, which you aren't using? > Could that be the/a problem? (See my other comment about asking for > someone to doc how to choose a BlockSize. When is a high or low > BlockSize preferred?) There is a tradeoff between better performance with large blocks, and more wasted space at the end of files. You can calculate the wasted space by dividing each file size by the block size, subtract the remainder from the block size, and sum over all files. E.g. File name == foo.txt File size == 4100 bytes Block size == 4096 bytes File % Block == 4 bytes Wasted == 4096 - 4 == 4092 bytes > By the way, where are the account limits stored? Are they in info.rfw? Not sure, but I think it's either there or accounts.txt. > I'm too exhausted at the moment to freak out about the fact that I have > to upgrade my clients. I would, if I were you. I didn't like that either. > I had thought that the constraint was the other > way around...stupid me, it's pretty clear at > http://www.fluffy.co.uk/boxbackup/ but not mentioned on the wiki, I > think. Please could you add it in an appropriate place (or two)? > Any pointers to the easiest way to upgrade Windows clients from 0.09 to > 0.10? Should be quite easy: * stop the old client service * unzip the new zipfile * copy all the contents into the directory where the old client is stored * remove the old service with "bbackupd.exe -r" * install it again with "bbackupd -i" (and the path to the configuration file, if not "C:\Program Files\Box Backup\bbackupd.conf") * start the service, check that it stays running. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 14 21:35:58 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 14 May 2006 21:35:58 +0100 Subject: [Box Backup] How to move store to new machine In-Reply-To: References: <20060514065307.9491.qmail@web60615.mail.yahoo.com> Message-ID: On 14 May 2006, at 16:30, Chris Wilson wrote: > >> user at old:~$ ps -ef | grep bbstored >> >> >> user at old:~$ sudo find /root/.bu/ -name write.lock -exec ls -l {} \; >> -rw------- 1 _bbstored users 0 2006-05-14 00:51 >> /root/.bu/backup//write.lock > > Pete, are you sure there were no clients logged into the store when > you killed bbstored? It doesn't look like there were, but it might > be a good idea to sanity check. > > I don't think the lock files should be there, but the fact that > they are, probably means that the server crashed while all eight > accounts were backing up. This probably means that you have > corruption on the old server, and you should run "bbstoreaccounts > check fix" on there, and then rsync to the new server again. write.lock files aren't deleted. The fact it's there doesn't mean anything, it's whether a process has a lock on it which is important. When a client logs in, the write.lock file will be recreated. > >> Hmm, I just noticed that my old store raidfile.conf has: >> BlockSize = 4096 >> and my new store raidfile.conf has: >> BlockSize = 1024 > > I don't think it makes any difference without RAID, which you > aren't using? No, block size is ignored for non-raid files. But do keep the configs the same. > >> Could that be the/a problem? (See my other comment about asking for >> someone to doc how to choose a BlockSize. When is a high or low >> BlockSize preferred?) > > There is a tradeoff between better performance with large blocks, > and more wasted space at the end of files. You can calculate the > wasted space by dividing each file size by the block size, subtract > the remainder from the block size, and sum over all files. E.g. > > File name == foo.txt > File size == 4100 bytes > Block size == 4096 bytes > File % Block == 4 bytes > Wasted == 4096 - 4 == 4092 bytes Incorrect. Block size is used by the raidfile lib to make sure that it doesn't waste filing system blocks. So you don't "choose" a block size, you set it to what your filing system uses as the underlying unit of storage. > > >> I'm too exhausted at the moment to freak out about the fact that I >> have >> to upgrade my clients. > > I would, if I were you. I didn't like that either. This decision was made to avoid releasing code which would be used in an untested manner. Really, I would suggest starting again. * Stop bbstored. (and make sure all the child processes terminate too -- they may delay doing so until the user has logged out) * As _bbstored, tar up the accounts. * Transfer the tar file. (if you don't have enough disc space, use netcat to transfer it as it's created) * As _bbstored, untar the accounts. * Restart bbstored on the second machine. It seems like "something funny" has happened. This should all just work. Ben From boxbackup at fluffy.co.uk Sun May 14 22:10:46 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Sun, 14 May 2006 22:10:46 +0100 Subject: [Box Backup] How to move store to new machine In-Reply-To: References: <20060514065307.9491.qmail@web60615.mail.yahoo.com> Message-ID: <1147641046.19324.14.camel@avenin.ebourne.me.uk> On Sun, 2006-05-14 at 21:35 +0100, Ben Summers wrote: > ... > >> I'm too exhausted at the moment to freak out about the fact that I > >> have > >> to upgrade my clients. > > > > I would, if I were you. I didn't like that either. > > This decision was made to avoid releasing code which would be used in > an untested manner. While I agree with the problems about testing I think it's unworkable to not provide any kind of backwards compatibility in the future. It's just not possible to simultaneously upgrade all clients and servers at the same time in most situations. I think the solution here would be to provide very limited backwards compatibility just to ease the upgrade transition. eg. The server supports its current and previous protocol versions, and the client just supports its current version. This does double the amount of testing, but also provides for the upgrade path of server first then clients. We should be able to come up with automated tests based on the current ones to help us with this testing. Cheers, Martin. From boxbackup at fluffy.co.uk Mon May 15 16:27:09 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Mon, 15 May 2006 18:27:09 +0300 Subject: [Box Backup] How to Ignore file backup? Message-ID: <948a6d890605150827x7b832456l42b6de4d5fb61a68@mail.gmail.com> Hello, I've set up box backup to backup my files Now I have the following problem: I receive this error by mail everytime when the following happens: May 15 06:39:29 secure1 bbackupd[6503]: Backup object failed, error when reading /opt/nagios/instances/jobmedia/var/nagios.tmpFgvklz Can please somebody tell me how to setup ExcludeFilesRegex so it does not try to backup files that match this pattern: nagios.tmp* I tried ExcludeFilesRegex =3D nagios.tmp* but it does not work Please help thank you From boxbackup at fluffy.co.uk Mon May 15 20:49:42 2006 From: boxbackup at fluffy.co.uk (Wendell MacKenzie) Date: Mon, 15 May 2006 15:49:42 -0400 Subject: [Box Backup] error during backup Message-ID: <4468DB56.7070005@sympatico.ca> Hi: I've got BoxBackup 0.10 setup and running my SUSE 10.0 laptop backing it up to a network server which I have the filesystem disk mounted via SAMBA as a shared drive. This drive mountpoint is in my raidfile.conf. During the backup cycle, I get the following at some point into the backup...approximately 6GB Into it... May 10 10:04:31 macduff bbackupd[13149]: Send GetBlockIndexByName(0x2fc97,".xsession-errors") May 10 10:04:31 macduff bbstored[16010]: Receive GetBlockIndexByName(0x2fc97,OPAQUE) May 10 10:04:31 macduff bbstored[16010]: Receive GetBlockIndexByName(0x2fc97,OPAQUE) May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) May 10 10:04:31 macduff bbstored[16010]: Sending stream, size 700 May 10 10:04:31 macduff bbackupd[13149]: Receive Success(0x2fd6e) May 10 10:04:31 macduff bbackupd[13149]: Receiving stream, size 700 May 10 10:04:31 macduff bbackupd[13149]: Send StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,".xsession-errors") May 10 10:04:31 macduff bbackupd[13149]: Sending stream, size uncertain May 10 10:04:31 macduff bbstored[16010]: Receive StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) May 10 10:04:31 macduff bbstored[16010]: Receive StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) May 10 10:04:31 macduff bbstored[16010]: Receiving stream, size uncertain May 10 10:04:31 macduff bbstored[16010]: Connection statistics for BACKUP-388: IN=745809 OUT=16231079 TOTAL=16976888 May 10 10:04:31 macduff bbstored[16010]: in server child, exception Common OSFileWriteError (1/15) -- terminating child May 10 10:04:31 macduff bbackupd[13149]: Send SetClientStoreMarker(0x4136f951f59c0) May 10 10:04:31 macduff bbackupd[13149]: Exception caught (Connection TLSReadFailed (Probably a network issue between client and server.) 7/34), reset state and waiting to retry... May 10 10:04:41 macduff bbackupd[13149]: File statistics: total file size uploaded 101978, bytes already on server 96070, encoded size 3195 Does boxbackup not work with shared disks as mount points? Any suggestions? thx, Wendell Mackenzie From boxbackup at fluffy.co.uk Mon May 15 20:53:02 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Mon, 15 May 2006 20:53:02 +0100 Subject: [Box Backup] error during backup In-Reply-To: References: Message-ID: <404FC9AF-4ED7-41E5-8B9C-242F716FD006@fluffy.co.uk> On 15 May 2006, at 20:49, Wendell MacKenzie wrote: > Hi: > > I've got BoxBackup 0.10 setup and running my SUSE 10.0 laptop > backing > it up to a network server which I have the filesystem disk mounted > via SAMBA as a shared drive. This drive mountpoint is in my > raidfile.conf. > > During the backup cycle, I get the following at some point into the > backup...approximately 6GB Into it... > > May 10 10:04:31 macduff bbackupd[13149]: Send > GetBlockIndexByName(0x2fc97,".xsession-errors") May 10 10:04:31 > macduff > bbstored[16010]: Receive GetBlockIndexByName(0x2fc97,OPAQUE) > May 10 10:04:31 macduff bbstored[16010]: Receive > GetBlockIndexByName(0x2fc97,OPAQUE) > May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) > May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) > May 10 10:04:31 macduff bbstored[16010]: Sending stream, size 700 > May 10 10:04:31 macduff bbackupd[13149]: Receive Success(0x2fd6e) > May 10 10:04:31 macduff bbackupd[13149]: Receiving stream, size 700 > May 10 10:04:31 macduff bbackupd[13149]: Send > StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f, > 0x2fd6e,".xsession-errors") > May 10 10:04:31 macduff bbackupd[13149]: Sending stream, size > uncertain > May 10 10:04:31 macduff bbstored[16010]: Receive > StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) > May 10 10:04:31 macduff bbstored[16010]: Receive > StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) > May 10 10:04:31 macduff bbstored[16010]: Receiving stream, size > uncertain > May 10 10:04:31 macduff bbstored[16010]: Connection statistics for > BACKUP-388: IN=745809 OUT=16231079 TOTAL=16976888 > May 10 10:04:31 macduff bbstored[16010]: in server child, exception > Common OSFileWriteError (1/15) -- terminating child > May 10 10:04:31 macduff bbackupd[13149]: Send > SetClientStoreMarker(0x4136f951f59c0) > May 10 10:04:31 macduff bbackupd[13149]: Exception caught (Connection > TLSReadFailed (Probably a network issue between client and server.) > 7/34), reset state and waiting to retry... > May 10 10:04:41 macduff bbackupd[13149]: File statistics: total file > size uploaded 101978, bytes already on server 96070, encoded size 3195 > > > Does boxbackup not work with shared disks as mount points? Any > suggestions? It hasn't been tested, but as long as the shared disc provides the standard locking semantics, then you should be fine. This is a clue: OSFileWriteError -- bbstored can't write a file. You need to work out why. Have you run out of disc space on the share? Ben From boxbackup at fluffy.co.uk Mon May 15 21:04:44 2006 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Mon, 15 May 2006 22:04:44 +0200 Subject: [Box Backup] How to Ignore file backup? References: <948a6d890605150827x7b832456l42b6de4d5fb61a68@mail.gmail.com> Message-ID: <009401c6785a$cf316b30$0101050a@stenorhem> > I tried ExcludeFilesRegex = nagios.tmp* but it does not work This has been up several times... It should be nagios.* as in Perl regexps. http://www.boost.org/libs/regex/doc/syntax_perl.html Regards, Stefan From boxbackup at fluffy.co.uk Mon May 15 22:25:49 2006 From: boxbackup at fluffy.co.uk (Wendell MacKenzie) Date: Mon, 15 May 2006 17:25:49 -0400 Subject: [Box Backup] error during backup In-Reply-To: <404FC9AF-4ED7-41E5-8B9C-242F716FD006@fluffy.co.uk> References: <404FC9AF-4ED7-41E5-8B9C-242F716FD006@fluffy.co.uk> Message-ID: <4468F1DD.9060205@sympatico.ca> Ben Summers wrote: > > On 15 May 2006, at 20:49, Wendell MacKenzie wrote: > >> Hi: >> >> I've got BoxBackup 0.10 setup and running my SUSE 10.0 laptop backing >> it up to a network server which I have the filesystem disk mounted >> via SAMBA as a shared drive. This drive mountpoint is in my >> raidfile.conf. >> >> During the backup cycle, I get the following at some point into the >> backup...approximately 6GB Into it... >> >> May 10 10:04:31 macduff bbackupd[13149]: Send >> GetBlockIndexByName(0x2fc97,".xsession-errors") May 10 10:04:31 macduff >> bbstored[16010]: Receive GetBlockIndexByName(0x2fc97,OPAQUE) >> May 10 10:04:31 macduff bbstored[16010]: Receive >> GetBlockIndexByName(0x2fc97,OPAQUE) >> May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) >> May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) >> May 10 10:04:31 macduff bbstored[16010]: Sending stream, size 700 >> May 10 10:04:31 macduff bbackupd[13149]: Receive Success(0x2fd6e) >> May 10 10:04:31 macduff bbackupd[13149]: Receiving stream, size 700 >> May 10 10:04:31 macduff bbackupd[13149]: Send >> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,".xsession-errors") >> >> May 10 10:04:31 macduff bbackupd[13149]: Sending stream, size uncertain >> May 10 10:04:31 macduff bbstored[16010]: Receive >> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) >> May 10 10:04:31 macduff bbstored[16010]: Receive >> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) >> May 10 10:04:31 macduff bbstored[16010]: Receiving stream, size >> uncertain >> May 10 10:04:31 macduff bbstored[16010]: Connection statistics for >> BACKUP-388: IN=745809 OUT=16231079 TOTAL=16976888 >> May 10 10:04:31 macduff bbstored[16010]: in server child, exception >> Common OSFileWriteError (1/15) -- terminating child >> May 10 10:04:31 macduff bbackupd[13149]: Send >> SetClientStoreMarker(0x4136f951f59c0) >> May 10 10:04:31 macduff bbackupd[13149]: Exception caught (Connection >> TLSReadFailed (Probably a network issue between client and server.) >> 7/34), reset state and waiting to retry... >> May 10 10:04:41 macduff bbackupd[13149]: File statistics: total file >> size uploaded 101978, bytes already on server 96070, encoded size 3195 >> >> >> Does boxbackup not work with shared disks as mount points? Any >> suggestions? > > It hasn't been tested, but as long as the shared disc provides the > standard locking semantics, then you should be fine. > > This is a clue: OSFileWriteError -- bbstored can't write a file. You > need to work out why. Have you run out of disc space on the share? > > Ben > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > Nope, server has 300G free.... From boxbackup at fluffy.co.uk Mon May 15 22:31:30 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Mon, 15 May 2006 22:31:30 +0100 Subject: [Box Backup] error during backup In-Reply-To: References: <404FC9AF-4ED7-41E5-8B9C-242F716FD006@fluffy.co.uk> Message-ID: <4F9146BB-A829-4A96-93DD-E70CE0314BDB@fluffy.co.uk> On 15 May 2006, at 22:25, Wendell MacKenzie wrote: > Ben Summers wrote: >> >> On 15 May 2006, at 20:49, Wendell MacKenzie wrote: >> >>> Hi: >>> >>> I've got BoxBackup 0.10 setup and running my SUSE 10.0 laptop >>> backing >>> it up to a network server which I have the filesystem disk mounted >>> via SAMBA as a shared drive. This drive mountpoint is in my >>> raidfile.conf. >>> >>> During the backup cycle, I get the following at some point >>> into the >>> backup...approximately 6GB Into it... >>> >>> May 10 10:04:31 macduff bbackupd[13149]: Send >>> GetBlockIndexByName(0x2fc97,".xsession-errors") May 10 10:04:31 >>> macduff >>> bbstored[16010]: Receive GetBlockIndexByName(0x2fc97,OPAQUE) >>> May 10 10:04:31 macduff bbstored[16010]: Receive >>> GetBlockIndexByName(0x2fc97,OPAQUE) >>> May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) >>> May 10 10:04:31 macduff bbstored[16010]: Send Success(0x2fd6e) >>> May 10 10:04:31 macduff bbstored[16010]: Sending stream, size 700 >>> May 10 10:04:31 macduff bbackupd[13149]: Receive Success(0x2fd6e) >>> May 10 10:04:31 macduff bbackupd[13149]: Receiving stream, size 700 >>> May 10 10:04:31 macduff bbackupd[13149]: Send >>> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f, >>> 0x2fd6e,".xsession-errors") >>> >>> May 10 10:04:31 macduff bbackupd[13149]: Sending stream, size >>> uncertain >>> May 10 10:04:31 macduff bbstored[16010]: Receive >>> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) >>> May 10 10:04:31 macduff bbstored[16010]: Receive >>> StoreFile(0x2fc97,0x4136f94f19300,0x1051cde12b2a2d7f,0x2fd6e,OPAQUE) >>> May 10 10:04:31 macduff bbstored[16010]: Receiving stream, size >>> uncertain >>> May 10 10:04:31 macduff bbstored[16010]: Connection statistics for >>> BACKUP-388: IN=745809 OUT=16231079 TOTAL=16976888 >>> May 10 10:04:31 macduff bbstored[16010]: in server child, exception >>> Common OSFileWriteError (1/15) -- terminating child >>> May 10 10:04:31 macduff bbackupd[13149]: Send >>> SetClientStoreMarker(0x4136f951f59c0) >>> May 10 10:04:31 macduff bbackupd[13149]: Exception caught >>> (Connection >>> TLSReadFailed (Probably a network issue between client and server.) >>> 7/34), reset state and waiting to retry... >>> May 10 10:04:41 macduff bbackupd[13149]: File statistics: total file >>> size uploaded 101978, bytes already on server 96070, encoded size >>> 3195 >>> >>> >>> Does boxbackup not work with shared disks as mount points? Any >>> suggestions? >> >> It hasn't been tested, but as long as the shared disc provides the >> standard locking semantics, then you should be fine. >> >> This is a clue: OSFileWriteError -- bbstored can't write a file. You >> need to work out why. Have you run out of disc space on the share? > > Nope, server has 300G free.... Quotas? Anyway, you should look for reasons why it can't write that file. Ben From boxbackup at fluffy.co.uk Tue May 16 06:18:55 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Mon, 15 May 2006 22:18:55 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516051855.45152.qmail@web60622.mail.yahoo.com> Thanks, Ben. Replies below. Most urgently, at the very bottom: Exception caught (Common RegexNotSupportedOnThisPlatform (Your platform does not have built in regular expression libraries.) 1/30), reset state and waiting to retry... On the new store I've got: user at new:~$ dpkg -l | grep -e boost -e pcre ii libboost-dev 1.33.1-2 Boost C++ Libraries development files ii libboost-regex-dev 1.33.1-2 regular expression library for C++ ii libboost-regex1.33.1 1.33.1-2 regular expression library for C++ ii libpcre3 6.4-1.1ubuntu2 Perl 5 Compatible Regular Expression Lib ii pcregrep 6.4-1.1ubuntu4 grep utility that uses perl 5 compatible What should I do? Do I have to reboot after installing boost? Do I have too much boost? Thanks, Pete --- Ben Summers wrote: > > On 14 May 2006, at 16:30, Chris Wilson wrote: > > > >> Hmm, I just noticed that my old store raidfile.conf has: > >> BlockSize = 4096 > >> and my new store raidfile.conf has: > >> BlockSize = 1024 > > > > I don't think it makes any difference without RAID, which you > > aren't using? > > No, block size is ignored for non-raid files. But do keep the configs > the same. > > Block size is used by the raidfile lib to make sure that > it doesn't waste filing system blocks. So you don't "choose" a block > size, you set it to what your filing system uses as the underlying > unit of storage. We should probably be clearer when we talk about raid. I _am_ using raid (software md raid tools, RAID1 (mirror)), but not boxbackup userland raid. So, for the search engines, after some googling, I figured out "howto determine block size": user at old:~$ sudo tune2fs -l /dev/hda1 | grep "Block size" Block size: 4096 user at new:~$ sudo tune2fs -l /dev/md0 | grep "Block size" Block size: 4096 dumpe2fs also works. (I'll add a thing about block size to the wiki.) But, I figured that if I'm gonna start from scratch, I might as well tune this baby to meet my needs (with a reasonable sample of 8 clients, 85,000 files, and 9GB), so I did: user at old:~$ for x in 1 2 3 4 8 16 512 1024 10000 100000 1000000 10000000 100000000 ; do echo -n "${x}: " ; sudo find /root/.bu/backup -size -$x -type f | wc -l ; done 1: 8 2: 21886 3: 39772 4: 47932 8: 60383 16: 69076 512: 82582 1024: 83371 10000: 85227 100000: 85473 1000000: 85492 10000000: 85493 100000000: 85493 So, while my average filesize is 9GB/80K = ~100K, it looks like more than half my files are under 4k, so I'll keep that as my block size. (The clients are a small tax firm, property management firm, environmental consultant, a database guy, a hardware broker, me (x2), and Mom -- maybe that's a reasonable cross-section of small businesses.) > >> I'm too exhausted at the moment to freak out about the fact that I > >> have to upgrade my clients. > This decision was made to avoid releasing code which would be used in > an untested manner. Sorry, Ben, the freaking was a comment about my own current lifeload, not at all about the product. > Really, I would suggest starting again. > * Stop bbstored. (and make sure all the child processes terminate too > -- they may delay doing so until the user has logged out) > > * As _bbstored, tar up the accounts. user at old:~$ su _bbstored su: User not known to the underlying authentication module user at old:~$ grep bb /etc/passwd _bbstored:x:1003:100::/home/_bbstored: user at old:~$ sudo passwd _bbstored Password: passwd: User not known to the underlying authentication module But then, persistence paid off: user at old:~$ sudo su _bbstored sh-3.00$ id uid=1003(_bbstored) gid=100(users) groups=100(users) I did that on both the old and new store server. > * Transfer the tar file. (if you don't have enough disc space, use > netcat to transfer it as it's created) > * As _bbstored, untar the accounts. On the new (target) store server, I first made a new store directory: sudo mkdir -p /media/bu2/backup sudo chown _bbstored /media/bu2 cd /media/bu2 I ran on the new store server, to get it waiting for the transfer, based on http://www.securitydocs.com/library/3376 : sh-3.00$ nc -l -p 4321 | tar xvpz ??? Then on the old (source) store server, I ran: sh-3.00$ tar zcp - ./backup/ | nc -w 3 new 4321 putting the files into a new store area. (I took out the "f" option because I got an error.) I modified my /etc/box/bbstored.conf to reflect that new store directory. That took a few hours, but not too long. user at new:~$ cat /etc/box/raidfile.conf disc0 { SetNumber = 0 BlockSize = 4096 Dir0 = /media/bu2 Dir1 = /media/bu2 Dir2 = /media/bu2 I checked my /etc/box/bbstored.conf and /etc/box/bbstored/accounts.txt to make sure they were ok. > * Restart bbstored on the second machine. user at new:~$ sudo /usr/local/bin/bbstored user at new:~$ ps -ef | grep bb 1005 22047 1 0 23:27 ? 00:00:00 /usr/local/bin/bbstored 1005 22048 22047 16 23:27 ? 00:00:01 /usr/local/bin/bbstored I then did: user at new:~$ sudo crontab -e and inserted: #restart bbstored if it isn't running */15 * * * * if [[ `ps -ef | grep bbstored | grep -v "grep bbstored" | wc -l` -lt 2 ]] ; then killall bbstored ; /usr/local/bin/bbstored ; fi I then sent a mail to all my clients saying that they are not backing up at the moment, and that I need to upgrade their client backup software ASAP. (Maybe I shoulda woulda coulda done them one at a time...) So, now I'm setting up via Remote Desktop Connection one of my clients, following the Upgrading instructions at http://boxbackup.hostworks.ca/index.php/Compile_on_Windows . The install went fine (first make a backup copy of your C:\Program Files\Box Backup\bbackupd.conf, and make sure that the zip Extract function actually worked (you should see "Replace...?" messages); I changed the CommandSocket to "pipe" because that was in the bbackupd.conf that came with 0.10, and I extended MaximumDiffingTime to 40, but I'm not sure why.) And so, it's trying to connect to my new server, but now on the client I get: Exception caught (Common RegexNotSupportedOnThisPlatform (Your platform does not have built in regular expression libraries.) 1/30), reset state and waiting to retry... On the new store I've got: user at new:~$ dpkg -l | grep -e boost -e pcre ii libboost-dev 1.33.1-2 Boost C++ Libraries development files ii libboost-regex-dev 1.33.1-2 regular expression library for C++ ii libboost-regex1.33.1 1.33.1-2 regular expression library for C++ ii libpcre3 6.4-1.1ubuntu2 Perl 5 Compatible Regular Expression Lib ii pcregrep 6.4-1.1ubuntu4 grep utility that uses perl 5 compatible The server /var/log/box shows this client (and all the others) logging in, and for this client, show listing directories, setting store marker, session finished, but no mention of any problem. What am I missing? Thanks, Pete > It seems like "something funny" has happened. This should all just > work. > > Ben > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue May 16 07:31:00 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 16 May 2006 07:31:00 +0100 Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516051855.45152.qmail@web60622.mail.yahoo.com> References: <20060516051855.45152.qmail@web60622.mail.yahoo.com> Message-ID: Few comments... * Block size is not used if you're not using userland raid. * What's up with the _bbstored user? * bbstored doesn't need any regex libraries, only the clients do. =20 UNIX clients find your regex support (man re_format), Windows clients =20= use boost. * To remove the exception, comment out the regex directives in =20 bbackupd.conf. But that will backup more files, so Chris needs to =20 help you with the Win32 client. Ben On 16 May 2006, at 06:18, E.W. Peter Jalajas wrote: > Thanks, Ben. Replies below. > > Most urgently, at the very bottom: > Exception caught (Common RegexNotSupportedOnThisPlatform (Your =20 > platform > does not have built in regular expression libraries.) 1/30), reset > state and waiting to retry... > > On the new store I've got: > user at new:~$ dpkg -l | grep -e boost -e pcre > ii libboost-dev 1.33.1-2 > Boost C++ Libraries development files > ii libboost-regex-dev 1.33.1-2 > regular expression library for C++ > ii libboost-regex1.33.1 1.33.1-2 > regular expression library for C++ > ii libpcre3 6.4-1.1ubuntu2 > Perl 5 Compatible Regular Expression Lib > ii pcregrep 6.4-1.1ubuntu4 > grep utility that uses perl 5 compatible > > What should I do? Do I have to reboot after installing boost? Do I > have too much boost? > > Thanks, > Pete > > --- Ben Summers wrote: >> >> On 14 May 2006, at 16:30, Chris Wilson wrote: >>> >>>> Hmm, I just noticed that my old store raidfile.conf has: >>>> BlockSize =3D 4096 >>>> and my new store raidfile.conf has: >>>> BlockSize =3D 1024 >>> >>> I don't think it makes any difference without RAID, which you >>> aren't using? >> >> No, block size is ignored for non-raid files. But do keep the configs >> the same. >> >> Block size is used by the raidfile lib to make sure that >> it doesn't waste filing system blocks. So you don't "choose" a block >> size, you set it to what your filing system uses as the underlying >> unit of storage. > > We should probably be clearer when we talk about raid. I _am_ using > raid (software md raid tools, RAID1 (mirror)), but not boxbackup > userland raid. > > So, for the search engines, after some googling, I figured out "howto > determine block size": > > user at old:~$ sudo tune2fs -l /dev/hda1 | grep "Block size" > Block size: 4096 > user at new:~$ sudo tune2fs -l /dev/md0 | grep "Block size" > Block size: 4096 > > dumpe2fs also works. > > (I'll add a thing about block size to the wiki.) > > But, I figured that if I'm gonna start from scratch, I might as well > tune this baby to meet my needs (with a reasonable sample of 8 =20 > clients, > 85,000 files, and 9GB), so I did: > > user at old:~$ for x in 1 2 3 4 8 16 512 1024 10000 100000 1000000 > 10000000 100000000 ; do echo -n "${x}: " ; sudo find /root/.bu/backup > -size -$x -type f | wc -l ; done > 1: 8 > 2: 21886 > 3: 39772 > 4: 47932 > 8: 60383 > 16: 69076 > 512: 82582 > 1024: 83371 > 10000: 85227 > 100000: 85473 > 1000000: 85492 > 10000000: 85493 > 100000000: 85493 > > So, while my average filesize is 9GB/80K =3D ~100K, it looks like more > than half my files are under 4k, so I'll keep that as my block size. > > (The clients are a small tax firm, property management firm, > environmental consultant, a database guy, a hardware broker, me (x2), > and Mom -- maybe that's a reasonable cross-section of small > businesses.) > >>>> I'm too exhausted at the moment to freak out about the fact that I >>>> have to upgrade my clients. > >> This decision was made to avoid releasing code which would be used in >> an untested manner. > > Sorry, Ben, the freaking was a comment about my own current lifeload, > not at all about the product. > >> Really, I would suggest starting again. > >> * Stop bbstored. (and make sure all the child processes terminate too >> -- they may delay doing so until the user has logged out) >> >> * As _bbstored, tar up the accounts. > > user at old:~$ su _bbstored > su: User not known to the underlying authentication module > user at old:~$ grep bb /etc/passwd > _bbstored:x:1003:100::/home/_bbstored: > user at old:~$ sudo passwd _bbstored > Password: > passwd: User not known to the underlying authentication module > But then, persistence paid off: > user at old:~$ sudo su _bbstored > sh-3.00$ id > uid=3D1003(_bbstored) gid=3D100(users) groups=3D100(users) > I did that on both the old and new store server. > >> * Transfer the tar file. (if you don't have enough disc space, use >> netcat to transfer it as it's created) >> * As _bbstored, untar the accounts. > > On the new (target) store server, I first made a new store directory: > sudo mkdir -p /media/bu2/backup > sudo chown _bbstored /media/bu2 > cd /media/bu2 > > I ran on the new store server, to get it waiting for the transfer, > based on http://www.securitydocs.com/library/3376 : > sh-3.00$ nc -l -p 4321 | tar xvpz =E2=80=93 > > Then on the old (source) store server, I ran: > sh-3.00$ tar zcp - ./backup/ | nc -w 3 new 4321 > putting the files into a new store area. (I took out the "f" option > because I got an error.) I modified my /etc/box/bbstored.conf to > reflect that new store directory. > > That took a few hours, but not too long. > > user at new:~$ cat /etc/box/raidfile.conf > disc0 > { > SetNumber =3D 0 > BlockSize =3D 4096 > Dir0 =3D /media/bu2 > Dir1 =3D /media/bu2 > Dir2 =3D /media/bu2 > > I checked my > /etc/box/bbstored.conf > and > /etc/box/bbstored/accounts.txt > to make sure they were ok. > >> * Restart bbstored on the second machine. > > user at new:~$ sudo /usr/local/bin/bbstored > user at new:~$ ps -ef | grep bb > 1005 22047 1 0 23:27 ? 00:00:00 /usr/local/bin/=20 > bbstored > 1005 22048 22047 16 23:27 ? 00:00:01 /usr/local/bin/=20 > bbstored > > I then did: > user at new:~$ sudo crontab -e > and inserted: > #restart bbstored if it isn't running > */15 * * * * if [[ `ps -ef | grep bbstored | grep -v "grep bbstored" | > wc -l` -lt 2 ]] ; then killall bbstored ; /usr/local/bin/bbstored ; fi > > I then sent a mail to all my clients saying that they are not backing > up at the moment, and that I need to upgrade their client backup > software ASAP. (Maybe I shoulda woulda coulda done them one at a > time...) > > So, now I'm setting up via Remote Desktop Connection one of my =20 > clients, > following the Upgrading instructions at > http://boxbackup.hostworks.ca/index.php/Compile_on_Windows . The > install went fine (first make a backup copy of your C:\Program > Files\Box Backup\bbackupd.conf, and make sure that the zip Extract > function actually worked (you should see "Replace...?" messages); I > changed the CommandSocket to "pipe" because that was in the > bbackupd.conf that came with 0.10, and I extended =20 > MaximumDiffingTime to > 40, but I'm not sure why.) > > And so, it's trying to connect to my new server, but now on the client > I get: > Exception caught (Common RegexNotSupportedOnThisPlatform (Your =20 > platform > does not have built in regular expression libraries.) 1/30), reset > state and waiting to retry... > On the new store I've got: > user at new:~$ dpkg -l | grep -e boost -e pcre > ii libboost-dev 1.33.1-2 > Boost C++ Libraries development files > ii libboost-regex-dev 1.33.1-2 > regular expression library for C++ > ii libboost-regex1.33.1 1.33.1-2 > regular expression library for C++ > ii libpcre3 6.4-1.1ubuntu2 > Perl 5 Compatible Regular Expression Lib > ii pcregrep 6.4-1.1ubuntu4 > grep utility that uses perl 5 compatible > > The server /var/log/box shows this client (and all the others) logging > in, and for this client, show listing directories, setting store > marker, session finished, but no mention of any problem. > > What am I missing? > > Thanks, > Pete > >> It seems like "something funny" has happened. This should all just >> work. >> >> Ben >> >> >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue May 16 11:48:51 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 10:48:51 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: References: <20060516051855.45152.qmail@web60622.mail.yahoo.com> Message-ID: Hi Ben and Pete, On Tue, 16 May 2006, Ben Summers wrote: > * bbstored doesn't need any regex libraries, only the clients do. UNIX > clients find your regex support (man re_format), Windows clients use > boost. Windows clients don't use Boost any more, as we ripped that out. They use PCRE now. But it should work in more or less the same way. The libraries must be present, detected and used at compile time (for the client), not at run time. > * To remove the exception, comment out the regex directives in > bbackupd.conf. But that will backup more files, so Chris needs to help > you with the Win32 client. Which exact version of the client are you using? I cannot test the latest one, as I'm getting certificate errors for some unknown reason, but I'm pretty sure that r538 had working regular expression support build in. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 14:07:12 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 06:07:12 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516130712.88136.qmail@web60614.mail.yahoo.com> Mornin' Guys, I used the windows zip linked at the bottom of http://boxbackup.hostworks.ca/index.php/0.10 and the instructions at the bottom of: http://boxbackup.hostworks.ca/index.php/Compile_on_Windows When the service starts, the Event Viewer reports version chris_general_564. In C:\Program Files\Box Backup\, I see the pcreposix.dll, 61,563 bytes, created Feb 9, 2006, modified May 5, 2006. I backed up my C:\Program Files\Box Backup\bbackupd.conf, unzipped the new client into a separate directory, copied those new files, and pasted them into C:\Program Files\Box Backup\, saying Yes to many overwrite warnings. (I got a handshake exception before I saw the overwrite warning, then I got the regex exception after.) (I didn't compile anything. I didn't run either of the 2 install* files.) When I Terminal Serviced back in this a.m., I saw an application failure or error box saying box backup crashed and do I want to report this to Microsoft; I chose no. The service was running when I left it last night, it is not running now. The Event Viewer says Application Error, Reporting queued error, faulting application bbackupd.exe, version 0.0.0.0, faulting module unknown, version 0.0.0.0, fault address 0x00000000. I notice that that event was logged right when I logged in, not some random time during the night. Looking backward, I see at 1:14 a.m. last night, I got Send SetClientStoreMarker, Receive Success, Send Finished, Receive Finished, RegEx exception, File Stats 0 uploaded, then at 1:27 Service shutdown (that was me, I think, shutting it down), and then 1:27:03, Application Error, faulting application bbackupd.exe. This is on Windows Server 2003 SE SP1. By the way, I got the same regex error on my Mother's XP home box a few days ago (that's when I installed the boost pkgs, but I haven't had the time to vnc-sc (a really neat tool) back into her machine to follow up). Thanks guys. I appreciate the effort. I gotta get everyone back up and running ASAP. Pete --- Chris Wilson wrote: > Hi Ben and Pete, > > On Tue, 16 May 2006, Ben Summers wrote: > > > * bbstored doesn't need any regex libraries, only the clients do. > UNIX > > clients find your regex support (man re_format), Windows clients > use > > boost. > > Windows clients don't use Boost any more, as we ripped that out. They > use > PCRE now. But it should work in more or less the same way. The > libraries > must be present, detected and used at compile time (for the client), > not > at run time. > > > * To remove the exception, comment out the regex directives in > > bbackupd.conf. But that will backup more files, so Chris needs to > help > > you with the Win32 client. > > Which exact version of the client are you using? I cannot test the > latest > one, as I'm getting certificate errors for some unknown reason, but > I'm > pretty sure that r538 had working regular expression support build > in. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 14:30:27 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 06:30:27 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516133027.87073.qmail@web60618.mail.yahoo.com> --- Ben Summers wrote: > > Few comments... > > * Block size is not used if you're not using userland raid. In my ignorance, I thought that block size was used as part of the "rsync algorithm" (which, by the way, I don't see referenced on http://boxbackup.hostworks.ca/index.php/What_it_is%2C_how_it_works , but it is mentioned http://www.fluffy.co.uk/boxbackup/ ). But I guess now that I think about it, that would be used then only on the client side. Could maybe someone draft up or point to a little thing about how that rsync changed-blocks-only thing works on the client and server (I tell my eyes-glazing clients, "trust me"); in the meantime, I'll read this again: http://samba.anu.edu.au/rsync/how-rsync-works.html . > * What's up with the _bbstored user? Not sure what you mean? He came from: http://www.fluffy.co.uk/boxbackup/server.html and/or http://boxbackup.hostworks.ca/index.php/Configuring_a_server > * To remove the exception, comment out the regex directives in > bbackupd.conf. But that will backup more files, so Chris needs to > help you with the Win32 client. The regex is filtering out tons of stuff, so I don't think I can do that, but I'll check it out, depending upon what Chris and I can work out. > Ben P.S. When you all have some free time, check out http://www.loband.org/loband/main which I found poking around Chris' site. I presume he had something to do with it. From boxbackup at fluffy.co.uk Tue May 16 16:27:00 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 15:27:00 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516130712.88136.qmail@web60614.mail.yahoo.com> References: <20060516130712.88136.qmail@web60614.mail.yahoo.com> Message-ID: Hi Pete, > I used the windows zip linked at the bottom of > http://boxbackup.hostworks.ca/index.php/0.10 > and the instructions at the bottom of: > http://boxbackup.hostworks.ca/index.php/Compile_on_Windows > > When the service starts, the Event Viewer reports version > chris_general_564. Please could you try this older version: http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/boxbackup-chris_general_538-backup-client-mingw32.zip I didn't test 564 before releasing it, because I thought it was a trivial change, but it's quite possible that I've broken something. I'm pretty sure that 538 works with regexes. I don't understand the crash either. Did it happen immediately after you stopped the service? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 16:37:30 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 15:37:30 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516133027.87073.qmail@web60618.mail.yahoo.com> References: <20060516133027.87073.qmail@web60618.mail.yahoo.com> Message-ID: Hi Pete, On Tue, 16 May 2006, E.W. Peter Jalajas wrote: >> * What's up with the _bbstored user? > > Not sure what you mean? He came from: > http://www.fluffy.co.uk/boxbackup/server.html > and/or > http://boxbackup.hostworks.ca/index.php/Configuring_a_server I think Ben was asking why it shows up as a numeric ID, with the wrong number, in your process listing? Is there any chance that you have two _bbstored users, or that a bbstored process started by a different user is still running? > P.S. When you all have some free time, check out > http://www.loband.org/loband/main which I found poking around Chris' > site. I presume he had something to do with it. Yeah, I was one of the developers who created it, and still work on it sometimes, when I have time (which is not very often, because it's now a completely unfunded project). Do you find it useful? We're always interested in feedback on how people use it. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 17:35:59 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 09:35:59 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516163559.24669.qmail@web60624.mail.yahoo.com> Hi Chris, Sorry, I'm still not sure what you're getting at re the _bbstored user. Can you paste the lines that you think are odd? One thing may be that I do usually start the bbstored process by doing sudo /etc/init.d/bbstore on the old server (I'm keeping it in /usr/local/bin/ on the new server, but I'm not sure why, but maybe because it doesn't have the start/stop function). (Do I recall reading somewhere that you start it as root, then _bbstored takes over, or was that some other project that I'm thinking about?) Sorry, a little off-topic, but your massive contribution to box backup should be rewarded with a little chat about your passions outside of bb: Re http://www.loband.org/loband/main, I don't really have a need for it (yet), but it was interesting to see how fast it was, how clean the output looked, and how it might be used for rich-country limited bandwidth situations like PDAs, cell phones, dial-up, etc. (easier than recoding ones entire website as whtml or whatever that wireless format is; just train those users to surf through loband.org). Thanks again, Pete --- Chris Wilson wrote: > Hi Pete, > > On Tue, 16 May 2006, E.W. Peter Jalajas wrote: > > >> * What's up with the _bbstored user? > > > > Not sure what you mean? He came from: > > http://www.fluffy.co.uk/boxbackup/server.html > > and/or > > http://boxbackup.hostworks.ca/index.php/Configuring_a_server > > I think Ben was asking why it shows up as a numeric ID, with the > wrong > number, in your process listing? Is there any chance that you have > two > _bbstored users, or that a bbstored process started by a different > user is > still running? > > > P.S. When you all have some free time, check out > > http://www.loband.org/loband/main which I found poking around > Chris' > > site. I presume he had something to do with it. > > Yeah, I was one of the developers who created it, and still work on > it > sometimes, when I have time (which is not very often, because it's > now a > completely unfunded project). Do you find it useful? We're always > interested in feedback on how people use it. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 17:58:04 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 09:58:04 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516165804.17097.qmail@web60613.mail.yahoo.com> That worked perfectly, Chris, thanks! It's catching up on the backups now. Is it me, or is 0.10 substantially faster than 0.09? After I restarted the service, by the time I could get to Event Veiwer, dozens of files had already been transfered. I don't recall that happening before. Now that I look at it again, I guess that is the number one new feature mentioned at: http://boxbackup.hostworks.ca/index.php/0.10 "Optimised diffing algorithm -- massive speed improvements" Re the 0.10 crash error, yes, I think it might have happened right after stopping the service, but I'm not absolutely sure. My recollection was that I thought it was odd that I was getting that message when I had thought that I had stopped the service a while ago (whatever "a while" means), like I might have said to myself, "I thought I stopped that? Was that still running?" or something like that, but I was so busy with other stuff that I didn't track it down. Thanks! again, Chris. Pete --- Chris Wilson wrote: > Hi Pete, > > > I used the windows zip linked at the bottom of > > http://boxbackup.hostworks.ca/index.php/0.10 > > and the instructions at the bottom of: > > http://boxbackup.hostworks.ca/index.php/Compile_on_Windows > > > > When the service starts, the Event Viewer reports version > > chris_general_564. > > Please could you try this older version: > > http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/boxbackup-chris_general_538-backup-client-mingw32.zip > > I didn't test 564 before releasing it, because I thought it was a > trivial > change, but it's quite possible that I've broken something. I'm > pretty > sure that 538 works with regexes. > > I don't understand the crash either. Did it happen immediately after > you > stopped the service? > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 18:00:15 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 17:00:15 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516163559.24669.qmail@web60624.mail.yahoo.com> References: <20060516163559.24669.qmail@web60624.mail.yahoo.com> Message-ID: Hi Pete, > Sorry, I'm still not sure what you're getting at re the _bbstored user. > Can you paste the lines that you think are odd? OK, I missed the fact that you were running the commands on different servers. On the old server, _bbstored has uid 1003, and on the new one 1005. It shouldn't matter, but make sure that your restored tarball has the correct permissions to allow the _bbstored user on new to write to it. It may not matter either if you're decommissioning the server, but this worries me: > user at old:~$ su _bbstored > su: User not known to the underlying authentication module > user at old:~$ grep bb /etc/passwd > _bbstored:x:1003:100::/home/_bbstored: This implies that your PAM setup is broken on the old machine. I hope this works on the new machine? Did you already try fixing the store on the old machine before copying? If not, please try that. > (Do I recall reading somewhere that you start it as root, then _bbstored > takes over, or was that some other project that I'm thinking about?) I'm not sure. There is a "user" option in the server config, but I haven't used it, so I don't know if it works. > Sorry, a little off-topic, but your massive contribution to box backup Thanks, but I hardly made a "massive contribution", especially in comparison to what the others have done. I just cleaned up Nick's port a bit, got it merged into the tree, and now I'm helping out with a bit of tech support when I have time. > Re http://www.loband.org/loband/main, I don't really have a need for it > (yet), but it was interesting to see how fast it was, how clean the > output looked, and how it might be used for rich-country limited > bandwidth situations like PDAs, cell phones, dial-up, etc. Thanks, I'm glad you liked it. It's not really designed for mobile phones, but some people have reported using it successfully with them. And it worked pretty well on my Treo, before that had a heart attack one day. And it is sometimes useful in developing countries. Here in Ghana, Sourceforge pages take forever to load; it's about three times faster through Loband. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 18:09:47 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 17:09:47 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516165804.17097.qmail@web60613.mail.yahoo.com> References: <20060516165804.17097.qmail@web60613.mail.yahoo.com> Message-ID: Hi Pete, > That worked perfectly, Chris, thanks! It's catching up on the backups > now. OK, that's good, it's just the last version that's borked. I'll make the wiki point to the old one again, for now. If I make a new release, would you be able to test it for me please, to verify that regexes are working properly? I'm making a few fixes to the command line handling for adding and removing services at the moment. I can't test it easily because my local server is borked. > Re the 0.10 crash error, yes, I think it might have happened right after > stopping the service, but I'm not absolutely sure. My recollection was > that I thought it was odd that I was getting that message when I had > thought that I had stopped the service a while ago (whatever "a while" > means), like I might have said to myself, "I thought I stopped that? Was > that still running?" or something like that, but I was so busy with > other stuff that I didn't track it down. Please could you have a look at your event viewer logs again. I think they showed the shutdown and the error happening almost simultaneously: > at 1:27 Service shutdown (that was me, I think, shutting it down), and > then 1:27:03, Application Error, faulting application bbackupd.exe Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 18:18:23 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 10:18:23 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516171823.29649.qmail@web60623.mail.yahoo.com> --- Chris Wilson wrote: > Hi Pete, > > > That worked perfectly, Chris, thanks! It's catching up on the > backups > > now. > > OK, that's good, it's just the last version that's borked. I'll make > the > wiki point to the old one again, for now. > > If I make a new release, would you be able to test it for me please, > to > verify that regexes are working properly? I'm making a few fixes to > the > command line handling for adding and removing services at the moment. > I > can't test it easily because my local server is borked. > > > Re the 0.10 crash error, yes, I think it might have happened right > after > > stopping the service, but I'm not absolutely sure. My recollection > was > > that I thought it was odd that I was getting that message when I > had > > thought that I had stopped the service a while ago (whatever "a > while" > > means), like I might have said to myself, "I thought I stopped > that? Was > > that still running?" or something like that, but I was so busy with > > > other stuff that I didn't track it down. > > Please could you have a look at your event viewer logs again. I think > they > showed the shutdown and the error happening almost simultaneously: > > > at 1:27 Service shutdown (that was me, I think, shutting it down), > and > > then 1:27:03, Application Error, faulting application bbackupd.exe > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 18:28:47 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Tue, 16 May 2006 18:28:47 +0100 Subject: [Box Backup] How to move store to new machine In-Reply-To: References: <20060516163559.24669.qmail@web60624.mail.yahoo.com> Message-ID: <20060516182847.29onx52xw0soocc8@ebourne.me.uk> Chris Wilson wrote: > ... >> Sorry, a little off-topic, but your massive contribution to box backup > > Thanks, but I hardly made a "massive contribution", especially in > comparison to what the others have done. I just cleaned up Nick's port > a bit, got it merged into the tree, and now I'm helping out with a bit > of tech support when I have time. I for one think yours counts as a very significant contribution! Rather more so than you claim. Cheers, Martin. From boxbackup at fluffy.co.uk Tue May 16 18:42:03 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 10:42:03 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516174203.27399.qmail@web60621.mail.yahoo.com> Sorry, clicked the wrong button... Yes, I'd be honored to try to test them for you. I have 5 or 6 more Windows clients to upgrade, so as I'm upgrading each one over the next day or two, I'll first try whatever your latest new one is and if that one doesn't work, I'll immediately fall back to this one that just worked. After that, I can do a little experimenting on my home WinXP box. Or maybe I can try to scavenge something from my digital boneyard, but I'm not sure if anything is beefy enough to support my old Win2K install disks--we'll see, I'll figure somethin' out. Sorry I can't be more useful on the crash. Let me see what I can do to re-create the problem when I get home... --- Chris Wilson wrote: > Hi Pete, > > > That worked perfectly, Chris, thanks! It's catching up on the > backups > > now. > > OK, that's good, it's just the last version that's borked. I'll make > the > wiki point to the old one again, for now. > > If I make a new release, would you be able to test it for me please, > to > verify that regexes are working properly? I'm making a few fixes to > the > command line handling for adding and removing services at the moment. > I > can't test it easily because my local server is borked. > > > Re the 0.10 crash error, yes, I think it might have happened right > after > > stopping the service, but I'm not absolutely sure. My recollection > was > > that I thought it was odd that I was getting that message when I > had > > thought that I had stopped the service a while ago (whatever "a > while" > > means), like I might have said to myself, "I thought I stopped > that? Was > > that still running?" or something like that, but I was so busy with > > > other stuff that I didn't track it down. > > Please could you have a look at your event viewer logs again. I think > they > showed the shutdown and the error happening almost simultaneously: > > > at 1:27 Service shutdown (that was me, I think, shutting it down), > and > > then 1:27:03, Application Error, faulting application bbackupd.exe > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 19:00:42 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 11:00:42 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516174203.27399.qmail@web60621.mail.yahoo.com> Message-ID: <20060516180042.78799.qmail@web60620.mail.yahoo.com> Chris, I guess I should ask, is it worth me not having my clients backing up for a day or two (yikes!) to wait for us to work through the regex problem before I upgrade them? What new features/fixes are you/we trying to get? In my ignorance, the command line services tools seem fine to me the way they are, so those aren't worth me waiting, so in my situation, I'm inclined to just use the older 538 instead of 568 or later. Make sense? --- "E.W. Peter Jalajas" wrote: > Sorry, clicked the wrong button... > > Yes, I'd be honored to try to test them for you. I have 5 or 6 more > Windows clients to upgrade, so as I'm upgrading each one over the > next > day or two, I'll first try whatever your latest new one is and if > that > one doesn't work, I'll immediately fall back to this one that just > worked. After that, I can do a little experimenting on my home WinXP > box. Or maybe I can try to scavenge something from my digital > boneyard, but I'm not sure if anything is beefy enough to support my > old Win2K install disks--we'll see, I'll figure somethin' out. > > Sorry I can't be more useful on the crash. Let me see what I can do > to > re-create the problem when I get home... > > --- Chris Wilson wrote: > > > Hi Pete, > > > > > That worked perfectly, Chris, thanks! It's catching up on the > > backups > > > now. > > > > OK, that's good, it's just the last version that's borked. I'll > make > > the > > wiki point to the old one again, for now. > > > > If I make a new release, would you be able to test it for me > please, > > to > > verify that regexes are working properly? I'm making a few fixes to > > the > > command line handling for adding and removing services at the > moment. > > I > > can't test it easily because my local server is borked. > > > > > Re the 0.10 crash error, yes, I think it might have happened > right > > after > > > stopping the service, but I'm not absolutely sure. My > recollection > > was > > > that I thought it was odd that I was getting that message when I > > had > > > thought that I had stopped the service a while ago (whatever "a > > while" > > > means), like I might have said to myself, "I thought I stopped > > that? Was > > > that still running?" or something like that, but I was so busy > with > > > > > other stuff that I didn't track it down. > > > > Please could you have a look at your event viewer logs again. I > think > > they > > showed the shutdown and the error happening almost simultaneously: > > > > > at 1:27 Service shutdown (that was me, I think, shutting it > down), > > and > > > then 1:27:03, Application Error, faulting application > bbackupd.exe > > > > Cheers, Chris. > > -- > > _ ___ __ _ > > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 19:08:50 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 11:08:50 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516174203.27399.qmail@web60621.mail.yahoo.com> Message-ID: <20060516180850.91925.qmail@web60618.mail.yahoo.com> Chris, I guess I should ask, is it worth me not having my clients backing up for a day or two (yikes!) to wait for us to work through the regex problem before I upgrade them? What new features/fixes are you/we trying to get? In my ignorance, the command line services tools seem fine to me the way they are, so those aren't worth me waiting, so in my situation, I'm inclined to just use the older 538 instead of 568 or later. Make sense? --- "E.W. Peter Jalajas" wrote: > Sorry, clicked the wrong button... > > Yes, I'd be honored to try to test them for you. I have 5 or 6 more > Windows clients to upgrade, so as I'm upgrading each one over the > next > day or two, I'll first try whatever your latest new one is and if > that > one doesn't work, I'll immediately fall back to this one that just > worked. After that, I can do a little experimenting on my home WinXP > box. Or maybe I can try to scavenge something from my digital > boneyard, but I'm not sure if anything is beefy enough to support my > old Win2K install disks--we'll see, I'll figure somethin' out. > > Sorry I can't be more useful on the crash. Let me see what I can do > to > re-create the problem when I get home... > > --- Chris Wilson wrote: > > > Hi Pete, > > > > > That worked perfectly, Chris, thanks! It's catching up on the > > backups > > > now. > > > > OK, that's good, it's just the last version that's borked. I'll > make > > the > > wiki point to the old one again, for now. > > > > If I make a new release, would you be able to test it for me > please, > > to > > verify that regexes are working properly? I'm making a few fixes to > > the > > command line handling for adding and removing services at the > moment. > > I > > can't test it easily because my local server is borked. > > > > > Re the 0.10 crash error, yes, I think it might have happened > right > > after > > > stopping the service, but I'm not absolutely sure. My > recollection > > was > > > that I thought it was odd that I was getting that message when I > > had > > > thought that I had stopped the service a while ago (whatever "a > > while" > > > means), like I might have said to myself, "I thought I stopped > > that? Was > > > that still running?" or something like that, but I was so busy > with > > > > > other stuff that I didn't track it down. > > > > Please could you have a look at your event viewer logs again. I > think > > they > > showed the shutdown and the error happening almost simultaneously: > > > > > at 1:27 Service shutdown (that was me, I think, shutting it > down), > > and > > > then 1:27:03, Application Error, faulting application > bbackupd.exe > > > > Cheers, Chris. > > -- > > _ ___ __ _ > > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 19:47:59 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 16 May 2006 19:47:59 +0100 Subject: [Box Backup] How to move store to new machine In-Reply-To: References: <20060516163559.24669.qmail@web60624.mail.yahoo.com> Message-ID: <4DEDD492-45A2-4CED-9381-09385CE28121@fluffy.co.uk> On 16 May 2006, at 18:00, Chris Wilson wrote: > Hi Pete, > >> Sorry, I'm still not sure what you're getting at re the _bbstored =20 >> user. >> Can you paste the lines that you think are odd? > > OK, I missed the fact that you were running the commands on =20 > different servers. On the old server, _bbstored has uid 1003, and =20 > on the new one 1005. It shouldn't matter, but make sure that your =20 > restored tarball has the correct permissions to allow the _bbstored =20= > user on new to write to it. That's rather important! I missed that one. :-) > > It may not matter either if you're decommissioning the server, but =20 > this worries me: > >> user at old:~$ su _bbstored >> su: User not known to the underlying authentication module >> user at old:~$ grep bb /etc/passwd >> _bbstored:x:1003:100::/home/_bbstored: > > This implies that your PAM setup is broken on the old machine. I =20 > hope this works on the new machine? The long process to su to that user worried me, which was the "odd" =20 thing I was referring to. > > Did you already try fixing the store on the old machine before =20 > copying? If not, please try that. > >> (Do I recall reading somewhere that you start it as root, then =20 >> _bbstored takes over, or was that some other project that I'm =20 >> thinking about?) > > I'm not sure. There is a "user" option in the server config, but I =20 > haven't used it, so I don't know if it works. This is how it works=10. Start as root, and it changes user to the user =20= specified. However, this is optional, and you can start it as any =20 other user whatever way you want. > >> Sorry, a little off-topic, but your massive contribution to box =20 >> backup > > Thanks, but I hardly made a "massive contribution", especially in =20 > comparison to what the others have done. I just cleaned up Nick's =20 > port a bit, got it merged into the tree, and now I'm helping out =20 > with a bit of tech support when I have time. I second Martin's comment about you underestimating your contribution. > >> Re http://www.loband.org/loband/main, I don't really have a need =20 >> for it (yet), but it was interesting to see how fast it was, how =20 >> clean the output looked, and how it might be used for rich-country =20= >> limited bandwidth situations like PDAs, cell phones, dial-up, etc. > > Thanks, I'm glad you liked it. It's not really designed for mobile =20 > phones, but some people have reported using it successfully with =20 > them. And it worked pretty well on my Treo, before that had a heart =20= > attack one day. > > And it is sometimes useful in developing countries. Here in Ghana, =20 > Sourceforge pages take forever to load; it's about three times =20 > faster through Loband. Maybe I'm missing something, but you don't appear to be gzipping your =20= responses. I assume the computers at the other end are capable of =20 decompressing faster than receiving the extra data, and anyway, it is =20= a negotiated thing rather than an all or nothing. It could be as =20 simple as enabling mod_gzip. Ben From boxbackup at fluffy.co.uk Tue May 16 19:58:36 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 11:58:36 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516185836.10954.qmail@web60618.mail.yahoo.com> Hi Chris, Funny to see myself typing this because I really have no idea, but I think the _bbstored user and PAM are fine. He is odd because he is created without a password (as instructed), which I think is secure because no one can become him except by doing sudo su _bbstored (which is simple to do when needed). Re the permissions, I have no idea either except for the netcat tar deal I did. Maybe this confirms that I'm ok: drwxr-xr-x 2 _bbstored users 4096 Mar 1 16:44 0e drwxr-xr-x 2 _bbstored users 4096 May 16 12:55 0f The old directories has the same perms as the new ones. I'm a little nervous at the moment about so many new files being created. -rw-r--r-- 1 _bbstored users 135 Feb 9 12:14 o23.rfw -rw-r--r-- 1 _bbstored users 67015 May 16 12:49 o24.rfw -rw-r--r-- 1 _bbstored users 135 Feb 9 12:25 o28.rfw -rw-r--r-- 1 _bbstored users 58343 May 16 12:49 o29.rfw -rw-r--r-- 1 _bbstored users 135 Feb 9 12:35 o2d.rfw -rw-r--r-- 1 _bbstored users 38055 May 16 12:49 o2e.rfw -rw-r--r-- 1 _bbstored users 135 Feb 9 12:37 o30.rfw -rw-r--r-- 1 _bbstored users 9703 May 16 12:50 o31.rfw user at new:~$ date ; du -s /media/bu2/backup/ ; find /media/bu2/backup/ -type f | wc -l Tue May 16 13:26:05 EDT 2006 3360376 /media/bu2/backup/ 23788 user at new:~$ date ; du -s /media/bu2/backup/; find /media/bu2/backup/ -type f | wc -l Tue May 16 14:45:41 EDT 2006 3396856 /media/bu2/backup/ 31050 (this redaction/obfuscation thing I'm doing is a big pain in the neck waste of time, isn't it?) And I can't see any files that are dated after Friday... I'll have to wait and see because of the weekend and the flooding we had yesterday Monday, maybe they didn't create any files yesterday... Heck, maybe this all fixed something that was broken before. I didn't, but should have, run /usr/local/bin/bbstoreaccounts check fix on all my accounts before I did the tar netcat. That should be added to the wiki. (I think I have at least 3 wiki edits I need to make now.) user at new:~$ grep -i user /etc/box/bbstored.conf User = _bbstored which was set during the server config with /usr/local/bin/bbstored-config /etc/box hostname _bbstored I believe. http://boxbackup.hostworks.ca/index.php/Configuring_a_server Thanks again, Pete --- Chris Wilson wrote: > Hi Pete, > > > Sorry, I'm still not sure what you're getting at re the _bbstored > user. > > Can you paste the lines that you think are odd? > > OK, I missed the fact that you were running the commands on different > > servers. On the old server, _bbstored has uid 1003, and on the new > one > 1005. It shouldn't matter, but make sure that your restored tarball > has > the correct permissions to allow the _bbstored user on new to write > to it. > > It may not matter either if you're decommissioning the server, but > this > worries me: > > > user at old:~$ su _bbstored > > su: User not known to the underlying authentication module > > user at old:~$ grep bb /etc/passwd > > _bbstored:x:1003:100::/home/_bbstored: > > This implies that your PAM setup is broken on the old machine. I hope > this > works on the new machine? > > Did you already try fixing the store on the old machine before > copying? If > not, please try that. > > > (Do I recall reading somewhere that you start it as root, then > _bbstored > > takes over, or was that some other project that I'm thinking > about?) > > I'm not sure. There is a "user" option in the server config, but I > haven't > used it, so I don't know if it works. > From boxbackup at fluffy.co.uk Tue May 16 20:18:29 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 12:18:29 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <4DEDD492-45A2-4CED-9381-09385CE28121@fluffy.co.uk> Message-ID: <20060516191829.74480.qmail@web60623.mail.yahoo.com> --- Ben Summers wrote: > On 16 May 2006, at 18:00, Chris Wilson wrote: > > Hi Pete, > > > > OK, I missed the fact that you were running the commands on > > different servers. > > That's rather important! I missed that one. :-) How you guys could possibly have missed that among my thousands and thousands of lines of drivel is certainly understandable. I won't make the false promise to try to be more concise. > >> user at old:~$ su _bbstored > >> su: User not known to the underlying authentication module > >> user at old:~$ grep bb /etc/passwd > >> _bbstored:x:1003:100::/home/_bbstored: > > > The long process to su to that user worried me, which was the "odd" > thing I was referring to. The long odd process was just me going through the process of discovering simple basic command line skills, however slowly. It _should_ worry you that I have to do that so often. (Trust me, it's scarier to watch in person.) (The method behind my odd madness was that I showed my work, including the false starts, [and similarly why I draft such lengthy messages] is so that maybe other noobs out there would maybe tip towards using the product and maybe even chime in on the list.) From boxbackup at fluffy.co.uk Tue May 16 20:35:41 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 19:35:41 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516180042.78799.qmail@web60620.mail.yahoo.com> References: <20060516180042.78799.qmail@web60620.mail.yahoo.com> Message-ID: Hi Pete, > Chris, I guess I should ask, is it worth me not having my clients > backing up for a day or two (yikes!) to wait for us to work through the > regex problem before I upgrade them? What new features/fixes are you/we > trying to get? In my ignorance, the command line services tools seem > fine to me the way they are, so those aren't worth me waiting, so in my > situation, I'm inclined to just use the older 538 instead of 568 or > later. Make sense? Yeah, makes sense to me. You might as well run 538 on the clients if it works for you. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 20:44:54 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 19:44:54 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <4DEDD492-45A2-4CED-9381-09385CE28121@fluffy.co.uk> References: <20060516163559.24669.qmail@web60624.mail.yahoo.com> <4DEDD492-45A2-4CED-9381-09385CE28121@fluffy.co.uk> Message-ID: Hi Ben, On Tue, 16 May 2006, Ben Summers wrote: > I second Martin's comment about you underestimating your contribution. Thank you both, it's nice to be appreciated even when it's not deserved :-) > Maybe I'm missing something, but you don't appear to be gzipping your > responses. I assume the computers at the other end are capable of > decompressing faster than receiving the extra data, and anyway, it is a > negotiated thing rather than an all or nothing. It could be as simple as > enabling mod_gzip. Compression is definitely very useful, and Loband should have it enabled. I just checked loading yahoo.com through it, and the response headers say it was gzipped. I think you might have missed it because small pages (such as the front page) are not gzipped by default, and we haven't changed that option. You had me worried though, because in the past it stopped compressing and it took us a while to notice :-( Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 20:56:48 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 12:56:48 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516185836.10954.qmail@web60618.mail.yahoo.com> Message-ID: <20060516195648.55914.qmail@web60617.mail.yahoo.com> I'm still not sure if all the new file transfers was a good thing or a bad one, but this info may make sense to you guys. Here is the sudo bbstoreaccounts info report that I set up to get emailed from my old server to me every two hours: Account ID: Last object ID: 29249 Blocks used: 831017 (3246.16Mb) Blocks used by old files: 152442 (595.48Mb) Blocks used by deleted files: 22203 (86.73Mb) Blocks used by directories: 2050 (8.01Mb) Block soft limit: 896000 (3500.00Mb) Block hard limit: 1536000 (6000.00Mb) Client store marker: 1147615061000000 user at new:~$ sudo bbstoreaccounts info Account ID: Last object ID: 41765 Blocks used: 846946 (3308.38Mb) Blocks used by old files: 168183 (656.96Mb) Blocks used by deleted files: 22237 (86.86Mb) Blocks used by directories: 2254 (8.80Mb) Block soft limit: 896000 (3500.00Mb) Block hard limit: 1536000 (6000.00Mb) Client store marker: 1147803312000000 So it looks like the new setup (client and server) found many more files to backup, and so all my stored deleted files were deleted to make room. No why did I have so many new files? Either because regex exclusions is not working (and not reporting an error), or I'm picking up new directories or files somehow. With 192MB freeboard between Blocks Used and the soft limit, I think we are pretty sure that we're getting all the files that we're supposed to get. I need to make sure that I get a new version of their 180MB quickbooks file tonight. I'm guardedly optimistic that we're ok. On to the next client...maybe. (For the search engine record,) I recommend making the hard limit about at least twice as large as your largest file (check your quickbooks files, Outlook and Outlook Express mailboxes, databases). I have several clients with GB+ mailboxes. So, you need that space between soft and hard limits for those files to come in, be handled correctly, and then old files are deleted. Do NOT do what I originally did and make the hard limit arbitrarily 25% or 50% larger than the soft limit; the gap needs to be a multiple of at least one or two times your largest expected file for that client account. Depending upon your specific conditions, don't be afraid to have a total sum of hard limits across all of your accounts which add up to much more than your server can hold because that storage (the amount between the soft and hard limits) is very rarely used for any length of time (kinda like overbooking an airplane, the chances of everyone showing up at the same time is quite small, and I think in this case, I'm hoping the box backup will simply back off and try to upload the second large file later so that everyone will get on board soon enough). --- "E.W. Peter Jalajas" wrote: > Hi Chris, > > Funny to see myself typing this because I really have no idea, but I > think the _bbstored user and PAM are fine. He is odd because he is > created without a password (as instructed), which I think is secure > because no one can become him except by doing sudo su _bbstored > (which > is simple to do when needed). > > Re the permissions, I have no idea either except for the netcat tar > deal I did. Maybe this confirms that I'm ok: > drwxr-xr-x 2 _bbstored users 4096 Mar 1 16:44 0e > drwxr-xr-x 2 _bbstored users 4096 May 16 12:55 0f > The old directories has the same perms as the new ones. > > I'm a little nervous at the moment about so many new files being > created. > -rw-r--r-- 1 _bbstored users 135 Feb 9 12:14 o23.rfw > -rw-r--r-- 1 _bbstored users 67015 May 16 12:49 o24.rfw > -rw-r--r-- 1 _bbstored users 135 Feb 9 12:25 o28.rfw > -rw-r--r-- 1 _bbstored users 58343 May 16 12:49 o29.rfw > -rw-r--r-- 1 _bbstored users 135 Feb 9 12:35 o2d.rfw > -rw-r--r-- 1 _bbstored users 38055 May 16 12:49 o2e.rfw > -rw-r--r-- 1 _bbstored users 135 Feb 9 12:37 o30.rfw > -rw-r--r-- 1 _bbstored users 9703 May 16 12:50 o31.rfw > > user at new:~$ date ; du -s /media/bu2/backup/ ; find > /media/bu2/backup/ -type f | wc -l > Tue May 16 13:26:05 EDT 2006 > 3360376 /media/bu2/backup/ > 23788 > user at new:~$ date ; du -s /media/bu2/backup/; find > /media/bu2/backup/ -type f | wc -l > Tue May 16 14:45:41 EDT 2006 > 3396856 /media/bu2/backup/ > 31050 > (this redaction/obfuscation thing I'm doing is a big pain in the neck > waste of time, isn't it?) > > And I can't see any files that are dated after Friday... I'll have to > wait and see because of the weekend and the flooding we had yesterday > Monday, maybe they didn't create any files yesterday... Heck, maybe > this all fixed something that was broken before. > > I didn't, but should have, run > /usr/local/bin/bbstoreaccounts check fix > on all my accounts before I did the tar netcat. That should be added > to the wiki. (I think I have at least 3 wiki edits I need to make > now.) > > user at new:~$ grep -i user /etc/box/bbstored.conf > User = _bbstored > which was set during the server config with > /usr/local/bin/bbstored-config /etc/box hostname _bbstored > I believe. > http://boxbackup.hostworks.ca/index.php/Configuring_a_server > > Thanks again, > Pete > > --- Chris Wilson wrote: > > > Hi Pete, > > > > > Sorry, I'm still not sure what you're getting at re the _bbstored > > user. > > > Can you paste the lines that you think are odd? > > > > OK, I missed the fact that you were running the commands on > different > > > > servers. On the old server, _bbstored has uid 1003, and on the new > > one > > 1005. It shouldn't matter, but make sure that your restored tarball > > has > > the correct permissions to allow the _bbstored user on new to write > > to it. > > > > It may not matter either if you're decommissioning the server, but > > this > > worries me: > > > > > user at old:~$ su _bbstored > > > su: User not known to the underlying authentication module > > > user at old:~$ grep bb /etc/passwd > > > _bbstored:x:1003:100::/home/_bbstored: > > > > This implies that your PAM setup is broken on the old machine. I > hope > > this > > works on the new machine? > > > > Did you already try fixing the store on the old machine before > > copying? If > > not, please try that. > > > > > (Do I recall reading somewhere that you start it as root, then > > _bbstored > > > takes over, or was that some other project that I'm thinking > > about?) > > > > I'm not sure. There is a "user" option in the server config, but I > > haven't > > used it, so I don't know if it works. > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue May 16 21:02:20 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 13:02:20 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516195648.55914.qmail@web60617.mail.yahoo.com> Message-ID: <20060516200220.56330.qmail@web60622.mail.yahoo.com> Sorry, I'm an idiot, I mis-read my "deleted files" numbers. --- "E.W. Peter Jalajas" wrote: > Blocks used: 831017 (3246.16Mb) > Blocks used by old files: 152442 (595.48Mb) > Blocks used by deleted files: 22203 (86.73Mb) > > Blocks used: 846946 (3308.38Mb) > Blocks used by old files: 168183 (656.96Mb) > Blocks used by deleted files: 22237 (86.86Mb) > So ignore this part: > So it looks like the new setup (client and server) found many more > files to backup, and so all my stored deleted files were deleted to > make room. From boxbackup at fluffy.co.uk Tue May 16 21:13:13 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 20:13:13 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516185836.10954.qmail@web60618.mail.yahoo.com> References: <20060516185836.10954.qmail@web60618.mail.yahoo.com> Message-ID: Hi Pete, > Re the permissions, I have no idea either except for the netcat tar > deal I did. Maybe this confirms that I'm ok: > drwxr-xr-x 2 _bbstored users 4096 Mar 1 16:44 0e > drwxr-xr-x 2 _bbstored users 4096 May 16 12:55 0f > The old directories has the same perms as the new ones. If those are the permissions on the new server, and as long as the _bbstored user has uid 1005, then it's fine. > (this redaction/obfuscation thing I'm doing is a big pain in the neck > waste of time, isn't it?) Yeah, I guess :-) I don't care if anyone knows my laptop or server or user account names. I don't think it makes it any easier to hack them. > And I can't see any files that are dated after Friday... On the old server? (I guess the newly created files are dated after Friday) > I didn't, but should have, run /usr/local/bin/bbstoreaccounts check > fix on all my accounts before I did the tar netcat. You should really run that on all accounts on the new server as soon as possible. > That should be added to the wiki. (I think I have at least 3 wiki edits > I need to make now.) Ah, sweet procrastination :-) Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 16 22:01:47 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 14:01:47 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060516210147.19262.qmail@web60619.mail.yahoo.com> Ok, ok, I'll stop redacting my computer names, paths, and account names. But I'll try to still redact my user names. Brick is the new store server. Account 00000014 is the first account that I put on the Windows client. user at brick:~$ date Tue May 16 16:18:06 EDT 2006 user at brick:~$ sudo bbstoreaccounts check 00000014 Check store account ID 00000014 Phase 1, check objects... Phase 2, check directories... Phase 3, check root... Phase 4, fix unattached objects... Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... Store account checked, no errors found. user at brick:~$ sudo bbstoreaccounts check 00000014 fix # just in case... NOTE: Will fix errors encountered during checking. Check store account ID 00000014 Phase 1, check objects... Phase 2, check directories... Phase 3, check root... Phase 4, fix unattached objects... Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... Store account checked, no errors found. user at brick:~$ id _bbstored uid=1005(_bbstored) gid=100(users) groups=100(users) I finally broke down and installed cygwin on the client (MS Server 2003), which adds great command line power to bbackupquery, like: ./bbackupquery "list -odtsr" quit | grep -e 2006-05-15 # Monday 00007242 f-X--- 2006-05-15T00:09:54 00001 DDataPublic/Timesheets/pending/Rolands 5-14.iif 00009342 f----- 2006-05-15T09:55:54 00001 DHome//Flood Punchlist.doc ... So, I still think we're ok... You know I'll let you know if I find out anything different... Someday soon I'll script up a command line to confirm the regex is working. Right after I update my clients...and update the wiki. Thanks again, Chris, Pete --- Chris Wilson wrote: > Hi Pete, > > > Re the permissions, I have no idea either except for the netcat tar > > deal I did. Maybe this confirms that I'm ok: > > drwxr-xr-x 2 _bbstored users 4096 Mar 1 16:44 0e > > drwxr-xr-x 2 _bbstored users 4096 May 16 12:55 0f > > The old directories has the same perms as the new ones. > > If those are the permissions on the new server, and as long as the > _bbstored user has uid 1005, then it's fine. > > > (this redaction/obfuscation thing I'm doing is a big pain in the > neck > > waste of time, isn't it?) > > Yeah, I guess :-) I don't care if anyone knows my laptop or server or > > user account names. I don't think it makes it any easier to hack > them. > > > And I can't see any files that are dated after Friday... > > On the old server? (I guess the newly created files are dated after > Friday) > > > I didn't, but should have, run /usr/local/bin/bbstoreaccounts check > > > fix on all my accounts before I did the tar netcat. > > You should really run that on all accounts on the new server as soon > as > possible. > > > That should be added to the wiki. (I think I have at least 3 wiki > edits > > I need to make now.) > > Ah, sweet procrastination :-) > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed May 17 00:41:55 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 16 May 2006 23:41:55 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516180850.91925.qmail@web60618.mail.yahoo.com> References: <20060516180850.91925.qmail@web60618.mail.yahoo.com> Message-ID: Hi Pete, > Chris, I guess I should ask, is it worth me not having my clients > backing up for a day or two (yikes!) to wait for us to work through the > regex problem before I upgrade them? What new features/fixes are you/we > trying to get? In my ignorance, the command line services tools seem > fine to me the way they are, so those aren't worth me waiting, so in my > situation, I'm inclined to just use the older 538 instead of 568 or > later. I've just uploaded a new client, revision 568. If you have time, please could you give it a try? [http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/boxbackup-chris_general_568-backup-client-mingw32.zip] Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 17 03:47:44 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 16 May 2006 19:47:44 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: Message-ID: <20060517024744.42952.qmail@web60624.mail.yahoo.com> Hi Chris, It worked perfectly on Mom's WinXP Home. (568 is the right number, right?) Thanks for all your hard work on it! Pete --- Chris Wilson wrote: > Hi Pete, > I'm inclined to just use the older 538 instead of 568 or > > later. > > I've just uploaded a new client, revision 568. If you have time, > please > could you give it a try? > > [http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/boxbackup-chris_general_568-backup-client-mingw32.zip] > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed May 17 10:18:32 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 17 May 2006 09:18:32 +0000 (GMT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060517024744.42952.qmail@web60624.mail.yahoo.com> References: <20060517024744.42952.qmail@web60624.mail.yahoo.com> Message-ID: Hi Pete, > It worked perfectly on Mom's WinXP Home. (568 is the right number, > right?) Yes, the new one is 568. Thanks for testing! Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 17 16:54:00 2006 From: boxbackup at fluffy.co.uk (Roy) Date: Wed, 17 May 2006 17:54:00 +0200 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi Message-ID: <446B4718.1040009@hostingbrothers.nl> Hi all, After a while I'm again testing the 0.10 server & client and the latest MinGW Windows version from Chris. But I'v got a couple of questions about the certificates. If you lose your certificates, but still have your -FileEncKeys.raw file. How can you regenerate your certificates for the client? If you can do that, I suppose you can only do that on linux at the moment? And what do you need from the server? Which certificates and how can you regenerate the others? Or do you have to completely backup the CA? And a question to Chris; what's the current status of the rewrite of Boxi? Or didn't you have the time to work on it in Ghana yet? Thanks in advance, Roy From boxbackup at fluffy.co.uk Wed May 17 17:38:42 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 17 May 2006 16:38:42 +0000 (GMT) Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: <446B4718.1040009@hostingbrothers.nl> References: <446B4718.1040009@hostingbrothers.nl> Message-ID: Hi Roy, > After a while I'm again testing the 0.10 server & client and the latest > MinGW Windows version from Chris. But I'v got a couple of questions > about the certificates. Please make sure you're not using version 564 of the Windows client. If it's not inconvenient, please could you test the new release, version 568? > If you lose your certificates, but still have your > -FileEncKeys.raw file. How can you regenerate your > certificates for the client? If you can do that, I suppose you can only > do that on linux at the moment? Yes, you can regenerate anything except the file encryption keys. However, it does require help from the server operator. The client either resubmits their original certificate request, if they still have that and the private key. Otherwise, they generate a new key (optional) and a new certificate request. The server operator signs the certificate request and returns the resulting certificate to the client. > And what do you need from the server? Which certificates and how can you > regenerate the others? Or do you have to completely backup the CA? I would definitely recommend that you back up the entire CA, although technically the server and client keys and root certificates should be enough. > And a question to Chris; what's the current status of the rewrite of > Boxi? Or didn't you have the time to work on it in Ghana yet? It's still progressing slowly, even while I'm here. I'm writing unit tests at the moment, and fixing the bugs that I find. I think it's currently usable to make backups, but not for restoring. It will be a while before the next release, sorry. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 17 18:15:39 2006 From: boxbackup at fluffy.co.uk (Roy) Date: Wed, 17 May 2006 19:15:39 +0200 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: References: <446B4718.1040009@hostingbrothers.nl> Message-ID: <446B5A3B.1020108@hostingbrothers.nl> Chris Wilson wrote: > Hi Roy, > >> After a while I'm again testing the 0.10 server & client and the >> latest MinGW Windows version from Chris. But I'v got a couple of >> questions about the certificates. > > Please make sure you're not using version 564 of the Windows client. > If it's not inconvenient, please could you test the new release, > version 568? > Yes I saw that on the dev-list so I'm now running 568. >> If you lose your certificates, but still have your >> -FileEncKeys.raw file. How can you regenerate your >> certificates for the client? If you can do that, I suppose you can >> only do that on linux at the moment? > > Yes, you can regenerate anything except the file encryption keys. > However, it does require help from the server operator. The client > either resubmits their original certificate request, if they still > have that and the private key. Otherwise, they generate a new key > (optional) and a new certificate request. > > The server operator signs the certificate request and returns the > resulting certificate to the client. Mmm I'm the server operator, but I don't know how the regenerate them. I used the script that come with Box Backup, and for all I know, you can't use the raw key file as source. With a new key, you lose your store or not? > >> And what do you need from the server? Which certificates and how can >> you regenerate the others? Or do you have to completely backup the CA? > > I would definitely recommend that you back up the entire CA, although > technically the server and client keys and root certificates should be > enough. > >> And a question to Chris; what's the current status of the rewrite of >> Boxi? Or didn't you have the time to work on it in Ghana yet? > > It's still progressing slowly, even while I'm here. I'm writing unit > tests at the moment, and fixing the bugs that I find. I think it's > currently usable to make backups, but not for restoring. It will be a > while before the next release, sorry. Waiting for it anxiously. Because that is especially handy for people that aren't used to command line programs. > > Cheers, Chris. Cheers, Roy From boxbackup at fluffy.co.uk Wed May 17 21:52:08 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 17 May 2006 21:52:08 +0100 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: <446B5A3B.1020108@hostingbrothers.nl> References: <446B4718.1040009@hostingbrothers.nl> <446B5A3B.1020108@hostingbrothers.nl> Message-ID: <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> On 17 May 2006, at 18:15, Roy wrote: >>> If you lose your certificates, but still have your - >>> FileEncKeys.raw file. How can you regenerate your certificates >>> for the client? If you can do that, I suppose you can only do >>> that on linux at the moment? >> >> Yes, you can regenerate anything except the file encryption keys. >> However, it does require help from the server operator. The client >> either resubmits their original certificate request, if they still >> have that and the private key. Otherwise, they generate a new key >> (optional) and a new certificate request. >> >> The server operator signs the certificate request and returns the >> resulting certificate to the client. > Mmm I'm the server operator, but I don't know how the regenerate > them. I used the script that come with Box Backup, and for all I > know, you can't use the raw key file as source. To regenerate client keys, run bbackupd-config again, and replace it's raw key file with the one from your secure backup. Sign the new certificates. Then you're up and running again. Similarly for a server. > With a new key, you lose your store or not? Yes. Which is why bbackupd-config urges you to keep a copy of the .raw file in a safe place. Ben From boxbackup at fluffy.co.uk Wed May 17 22:15:08 2006 From: boxbackup at fluffy.co.uk (Roy) Date: Wed, 17 May 2006 23:15:08 +0200 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> References: <446B4718.1040009@hostingbrothers.nl> <446B5A3B.1020108@hostingbrothers.nl> <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> Message-ID: <446B925C.1040606@hostingbrothers.nl> Ben Summers wrote: > > On 17 May 2006, at 18:15, Roy wrote: >>>> If you lose your certificates, but still have your >>>> -FileEncKeys.raw file. How can you regenerate your >>>> certificates for the client? If you can do that, I suppose you can >>>> only do that on linux at the moment? >>> >>> Yes, you can regenerate anything except the file encryption keys. >>> However, it does require help from the server operator. The client >>> either resubmits their original certificate request, if they still >>> have that and the private key. Otherwise, they generate a new key >>> (optional) and a new certificate request. >>> >>> The server operator signs the certificate request and returns the >>> resulting certificate to the client. >> Mmm I'm the server operator, but I don't know how the regenerate >> them. I used the script that come with Box Backup, and for all I >> know, you can't use the raw key file as source. > > To regenerate client keys, run bbackupd-config again, and replace it's > raw key file with the one from your secure backup. Sign the new > certificates. Then you're up and running again. > > Similarly for a server. > > >> With a new key, you lose your store or not? > > Yes. Which is why bbackupd-config urges you to keep a copy of the .raw > file in a safe place. > > Ben > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup Thanks for this explanation. Maybe this should be put somewhere on the Wiki or somewhere else in the documentation. Or is it already there and I didn't see it? Cheers, Roy From boxbackup at fluffy.co.uk Wed May 17 22:29:12 2006 From: boxbackup at fluffy.co.uk (Per Thomsen) Date: Wed, 17 May 2006 14:29:12 -0700 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: <446B925C.1040606@hostingbrothers.nl> References: <446B4718.1040009@hostingbrothers.nl> <446B5A3B.1020108@hostingbrothers.nl> <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> <446B925C.1040606@hostingbrothers.nl> Message-ID: <446B95A8.3010909@reedtz.com> On 5/17/06 2:15 PM, Roy wrote: > Ben Summers wrote: >> >> On 17 May 2006, at 18:15, Roy wrote: >>>>> If you lose your certificates, but still have your >>>>> -FileEncKeys.raw file. How can you regenerate your >>>>> certificates for the client? If you can do that, I suppose you can >>>>> only do that on linux at the moment? >>>> >>>> Yes, you can regenerate anything except the file encryption keys. >>>> However, it does require help from the server operator. The client >>>> either resubmits their original certificate request, if they still >>>> have that and the private key. Otherwise, they generate a new key >>>> (optional) and a new certificate request. >>>> >>>> The server operator signs the certificate request and returns the >>>> resulting certificate to the client. >>> Mmm I'm the server operator, but I don't know how the regenerate >>> them. I used the script that come with Box Backup, and for all I >>> know, you can't use the raw key file as source. >> >> To regenerate client keys, run bbackupd-config again, and replace >> it's raw key file with the one from your secure backup. Sign the new >> certificates. Then you're up and running again. >> >> Similarly for a server. >> >> >>> With a new key, you lose your store or not? >> >> Yes. Which is why bbackupd-config urges you to keep a copy of the >> .raw file in a safe place. >> > Thanks for this explanation. Maybe this should be put somewhere on the > Wiki or somewhere else in the documentation. Or is it already there > and I didn't see it? I don't think it's there. I will add it to the list of things for the documentation project. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Thu May 18 01:28:19 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 00:28:19 +0000 (GMT) Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> References: <446B4718.1040009@hostingbrothers.nl> <446B5A3B.1020108@hostingbrothers.nl> <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> Message-ID: Hi Roy, On 17 May 2006, at 18:15, Roy wrote: > With a new key, you lose your store or not? Ben Summers wrote: > Yes. Which is why bbackupd-config urges you to keep a copy of the .raw > file in a safe place. Just to avoid any confusion: there are actually two "keys" and it's important to know which one you're talking about. The "Private Key" goes with the certificate and is only used to authenticate yourself to the server. If you lose it, you can generate another one. The encryption key, named *-FileEncKeys.raw, is the one that's essential to recover your backups, and not replaceable. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 09:36:50 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Thu, 18 May 2006 11:36:50 +0300 Subject: [Box Backup] How can I delete files/directories from the store? Message-ID: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> Hello, Can please somebody tell me how can I delete files from the store using the bbackupquery ? For example I ran out of space and I want to remove something so I could add something else. Thanks From boxbackup at fluffy.co.uk Thu May 18 09:45:04 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 09:45:04 +0100 Subject: [Box Backup] How can I delete files/directories from the store? In-Reply-To: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> Message-ID: <16A28A0F-70BA-487C-9815-6BDB1184EEBD@fluffy.co.uk> On 18 May 2006, at 09:36, o omida parasita wrote: > Hello, > Can please somebody tell me how can I delete files from the store > using the bbackupquery ? > For example I ran out of space and I want to remove something so I > could add something else. Tell bbackupd not to back up those files (see comments in bbackupd.conf), restart bbackupd, and wait for the system to take care of it. Box Backup is designed to be "automatic" -- user intervention in the process is not necessary or encouraged. Just specify what you want to be backed up, and it will happen. In normal use, you should expect to find the space used on the server at the soft limit most of the time. Ben From boxbackup at fluffy.co.uk Thu May 18 09:45:39 2006 From: boxbackup at fluffy.co.uk (Roy) Date: Thu, 18 May 2006 10:45:39 +0200 Subject: [Box Backup] How to regenerate certificate and what's the status of Boxi In-Reply-To: References: <446B4718.1040009@hostingbrothers.nl> <446B5A3B.1020108@hostingbrothers.nl> <190B9913-1608-496E-BB4B-162A872ADFFB@fluffy.co.uk> Message-ID: <446C3433.7010802@hostingbrothers.nl> Chris Wilson wrote: > Hi Roy, > > On 17 May 2006, at 18:15, Roy wrote: >> With a new key, you lose your store or not? > > Ben Summers wrote: >> Yes. Which is why bbackupd-config urges you to keep a copy of the >> .raw file in a safe place. > > Just to avoid any confusion: there are actually two "keys" and it's > important to know which one you're talking about. > > The "Private Key" goes with the certificate and is only used to > authenticate yourself to the server. If you lose it, you can generate > another one. > > The encryption key, named *-FileEncKeys.raw, is the one that's > essential to recover your backups, and not replaceable. > > Cheers, Chris. Thanks for this clear explanation. Now I know exactly what to do if something is going wrong. Maybe this could be added to the doc's as well, Per. Greetings, Roy From boxbackup at fluffy.co.uk Thu May 18 09:51:40 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Thu, 18 May 2006 11:51:40 +0300 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> Message-ID: <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> > Tell bbackupd not to back up those files (see comments in bbackupd.conf), restart bbackupd, and wait for the system to take care of it. How much do I need to wait? I removed the location from bbackupd.conf 3 days ago and it was not deleted. Thanks From boxbackup at fluffy.co.uk Thu May 18 10:01:08 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 10:01:08 +0100 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> Message-ID: <8F28E758-2344-4AEF-A171-B864A8F17B56@fluffy.co.uk> On 18 May 2006, at 09:51, o omida parasita wrote: >> Tell bbackupd not to back up those files (see comments in > bbackupd.conf), restart bbackupd, and wait for the system to take > care of it. > > How much do I need to wait? I removed the location from bbackupd.conf > 3 days ago and it was not deleted. This is a slightly different question. You're not asking how to delete files, you're asking how to delete a location. When you remove a backup location, you will see a message in the logs saying it will delete it after two days or so. This counter is reset every time you start or restart bbackupd. Have you restarted bbackupd in the meantime in those three days? Also, files are only actually physically deleted from the server when the space is needed. Being a backup system, it tries to not delete data unless it has to. So, check to see if they're marked for deletion. Run the list command, see if it's got an 'X' in the flags column. The chances are it's marked, and bits of it are being removed as new data is uploaded. Or are you having a problem you're trying to resolve, and not telling us the full story? Ben From boxbackup at fluffy.co.uk Thu May 18 10:13:15 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Thu, 18 May 2006 12:13:15 +0300 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> Message-ID: <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> > This is a slightly different question. You're not asking how to delete files, you're asking how to delete a location. > > When you remove a backup location, you will see a message in the logs saying it will delete it after two days or so. This counter is reset every time you start or restart bbackupd. Have you restarted bbackupd in the meantime in those three days? > > Also, files are only actually physically deleted from the server when the space is needed. Being a backup system, it tries to not delete data unless it has to. > > So, check to see if they're marked for deletion. Run the list command, see if it's got an 'X' in the flags column. The chances are it's marked, and bits of it are being removed as new data is uploaded. > > Or are you having a problem you're trying to resolve, and not telling us the full story? Actually I restarted the client in this 3 days period so probably I will have to wait a little more and see what will happen. Here is a concern that I have. Imagine the following situation: Used 32738.2Mb 99% *********************************** Old files 122.1Mb 0% Deleted files 1182.5Mb 2% * Directories 195.3Mb 0% Soft limit 40960.0Mb 90% ************************************ Hard limit 45056.0Mb 100% **************************************** Suddenly I realise that I need 5GB NOW. I have in deleted and old files only 1.3GB. I can't wait 2 days so the removed locations from bbackuped.conf are deleted. What do I do in this situation? Isn't there a command to manually remove a location/directory/file? Thanks From boxbackup at fluffy.co.uk Thu May 18 10:19:26 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 10:19:26 +0100 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> Message-ID: <63E93562-FF7E-44FC-A4F3-FA38041939A6@fluffy.co.uk> On 18 May 2006, at 10:13, o omida parasita wrote: >> This is a slightly different question. You're not asking how to > delete files, you're asking how to delete a location. >> >> When you remove a backup location, you will see a message in the logs > saying it will delete it after two days or so. This counter is reset > every time you start or restart bbackupd. Have you restarted bbackupd > in the meantime in those three days? >> >> Also, files are only actually physically deleted from the server when > the space is needed. Being a backup system, it tries to not delete > data unless it has to. >> >> So, check to see if they're marked for deletion. Run the list > command, see if it's got an 'X' in the flags column. The chances are > it's marked, and bits of it are being removed as new data is uploaded. >> >> Or are you having a problem you're trying to resolve, and not telling > us the full story? > > Actually I restarted the client in this 3 days period so probably I > will have to wait a little more and see what will happen. This is why the location hasn't been removed. This is a safety measure to protect against mistakes in configuration. > > Here is a concern that I have. > Imagine the following situation: > Used 32738.2Mb 99% *********************************** > Old files 122.1Mb 0% > Deleted files 1182.5Mb 2% * > Directories 195.3Mb 0% > Soft limit 40960.0Mb 90% ************************************ > Hard limit 45056.0Mb 100% > **************************************** > > Suddenly I realise that I need 5GB NOW. I have in deleted and old > files only 1.3GB. I can't wait 2 days so the removed locations from > bbackuped.conf are deleted. What do I do in this situation? Increase the soft and hard limits by 5Gb. > Isn't > there a command to manually remove a location/directory/file? No. If there were, bbackupd would just upload it again. What you could do is put the Location back in bbackupd.conf, but add directives to exclude all files within it. Then they'd get marked as deleted immediately. Ben From boxbackup at fluffy.co.uk Thu May 18 10:26:27 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Thu, 18 May 2006 12:26:27 +0300 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> Message-ID: <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> > What you could do is put the Location back in bbackupd.conf, but add > directives to exclude all files within it. Then they'd get marked as > deleted immediately. Thats actually a good idea. Thanks. Can I ask you something? Could you please explain me what are these: soft limit and hard limit? if soft limit is 40GB and hard is 45GB how much space actually can be stored? 85GB? Thanks From boxbackup at fluffy.co.uk Thu May 18 10:36:18 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 10:36:18 +0100 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> Message-ID: On 18 May 2006, at 10:26, o omida parasita wrote: >> What you could do is put the Location back in bbackupd.conf, but add >> directives to exclude all files within it. Then they'd get marked as >> deleted immediately. > > Thats actually a good idea. Thanks. > > Can I ask you something? > Could you please explain me what are these: soft limit and hard limit? > if soft limit is 40GB and hard is 45GB how much space actually can be > stored? 85GB? This is in the documentation, although maybe it's not highlighted sufficiently. In the soft 40Gb hard 45Gb case, you will find the account hovers around the 40Gb usage level ('voluntary' by the client), but will not exceed 45Gb (enforced by the server). Ben From boxbackup at fluffy.co.uk Thu May 18 10:42:05 2006 From: boxbackup at fluffy.co.uk (o omida parasita) Date: Thu, 18 May 2006 12:42:05 +0300 Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> Message-ID: <948a6d890605180242t5ade5bfdu92e9d9c53db64748@mail.gmail.com> Thanks a lot for your help. Box Backup is just great. Is there a way to help/contribute to the project? From boxbackup at fluffy.co.uk Thu May 18 11:09:52 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 10:09:52 +0000 (GMT) Subject: [Box Backup] Re: How can I delete files/directories from the store? In-Reply-To: <948a6d890605180242t5ade5bfdu92e9d9c53db64748@mail.gmail.com> References: <948a6d890605180136r28304aabx74f2a6ebf433480f@mail.gmail.com> <948a6d890605180151u6e6e4dc1k7e4d6371aa950931@mail.gmail.com> <948a6d890605180213h114137b7g750859e1eeb760f7@mail.gmail.com> <948a6d890605180226r140fd2c6o7c3fd88d6cd74248@mail.gmail.com> <948a6d890605180242t5ade5bfdu92e9d9c53db64748@mail.gmail.com> Message-ID: Hi, > Thanks a lot for your help. > Box Backup is just great. > Is there a way to help/contribute to the project? Yes, definitely! This is open source, and volunteers are always welcome. It would help if you could tell us what to call you, Mr. O. :-) After that, please have a look at the feature requests [http://boxbackup.hostworks.ca/index.php/Feature_requests] and Planned Work [http://boxbackup.hostworks.ca/index.php/Planned_work], and see if any of those interest you. If not, please add feature requests that do interest you. Once you have some features that you are interested in, please write a brief specification of what needs to be done, and how. If you are a developer, or can read C++, please try to go as far as possible in the direction of saying what code needs to be changed and where, or even supplying a patch that implements the feature. It may not be accepted straight away, it may not be accepted at all, but you don't know until you try, and you will definitely be helping the project in any case. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 11:44:42 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Thu, 18 May 2006 03:44:42 -0700 (PDT) Subject: [Box Backup] configure: error: OpenSSL is not installed? Message-ID: <20060518104442.51077.qmail@web30403.mail.mud.yahoo.com> Hi, On ./configure I get errors: checking openssl/ssl.h usability... no checking openssl/ssl.h presence... no checking for openssl/ssl.h... no checking for SSL_read in -lssl... no configure: error: OpenSSL is not installed but is required The sys is stable Debian sarge 2.4.27-2-386 I got OpenSSL 0.9.7e 25 Oct 2004 in my path I got the default /etc/ssl/openssl.cnf The OpenSSL notes did not help me, What am I missing??? Thnx, Jeff __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Thu May 18 11:50:06 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Thu, 18 May 2006 11:50:06 +0100 Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <20060518104442.51077.qmail@web30403.mail.mud.yahoo.com> References: <20060518104442.51077.qmail@web30403.mail.mud.yahoo.com> Message-ID: <20060518115006.5pl33oe52oookcg0@ebourne.me.uk> vordoo wrote: > On ./configure I get errors: > > checking openssl/ssl.h usability... no > checking openssl/ssl.h presence... no > checking for openssl/ssl.h... no > checking for SSL_read in -lssl... no > configure: error: OpenSSL is not installed but is > required > > The sys is stable Debian sarge 2.4.27-2-386 > I got OpenSSL 0.9.7e 25 Oct 2004 in my path > I got the default /etc/ssl/openssl.cnf PATH won't help you. What directory is openssl installed in? Are the headers and libraries not in the main include/lib dirs? If not, then give the appropriate --with option to say where it is. Cheers, Martin. From boxbackup at fluffy.co.uk Thu May 18 12:08:02 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Thu, 18 May 2006 04:08:02 -0700 (PDT) Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <20060518115006.5pl33oe52oookcg0@ebourne.me.uk> Message-ID: <20060518110803.95642.qmail@web30414.mail.mud.yahoo.com> Thnx, Martin! >What directory is openssl installed in? /usr/bin/openssl > headers and libraries not in the main include/lib > dirs? How do I check it out?? >If not, then give the appropriate --with option to say >where it is. How do Ido that?? __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Thu May 18 12:13:06 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 12:13:06 +0100 Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <20060518110803.95642.qmail@web30414.mail.mud.yahoo.com> References: <20060518110803.95642.qmail@web30414.mail.mud.yahoo.com> Message-ID: <7EB05EC9-B076-4DBC-99BB-313A2A5703ED@fluffy.co.uk> On 18 May 2006, at 12:08, vordoo wrote: > Thnx, Martin! > >> What directory is openssl installed in? > /usr/bin/openssl > >> headers and libraries not in the main include/lib >> dirs? > How do I check it out?? > >> If not, then give the appropriate --with option to > say >where it is. > How do Ido that?? Have you read the LINUX.txt file in the boxbackup-0.10 directory? Have you used Google to find out about how to use configure commands? Ben From boxbackup at fluffy.co.uk Thu May 18 12:59:35 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Thu, 18 May 2006 04:59:35 -0700 (PDT) Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <7EB05EC9-B076-4DBC-99BB-313A2A5703ED@fluffy.co.uk> Message-ID: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> Sorry Ben, I'm a newbie to headers and libraries, I usualy apt-get install. I do not know how to find the location of ssl headers and libraries Box Backup needs, on my debian sys. Aftet reading, OpenSSL notes, LINUX.txt and googling I have failed, aperantly pointing to the wrong place or using the wrong switch, I decided to just ask. Sorry, Jeff --- Ben Summers wrote: > > On 18 May 2006, at 12:08, vordoo wrote: > > > Thnx, Martin! > > > >> What directory is openssl installed in? > > /usr/bin/openssl > > > >> headers and libraries not in the main include/lib > >> dirs? > > How do I check it out?? > > > >> If not, then give the appropriate --with option > to > > say >where it is. > > How do Ido that?? > > Have you read the LINUX.txt file in the > boxbackup-0.10 directory? > > Have you used Google to find out about how to use > configure commands? > > Ben > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Thu May 18 13:03:26 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 18 May 2006 13:03:26 +0100 Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> References: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> Message-ID: <14869448-298A-4F02-AF5B-4AE3E754FCC8@fluffy.co.uk> Did you install the -dev packages as suggested by LINUX.txt? Ben On 18 May 2006, at 12:59, vordoo wrote: > Sorry Ben, > > I'm a newbie to headers and libraries, I usualy > apt-get install. > > I do not know how to find the location of ssl headers > and libraries Box Backup needs, on my debian sys. > Aftet reading, OpenSSL notes, LINUX.txt and googling I > have failed, aperantly pointing to the wrong place or > using the wrong switch, I decided to just ask. > > Sorry, > Jeff > > --- Ben Summers wrote: > >> >> On 18 May 2006, at 12:08, vordoo wrote: >> >>> Thnx, Martin! >>> >>>> What directory is openssl installed in? >>> /usr/bin/openssl >>> >>>> headers and libraries not in the main include/lib >>>> dirs? >>> How do I check it out?? >>> >>>> If not, then give the appropriate --with option >> to >>> say >where it is. >>> How do Ido that?? >> >> Have you read the LINUX.txt file in the >> boxbackup-0.10 directory? >> >> Have you used Google to find out about how to use >> configure commands? >> >> Ben >> >> >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> > http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu May 18 13:59:34 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Thu, 18 May 2006 05:59:34 -0700 (PDT) Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <14869448-298A-4F02-AF5B-4AE3E754FCC8@fluffy.co.uk> Message-ID: <20060518125934.40049.qmail@web30404.mail.mud.yahoo.com> Thnxs Ben, I re-installed libssl-dev ./configure seems to work OK. Hopefully I will get there! Thanxs, Jeff --- Ben Summers wrote: > > Did you install the -dev packages as suggested by > LINUX.txt? > > Ben > > > On 18 May 2006, at 12:59, vordoo wrote: > > > Sorry Ben, > > > > I'm a newbie to headers and libraries, I usualy > > apt-get install. > > > > I do not know how to find the location of ssl > headers > > and libraries Box Backup needs, on my debian sys. > > Aftet reading, OpenSSL notes, LINUX.txt and > googling I > > have failed, aperantly pointing to the wrong place > or > > using the wrong switch, I decided to just ask. > > > > Sorry, > > Jeff > > > > --- Ben Summers wrote: > > > >> > >> On 18 May 2006, at 12:08, vordoo wrote: > >> > >>> Thnx, Martin! > >>> > >>>> What directory is openssl installed in? > >>> /usr/bin/openssl > >>> > >>>> headers and libraries not in the main > include/lib > >>>> dirs? > >>> How do I check it out?? > >>> > >>>> If not, then give the appropriate --with option > >> to > >>> say >where it is. > >>> How do Ido that?? > >> > >> Have you read the LINUX.txt file in the > >> boxbackup-0.10 directory? > >> > >> Have you used Google to find out about how to use > >> configure commands? > >> > >> Ben > >> > >> > >> > >> > >> _______________________________________________ > >> boxbackup mailing list > >> boxbackup at fluffy.co.uk > >> > > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > >> > > > > > > __________________________________________________ > > Do You Yahoo!? > > Tired of spam? Yahoo! Mail has the best spam > protection around > > http://mail.yahoo.com > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Thu May 18 14:04:16 2006 From: boxbackup at fluffy.co.uk (Jamie Neil) Date: Thu, 18 May 2006 14:04:16 +0100 Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> References: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> Message-ID: <446C70D0.90305@versado.net> vordoo wrote: > Sorry Ben, > > I'm a newbie to headers and libraries, I usualy > apt-get install. We have precompiled binaries of 0.10 packaged for Debian 3.1 (Sarge) at: ftp://ftp.versado.net/debian/ if you are interested. The packaging is based on the 0.9 version that was advertised on the list some months ago, with client, server and utils parts separated. We've been using them for a few months with no problems, but I can't guarantee that they are bug free. Just add: deb http://ftp.versado.net/debian sarge main contrib non-free to your sources.list, and apt-get install as normal. If you find them useful then feedback is always welcome. Regards, -- Jamie Neil | | 0870 7777 454 Versado I.T. Services Ltd. | http://versado.net/ | 0845 450 1254 From boxbackup at fluffy.co.uk Thu May 18 14:31:12 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Thu, 18 May 2006 06:31:12 -0700 (PDT) Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <446C70D0.90305@versado.net> Message-ID: <20060518133112.20677.qmail@web30413.mail.mud.yahoo.com> > We have precompiled binaries of 0.10 packaged for > Debian 3.1 (Sarge) at: > ftp://ftp.versado.net/debian/ Thnxs Jamie, I will check them out my next installation(a sec. box is awating one) For this box, I will tray to finesh compiling from source files, No doubt I need practice at it. Jeff. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Thu May 18 15:31:19 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 14:31:19 +0000 (GMT) Subject: [Box Backup] configure: error: OpenSSL is not installed? In-Reply-To: <446C70D0.90305@versado.net> References: <20060518115935.29273.qmail@web30414.mail.mud.yahoo.com> <446C70D0.90305@versado.net> Message-ID: Hi Jamie, > We have precompiled binaries of 0.10 packaged for Debian 3.1 (Sarge) at: > > ftp://ftp.versado.net/debian/ > > if you are interested. The packaging is based on the 0.9 version that > was advertised on the list some months ago, with client, server and > utils parts separated. We've been using them for a few months with no > problems, but I can't guarantee that they are bug free. Thanks for that! I've posted the URL and brief instructions on the Wiki at [http://boxbackup.hostworks.ca/index.php/0.10#Download]. If anyone knows of other packaged distributions of Box Backup 0.10, please add them to the Wiki as well, or announce them here. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 19:20:21 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 18 May 2006 20:20:21 +0200 Subject: [Box Backup] Tunneling TLS through SSH? Message-ID: <446CBAE5.3070302@homann.se> Is it possible to tunnel TLS through SSH? My backup server is Far Away, and I can't open the bbstored port on that machine. Thus, ssh to rescue. Or so I thought, but I have a hard time debugging the setup. I'm no root on the bbstored machine. Has anyone succeeded in doing this? How? Thanks, Magnus From boxbackup at fluffy.co.uk Thu May 18 20:23:00 2006 From: boxbackup at fluffy.co.uk (Pascal Lalonde) Date: Thu, 18 May 2006 15:23:00 -0400 Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <446CBAE5.3070302@homann.se> References: <446CBAE5.3070302@homann.se> Message-ID: <20060518192300.GB10609@lostsoul.doom.und> On Thu, May 18, 2006 at 08:20:21PM +0200, Magnus Homann wrote: > Is it possible to tunnel TLS through SSH? My backup server is Far Away, > and I can't open the bbstored port on that machine. Thus, ssh to rescue. > Or so I thought, but I have a hard time debugging the setup. I'm no root > on the bbstored machine. > > Has anyone succeeded in doing this? How? > The latest OpenSSH has a tunnel feature, making use of a tunnel pseudo-device. I don't know if this is limited to OpenBSD, but it might be what you're looking for. -- Pascal From boxbackup at fluffy.co.uk Thu May 18 20:26:04 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 19:26:04 +0000 (GMT) Subject: [Box Backup] Logging Message-ID: Hi all, I've written up what I believe is a fair conclusion from our discussions about logging in December. You can find it on the Wiki at [http://boxbackup.hostworks.ca/index.php/Logging_framework]. If you have any comments, criticisms, etc. please let me know. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 20:44:50 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 12:44:50 -0700 (PDT) Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <446CBAE5.3070302@homann.se> Message-ID: <20060518194450.52414.qmail@web60614.mail.yahoo.com> Hi Magnus, Do these pages help?: http://www.rzg.mpg.de/networking/tunnelling.html http://hacks.oreilly.com/pub/h/66 Making this up as I go, can you do something maybe like: In bbackupd.conf, change StoreHostname = yourhost to StoreHostname = localhost Then set up your ssh keys. Then create the tunnel in an init script and/or a cron job: ssh -L 2201:storeserver:2201 farawayrouter cat - (The cat - trick is new to me.) (I don't suppose you can use your box keys instead of having to generate new ones.) Leaving the tunnel open like that is a little weird, I guess, and maybe not stable. Maybe you can hack the bbbackud somehow to first create the tunnel? (I'm experimenting with openvpn http://openvpn.net/ at the moment for a client who wants to do off-site backups to his new big WinXP box at home. Don't know if that helps at all.) I hope that is close to being correct and helps a tiny bit. Removing all doubt, Pete --- Magnus Homann wrote: > Is it possible to tunnel TLS through SSH? My backup server is Far > Away, > and I can't open the bbstored port on that machine. Thus, ssh to > rescue. > Or so I thought, but I have a hard time debugging the setup. I'm no > root > on the bbstored machine. > > Has anyone succeeded in doing this? How? > > Thanks, > Magnus > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu May 18 20:53:16 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 12:53:16 -0700 (PDT) Subject: [Box Backup] Logging In-Reply-To: Message-ID: <20060518195316.18299.qmail@web60624.mail.yahoo.com> Hi Chris, Great job, as usual. There are a few log-relatd feature requests at: http://boxbackup.hostworks.ca/index.php/Feature_requests Some are more obviously log-related than others. Stuart's and mine there are definitely. Traffic related? Client vs server logging differences? Thanks, Pete --- Chris Wilson wrote: > Hi all, > > I've written up what I believe is a fair conclusion from our > discussions > about logging in December. You can find it on the Wiki at > [http://boxbackup.hostworks.ca/index.php/Logging_framework]. > > If you have any comments, criticisms, etc. please let me know. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu May 18 21:15:00 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 13:15:00 -0700 (PDT) Subject: [Box Backup] exception Connection Protocol_HandshakeFailed (7/48) Message-ID: <20060518201500.29658.qmail@web60612.mail.yahoo.com> One for the search engines: I was upgrading my tax guys' Windows XP (WinXP) box backup client today to Chris' new 568 client, and then after it was running awhile and we were chatting about other things, I accidentally on purpose turned off the router to reset his wi-fi for another desktop in his office. The backup came to a grinding halt of course. After starting the router again, box backup would connect to the server, but would stop, throwing this in the Event Viewer: exception Connection Protocol_HandshakeFailed (7/48) After duly flagellating myself like some kind of monk, and googling site:lists.warhead.org.uk HandshakeFailed and "box backup" HandshakeFailed and finding nothing and not having a clue as what to do next, I ssh'd back to the server, looked in /var/log/box and saw 3 bbstored processes running. I found the /var/log/box record that had the 3rd process id and my client account id, and saw that the last log record for that process id was it hanging on an upload that I knew was dead, and so I killed that process, restarted the Windows service, and all was good. I hope this helps someone down the road. From boxbackup at fluffy.co.uk Thu May 18 22:12:45 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 14:12:45 -0700 (PDT) Subject: [Box Backup] Windows Client 568 Integrity Info In-Reply-To: Message-ID: <20060518211245.84784.qmail@web60614.mail.yahoo.com> A little extra security, FWIW. I haven't used my keys for anything useful in a very long time, but maybe I'll digitally dust them off and sign this thing. $ date Thu May 18 17:06:22 EST 2006 In order of approximate descreasing security but increasing portability: $ sha1sum boxbackup-chris_general_568-backup-client-mingw32.zip 8a746b4e9393d4bcdc33788d6d384caa422dd067 *boxbackup-chris_general_568-backup-client-mingw32.zip $ md5sum boxbackup-chris_general_568-backup-client-mingw32.zip 8b273fca0afe9ca41febce5271f56856 *boxbackup-chris_general_568-backup-client-mingw32.zip $ cksum boxbackup-chris_general_568-backup-client-mingw32.zip 1005546371 2695228 boxbackup-chris_general_568-backup-client-mingw32.zip $ ls -s boxbackup-chris_general_568-backup-client-mingw32.zip 2636 boxbackup-chris_general_568-backup-client-mingw32.zip $ ls -sh boxbackup-chris_general_568-backup-client-mingw32.zip 2.6M boxbackup-chris_general_568-backup-client-mingw32.zip --- Chris Wilson wrote: > Hi Pete, > > > Chris, I guess I should ask, is it worth me not having my clients > > backing up for a day or two (yikes!) to wait for us to work through > the > > regex problem before I upgrade them? What new features/fixes are > you/we > > trying to get? In my ignorance, the command line services tools > seem > > fine to me the way they are, so those aren't worth me waiting, so > in my > > situation, I'm inclined to just use the older 538 instead of 568 or > > > later. > > I've just uploaded a new client, revision 568. If you have time, > please > could you give it a try? > > [http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/boxbackup-chris_general_568-backup-client-mingw32.zip] > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu May 18 22:35:11 2006 From: boxbackup at fluffy.co.uk (tbp) Date: Thu, 18 May 2006 23:35:11 +0200 Subject: [Box Backup] Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <20060518091400.7308.67281.Mailman@love.warhead.org.uk> References: <20060518091400.7308.67281.Mailman@love.warhead.org.uk> Message-ID: <446CE88F.5020407@kontrapunkt.com> Hello... Im maintaining a backup of 1TB+ worth of files from an office location to an offsite datacenter. Server and client are both OS X 10.4 and run boxbackup 0.10. Usually boxbackup quits on errors as seen below. What should I do to troubleshoot/correct the situation? Is this a timeout problem of some kind? Also, on the client boxbackup says "...reset state and waiting to retry...". Is boxbackup supposed to resume the backup run? This never happens. CLIENT: May 18 22:04:44 yoiko bbackupd[374]: Send StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,".DS_Store") May 18 22:04:44 yoiko bbackupd[374]: Sending stream, size uncertain May 18 22:04:44 yoiko bbackupd[374]: Receive Success(0x190dd8) May 18 22:04:44 yoiko bbackupd[374]: Send StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,"Exterior 3 (Beauburg) 040506.ai") May 18 22:04:44 yoiko bbackupd[374]: Sending stream, size uncertain May 18 22:07:20 yoiko bbackupd[374]: Send SetClientStoreMarker(0x414159562ee00) May 18 22:07:20 yoiko bbackupd[374]: SSL err during Write: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry May 18 22:07:20 yoiko bbackupd[374]: Exception caught (Connection TLSWriteFailed (Probably a network issue between client and server.) 7/33), reset state and waiting to retry... May 18 22:07:30 yoiko bbackupd[374]: File statistics: total file size uploaded 6026769275, bytes already on server 119254583, encoded size 3983556549 SERVER: May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size 49 May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd7) May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd7) May 18 22:04:44 maigo bbstored[19757]: Receive StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,OPAQUE) May 18 22:04:44 maigo bbstored[19757]: Receive StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,OPAQUE) May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size uncertain May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd8) May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd8) May 18 22:04:44 maigo bbstored[19757]: Receive StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,OPAQUE) May 18 22:04:44 maigo bbstored[19757]: Receive StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,OPAQUE) May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size uncertain May 18 22:05:13 maigo bbstored[19757]: Connection statistics for BACKUP-A: IN=4066005908 OUT=1010157578 TOTAL=5076163486\n May 18 22:05:13 maigo bbstored[19757]: in server child, exception Connection TLSReadFailed (Probably a network issue between client and server.) (7/34) -- terminating child Thanks, Tobias From boxbackup at fluffy.co.uk Thu May 18 22:38:46 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 21:38:46 +0000 (GMT) Subject: [Box Backup] Logging In-Reply-To: <20060518195316.18299.qmail@web60624.mail.yahoo.com> References: <20060518195316.18299.qmail@web60624.mail.yahoo.com> Message-ID: Hi Pete, > There are a few log-relatd feature requests at: > http://boxbackup.hostworks.ca/index.php/Feature_requests Some are more > obviously log-related than others. Stuart's and mine there are > definitely. Thanks for pointing that out. I guess you mean the following requests: 1. Prefix bbstored log messages with the client account number 2. In Windows Event Viewer, change "Exceeded storage limits" to Warning 3. Log good and bad results of proposed housekeeping health check 4. Log "not uploading changes to files" when appropriate on the store 5. Log amount of bandwidth used by each client session (if not already) Did I miss any? The first two are definitely related to the logging infrastructure. For number 1, I've added a little bit to the logging spec (SetContext/ClearContext) which would make it easy to log this information in bbstored. Number 2 is trivial, I've just checked it into my tree. Numbers 3-5 require additional work to other parts of the store daemon, which would then use the new logging infrastructure. I'm not proposing to implement these right now. > Traffic related? By traffic related, do you mean logging the amount of bandwidth used? > Client vs server logging differences? "Client" and "server" are tricky terms to use here. Both bbackupd and bbstored run as daemons, and their principle means of logging should be syslog. As you correctly pointed out, bbstored handles multiple clients, so it should identify messages that relate to a particular client with the client's account number. Apart from that, I can't see any obvious reason for differences. On the other hand, the command-line utilities, like bbackupquery and bbackupctl, would probably use the console logger exclusively, rather than logging anything to syslog. Does that make sense to everyone? Should there be a flag to make them log to syslog, for example to run them as scheduled tasks on Windows where there's no Cron? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 22:42:53 2006 From: boxbackup at fluffy.co.uk (Jamie Webb) Date: Thu, 18 May 2006 22:42:53 +0100 Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <446CBAE5.3070302@homann.se> References: <446CBAE5.3070302@homann.se> Message-ID: <20060518214253.GJ20943@digitalice.co.uk> On Thu, May 18, 2006 at 08:20:21PM +0200, Magnus Homann wrote: > Is it possible to tunnel TLS through SSH? My backup server is Far Away, and I can't open the bbstored port on that > machine. Thus, ssh to rescue. Or so I thought, but I have a hard time debugging the setup. I'm no root on the bbstored > machine. > > Has anyone succeeded in doing this? How? I've not tried, but it should be as simple as: ssh -L2201:localhost:2201 your.server.com Then tell bbackupd that the store is on localhost. You may want to use autossh to keep the connection live. -- Jamie Webb From boxbackup at fluffy.co.uk Thu May 18 23:14:26 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 15:14:26 -0700 (PDT) Subject: [Box Backup] Logging In-Reply-To: Message-ID: <20060518221426.58605.qmail@web60611.mail.yahoo.com> --- Chris Wilson wrote: > Hi Pete, > > > There are a few log-relatd feature requests at: > > http://boxbackup.hostworks.ca/index.php/Feature_requests Some are > more > > obviously log-related than others. Stuart's and mine there are > > definitely. > > Thanks for pointing that out. I guess you mean the following > requests: Sorry for my laziness... (I'm afraid to ask, are there other pages on the wiki that might yield logging ideas, while you're there under the hood? Future plans, 0.20 plans, Troubleshooting? Sorry.) > 1. Prefix bbstored log messages with the client account number > 2. In Windows Event Viewer, change "Exceeded storage limits" to > Warning > 3. Log good and bad results of proposed housekeeping health check > 4. Log "not uploading changes to files" when appropriate on the store > 5. Log amount of bandwidth used by each client session (if not > already) > > Did I miss any? Nope, good, thanks. > The first two are definitely related to the logging infrastructure. > > For number 1, I've added a little bit to the logging spec > (SetContext/ClearContext) which would make it easy to log this > information > in bbstored. > > Number 2 is trivial, I've just checked it into my tree. > > Numbers 3-5 require additional work to other parts of the store > daemon, > which would then use the new logging infrastructure. I'm not > proposing to > implement these right now. > > > Traffic related? > > By traffic related, do you mean logging the amount of bandwidth used? Yeah, I was just parroting what was in the feature request. > > Client vs server logging differences? > > "Client" and "server" are tricky terms to use here. Both bbackupd and > > bbstored run as daemons, and their principle means of logging should > be > syslog. As you correctly pointed out, bbstored handles multiple > clients, > so it should identify messages that relate to a particular client > with the > client's account number. Please see my feature request about keeping the account numbers in the logs consistent. I've since started making my account numbers as 8 characters starting with a non-zero, like 10001234, so that it appears the same (I think) in all the different log entries, but before that, my log grepping scripts had to trap the different formats of the account numbers (case, leading zeros, leading 0x(?), e.g.). Apart from that, I can't see any obvious > reason > for differences. My angle, as the store server admin, as listed in my feature requests, is that I need to constantly monitor my machine as much as possible to make sure my customers' data are safe. I won't have ready access to the client logs (and actually don't want to have to reach out to what I hope will be dozens and dozens of them), so I need to store server to be fairly chatty about how things are going, and to rely on the client to somehow get what might be a lot of info into the store server log. For example, and this relates to your Scheduled Task item below, in my feature request: Add automated periodic compare to client. Maybe a daily quick compare and a weekly late-night full (or nearly so) compare. http://lists.warhead.org.uk/pipermail/boxbackup/2006-February/002240.html (PJ) it'd be nice if I could see the results of those client-run compares in the store-server log somehow. > On the other hand, the command-line utilities, like bbackupquery and > bbackupctl, would probably use the console logger exclusively, rather > than > logging anything to syslog. Does that make sense to everyone? Should > there > be a flag to make them log to syslog, for example to run them as > scheduled > tasks on Windows where there's no Cron? (Not knowing anything about it, except that I am at the moment making backup copies of commercial dvds [Led Zeppelin, Buckethead; to keep my kids from scratching the originals] using cdrecord with the dvd patch and I happened to notice that it carries a cygwin1.dll with it, so I wonder if that would be a useful trick to use to set up a cygwin cron service to do what you need it to do? Sorry if that is way off.) From boxbackup at fluffy.co.uk Thu May 18 23:22:11 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 22:22:11 +0000 (GMT) Subject: [Box Backup] Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <446CE88F.5020407@kontrapunkt.com> References: <20060518091400.7308.67281.Mailman@love.warhead.org.uk> <446CE88F.5020407@kontrapunkt.com> Message-ID: Hi Tobias, On Thu, 18 May 2006, tbp wrote: > Usually boxbackup quits on errors as seen below. What should I do to > troubleshoot/correct the situation? Is this a timeout problem of some kind? Yes, it looks like an SSL timeout. Please try enabling the KeepAliveTime option to see if it helps (try setting it to about 30 seconds). > Also, on the client boxbackup says "...reset state and waiting to retry...". > Is boxbackup supposed to resume the backup run? This never happens. It should start another backup run, but if the previous one did not finish successfully, there will be no cached state, so it will start again from the beginning - and probably fail in the same place again. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 23:33:24 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 22:33:24 +0000 (GMT) Subject: [Box Backup] Logging In-Reply-To: <20060518221426.58605.qmail@web60611.mail.yahoo.com> References: <20060518221426.58605.qmail@web60611.mail.yahoo.com> Message-ID: Hi Pete, > Sorry for my laziness... (I'm afraid to ask, are there other pages on > the wiki that might yield logging ideas, while you're there under the > hood? Future plans, 0.20 plans, Troubleshooting? Sorry.) I'm not aware of any other places where such gremlins might be hiding - perhaps in the robot spam on our wiki? :-) If anybody knows, please speak now, or forever pause your console. > Please see my feature request about keeping the account numbers in the > logs consistent. I've since started making my account numbers as 8 > characters starting with a non-zero, like 10001234, so that it appears > the same (I think) in all the different log entries, but before that, my > log grepping scripts had to trap the different formats of the account > numbers (case, leading zeros, leading 0x(?), e.g.). Yes, I was going to make the logging of these account numbers consistent, as you requested. My plan was to format them all as 0x01234567 (i.e. 8 hex digits prefixed by 0x). Any objections? > My angle, as the store server admin, as listed in my feature requests, > is that I need to constantly monitor my machine as much as possible to > make sure my customers' data are safe. I won't have ready access to the > client logs (and actually don't want to have to reach out to what I hope > will be dozens and dozens of them), so I need to store server to be > fairly chatty about how things are going, and to rely on the client to > somehow get what might be a lot of info into the store server log. I can definitely understand that, but I think the server does not have all the information required to debug some problems, especially as the files on it are encrypted and opaque. When you do need to go to the client, you want the logs there to be clear, concise and accurate as well. > Add automated periodic compare to client. Maybe a daily quick compare > and a weekly late-night full (or nearly so) compare. > http://lists.warhead.org.uk/pipermail/boxbackup/2006-February/002240.html > (PJ) it'd be nice if I could see the results of those client-run > compares in the store-server log somehow. Only the client knows this. Perhaps we could add a protocol command that the client sends to the server to request it to log a message? That's not really about the logging infrastructure though. Can you add it to the feature requests please? > (Not knowing anything about it, except that I am at the moment making > backup copies of commercial dvds [Led Zeppelin, Buckethead; to keep my > kids from scratching the originals] using cdrecord with the dvd patch > and I happened to notice that it carries a cygwin1.dll with it, so I > wonder if that would be a useful trick to use to set up a cygwin cron > service to do what you need it to do? Sorry if that is way off.) It is possible to use Cygwin cron to schedule tasks on a Windows machine, but it's definitely not the right cup of tea for most Windows users. It's also quite heavyweight. Windows has its own was to schedule tasks, I believe, but unlike Cron it doesn't capture console output to mail to the admin. That's why it might be useful for the command line tools to be able to log to syslog. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu May 18 23:32:53 2006 From: boxbackup at fluffy.co.uk (Per Thomsen) Date: Thu, 18 May 2006 15:32:53 -0700 Subject: [Box Backup] Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <446CE88F.5020407@kontrapunkt.com> References: <20060518091400.7308.67281.Mailman@love.warhead.org.uk> <446CE88F.5020407@kontrapunkt.com> Message-ID: <446CF615.1030703@reedtz.com> On 5/18/06 2:35 PM, tbp wrote: > Hello... > > Im maintaining a backup of 1TB+ worth of files from an office location > to an offsite datacenter. > > Server and client are both OS X 10.4 and run boxbackup 0.10. > > Usually boxbackup quits on errors as seen below. What should I do to > troubleshoot/correct the situation? Is this a timeout problem of some > kind? Also, on the client boxbackup says "...reset state and waiting > to retry...". Is boxbackup supposed to resume the backup run? This > never happens. To get started on the troubleshooting, I have a few questions: Are you using the KeepAlive stuff? Looks like the client thinks there is a ~2.5 minute gap between the start of send and the failure. The server loses the connection about 29secs after starting to receive. What quality is the connection to the datacenter? Could there be short outages? If this happens even semi-reliably, can you put a debug build of the client (and, if possible, the server) in, to get more details on what's going on? Is it always the same file that's causing the problem? If so, how big is this file? HTH, Per Thomsen > > > > CLIENT: > May 18 22:04:44 yoiko bbackupd[374]: Send > StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,".DS_Store") > May 18 22:04:44 yoiko bbackupd[374]: Sending stream, size uncertain > May 18 22:04:44 yoiko bbackupd[374]: Receive Success(0x190dd8) > May 18 22:04:44 yoiko bbackupd[374]: Send > StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,"Exterior 3 > (Beauburg) 040506.ai") > May 18 22:04:44 yoiko bbackupd[374]: Sending stream, size uncertain > May 18 22:07:20 yoiko bbackupd[374]: Send > SetClientStoreMarker(0x414159562ee00) > May 18 22:07:20 yoiko bbackupd[374]: SSL err during Write: > error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry > May 18 22:07:20 yoiko bbackupd[374]: Exception caught (Connection > TLSWriteFailed (Probably a network issue between client and server.) > 7/33), reset state and waiting to retry... > May 18 22:07:30 yoiko bbackupd[374]: File statistics: total file size > uploaded 6026769275, bytes already on server 119254583, encoded size > 3983556549 > > > > SERVER: > May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size 49 > May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd7) > May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd7) > May 18 22:04:44 maigo bbstored[19757]: Receive > StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,OPAQUE) > May 18 22:04:44 maigo bbstored[19757]: Receive > StoreFile(0x190dd7,0x412f6051ec500,0x3cecde0cf68bd94f,0x0,OPAQUE) > May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size uncertain > May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd8) > May 18 22:04:44 maigo bbstored[19757]: Send Success(0x190dd8) > May 18 22:04:44 maigo bbstored[19757]: Receive > StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,OPAQUE) > May 18 22:04:44 maigo bbstored[19757]: Receive > StoreFile(0x190dd7,0x412f5fc6afc80,0x6b598ba4fb4270b9,0x0,OPAQUE) > May 18 22:04:44 maigo bbstored[19757]: Receiving stream, size uncertain > May 18 22:05:13 maigo bbstored[19757]: Connection statistics for > BACKUP-A: IN=4066005908 OUT=1010157578 TOTAL=5076163486\n > May 18 22:05:13 maigo bbstored[19757]: in server child, exception > Connection TLSReadFailed (Probably a network issue between client and > server.) (7/34) -- terminating child > > > Thanks, > Tobias > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Fri May 19 00:18:13 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Fri, 19 May 2006 00:18:13 +0100 Subject: [Box Backup] Logging In-Reply-To: References: Message-ID: <1147994293.6512.3.camel@avenin.ebourne.me.uk> On Thu, 2006-05-18 at 19:26 +0000, Chris Wilson wrote: > Hi all, > > I've written up what I believe is a fair conclusion from our discussions > about logging in December. You can find it on the Wiki at > [http://boxbackup.hostworks.ca/index.php/Logging_framework]. > > If you have any comments, criticisms, etc. please let me know. Looks fine. Just a couple of minor things: - Syslog(), Add(), etc. should presumably be public - Not sure that the Syslog() and Console() methods are really of any benefit. I would think that using Add() directly would be just as easy Cheers, Martin. From boxbackup at fluffy.co.uk Fri May 19 00:36:25 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 18 May 2006 23:36:25 +0000 (GMT) Subject: [Box Backup] Logging In-Reply-To: <1147994293.6512.3.camel@avenin.ebourne.me.uk> References: <1147994293.6512.3.camel@avenin.ebourne.me.uk> Message-ID: Hi Martin, > Looks fine. Just a couple of minor things: > - Syslog(), Add(), etc. should presumably be public Yes, you're absolutely right, sorry! > - Not sure that the Syslog() and Console() methods are really of any > benefit. I would think that using Add() directly would be just as easy The problem is the lifetime of the logger object. We're going to keep a reference to the object passed in, so it must outlive any uses of it. However, console or syslog logging would typically be started when the application starts, perhaps before any objects are instantiated. You might also want to start logging to the console in main(), and stop in Daemon::Main(), just after a successful fork(). But I can't see a clean way for Daemon::Main and main() to share a reference to a Logger object. Therefore, I was planning to hide the default console logger and the default syslog logger inside the Log class as private static members - is that possible? Otherwise I could keep them in auto_ptrs and create them when Log::ToSyslog() or Log::ToConsole() is first called. I added a couple of methods to set the log level on the console and syslog. Perhaps we don't need Add() and Remove() for now? By the way, one thing I changed from your spec was the BOX_LOG macro. I changed __FUNCTION__ to __FILE__, since __FUNCTION__ on its own isn't very helpful in tracking down which file the offending code is located in (I think). I'm also assuming that both __FILE__ and __LINE__ can be converted to std::string without problems. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Fri May 19 06:37:25 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 18 May 2006 22:37:25 -0700 (PDT) Subject: [Box Backup] Logging In-Reply-To: Message-ID: <20060519053725.15179.qmail@web60611.mail.yahoo.com> > > it'd be nice if I could see the results of those client-run > > compares in the store-server log somehow. > > Only the client knows this. Perhaps we could add a protocol command > that > the client sends to the server to request it to log a message? That's > not > really about the logging infrastructure though. Can you add it to the > feature requests please? Done, thanks. (I also deleted all the hidden wiki spam at the bottom; interesting trick they use.) > It is possible to use Cygwin cron to schedule tasks on a Windows > machine, > but it's definitely not the right cup of tea for most Windows users. > It's > also quite heavyweight. Windows has its own was to schedule tasks, I > believe, but unlike Cron it doesn't capture console output to mail to > the > admin. That's why it might be useful for the command line tools to be > able > to log to syslog. In my ignorance, I was thinking that maybe it was as simple as carrying along a very small handful of files, like maybe cygwin1.dll, cron.exe, cygrunsrv.exe, or something like that. From boxbackup at fluffy.co.uk Fri May 19 22:25:27 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Fri, 19 May 2006 23:25:27 +0200 Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <20060518214253.GJ20943@digitalice.co.uk> References: <446CBAE5.3070302@homann.se> <20060518214253.GJ20943@digitalice.co.uk> Message-ID: <446E37C7.9030207@homann.se> Jamie Webb wrote: > On Thu, May 18, 2006 at 08:20:21PM +0200, Magnus Homann wrote: >> Is it possible to tunnel TLS through SSH? My backup server is Far Away, and I can't open the bbstored port on that >> machine. Thus, ssh to rescue. Or so I thought, but I have a hard time debugging the setup. I'm no root on the bbstored >> machine. >> >> Has anyone succeeded in doing this? How? > > I've not tried, but it should be as simple as: > > ssh -L2201:localhost:2201 your.server.com > > Then tell bbackupd that the store is on localhost. You may want to use > autossh to keep the connection live. No, it's not that easy! :-) Any info on which machines and for which machines the certificates should be generated. It doesn't help not having access too syslog either. From boxbackup at fluffy.co.uk Fri May 19 22:27:31 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Fri, 19 May 2006 23:27:31 +0200 Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <20060518194450.52414.qmail@web60614.mail.yahoo.com> References: <20060518194450.52414.qmail@web60614.mail.yahoo.com> Message-ID: <446E3843.7090303@homann.se> E.W. Peter Jalajas wrote: > Hi Magnus, > > Do these pages help?: > http://www.rzg.mpg.de/networking/tunnelling.html > http://hacks.oreilly.com/pub/h/66 Thanks, I've tried most of that already. I started with setying up client and server on the same machine, and then copying the bbstored files to anothe machine, and also copied the bbstored certificates. I guess this should work, the certificates are not connected to the hostname, right? -- Magnus Homann magnus at homann.se +46 702 399 558 From boxbackup at fluffy.co.uk Fri May 19 22:27:41 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Fri, 19 May 2006 14:27:41 -0700 (PDT) Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <446E3843.7090303@homann.se> Message-ID: <20060519212741.12970.qmail@web60620.mail.yahoo.com> --- Magnus Homann wrote: > E.W. Peter Jalajas wrote: > > Hi Magnus, > > > > Do these pages help?: > > http://www.rzg.mpg.de/networking/tunnelling.html > > http://hacks.oreilly.com/pub/h/66 > > Thanks, I've tried most of that already. > > I started with setying up client and server on the same machine, and > then copying the bbstored files to anothe machine, and also copied > the > bbstored certificates. I guess this should work, the certificates are > > not connected to the hostname, right? I believe that is correct, but someone else on the list should verify. It is the contents of the digital keys that need to connect. You'd of course have to install the matching version of bbstored on the other machine, including what'll probably be essentially a completely new /etc/box/bbstored.conf. You'd also of course have to modify the client /etc/box/bbackud.conf to point to the other machine by the correct name. And your dns, routing (including port forwarding), and firewalls would all have to be configured correctly. And probably other things that I'm forgetting at the moment. > -- > Magnus Homann > magnus at homann.se > +46 702 399 558 > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Fri May 19 22:52:48 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Fri, 19 May 2006 14:52:48 -0700 (PDT) Subject: [Box Backup] Howto Upgrade Box Backup from 0.09 to 0.10? Message-ID: <20060519215248.83017.qmail@web60619.mail.yahoo.com> I have an Ubuntu Breezy (Debian) 0.09 client that I need to upgrade to talk to my newly migrated (thanks to much help from Chris and Ben on this list) 0.10 server (on Ubuntu Dapper, by the way). The client already has an account on the new server (like all the Windows clients that I've successfully moved over to there). So, can you guys tell me the best way to upgrade the 0.09 linux client to 0.10? I'll document my steps and add a wiki. (A little more detail: the client was actually moved from my aging ubuntu laptop after it's hard drive crashed. So, in this case, both the server and client have moved machines in recent weeks.) Thanks, Pete From boxbackup at fluffy.co.uk Sat May 20 11:35:11 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sat, 20 May 2006 11:35:11 +0100 Subject: [Box Backup] Tunneling TLS through SSH? In-Reply-To: <20060519212741.12970.qmail@web60620.mail.yahoo.com> References: <20060519212741.12970.qmail@web60620.mail.yahoo.com> Message-ID: <197ABE10-57A8-4831-B7FD-7F59471CA62E@fluffy.co.uk> On 19 May 2006, at 22:27, E.W. Peter Jalajas wrote: > --- Magnus Homann wrote: > >> E.W. Peter Jalajas wrote: >>> Hi Magnus, >>> >>> Do these pages help?: >>> http://www.rzg.mpg.de/networking/tunnelling.html >>> http://hacks.oreilly.com/pub/h/66 >> >> Thanks, I've tried most of that already. >> >> I started with setying up client and server on the same machine, and >> then copying the bbstored files to anothe machine, and also copied >> the >> bbstored certificates. I guess this should work, the certificates are >> >> not connected to the hostname, right? > > I believe that is correct, but someone else on the list should verify. > It is the contents of the digital keys that need to connect. > > You'd of course have to install the matching version of bbstored on > the > other machine, including what'll probably be essentially a completely > new /etc/box/bbstored.conf. You'd also of course have to modify the > client /etc/box/bbackud.conf to point to the other machine by the > correct name. And your dns, routing (including port forwarding), and > firewalls would all have to be configured correctly. And probably > other things that I'm forgetting at the moment. It's just a TCP/IP connection on port 2201. No magic, not trickyness. I suggest the following approach: * Configure the server as normal, except use 'localhost' as the address for bbstored-config * Configure the client as normal, except use 'localhost' as the address of the server in bbackupd-config * Use ssh to tunnel with appropriate users and keys set up. Note that the ssh user does not have to be the same one as bbstored is running under, it just needs to be able to make a TCP/IP connection to port 2201 on localhost. ssh -L 2201:servers-real-address:2201 user at servers-real-address * You'll need to make sure that the tunnel is open when bbackupd wants to connect. May I suggest a cunning script using the SyncAllowScript, which is always run just before a sync starts by bbackupd. I suspect people are assuming it's more complex than it actually is. Ben From boxbackup at fluffy.co.uk Sat May 20 11:36:15 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sat, 20 May 2006 11:36:15 +0100 Subject: [Box Backup] Howto Upgrade Box Backup from 0.09 to 0.10? In-Reply-To: <20060519215248.83017.qmail@web60619.mail.yahoo.com> References: <20060519215248.83017.qmail@web60619.mail.yahoo.com> Message-ID: <890A5640-A7DF-4651-8996-1CFA7A9C7344@fluffy.co.uk> On 19 May 2006, at 22:52, E.W. Peter Jalajas wrote: > I have an Ubuntu Breezy (Debian) 0.09 client that I need to upgrade to > talk to my newly migrated (thanks to much help from Chris and Ben on > this list) 0.10 server (on Ubuntu Dapper, by the way). The client > already has an account on the new server (like all the Windows clients > that I've successfully moved over to there). > > So, can you guys tell me the best way to upgrade the 0.09 linux client > to 0.10? I'll document my steps and add a wiki. > > (A little more detail: the client was actually moved from my aging > ubuntu laptop after it's hard drive crashed. So, in this case, both > the > server and client have moved machines in recent weeks.) Compile a new version on the client (or on another machine with the exact same config -- VMWare is quite handy for this.) Replace binaries. Run. Ben From boxbackup at fluffy.co.uk Sat May 20 18:28:07 2006 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Sat, 20 May 2006 19:28:07 +0200 Subject: [Box Backup] Tunneling TLS through SSH works, but got (7/34) instead In-Reply-To: <197ABE10-57A8-4831-B7FD-7F59471CA62E@fluffy.co.uk> References: <20060519212741.12970.qmail@web60620.mail.yahoo.com> <197ABE10-57A8-4831-B7FD-7F59471CA62E@fluffy.co.uk> Message-ID: <446F51A7.9010409@homann.se> Ben Summers wrote: > > On 19 May 2006, at 22:27, E.W. Peter Jalajas wrote: > >> --- Magnus Homann wrote: >> >>> E.W. Peter Jalajas wrote: >>>> Hi Magnus, >>>> >>>> Do these pages help?: >>>> http://www.rzg.mpg.de/networking/tunnelling.html >>>> http://hacks.oreilly.com/pub/h/66 >>> >>> Thanks, I've tried most of that already. >>> >>> I started with setying up client and server on the same machine, and >>> then copying the bbstored files to anothe machine, and also copied >>> the >>> bbstored certificates. I guess this should work, the certificates are >>> >>> not connected to the hostname, right? >> >> I believe that is correct, but someone else on the list should verify. >> It is the contents of the digital keys that need to connect. >> >> You'd of course have to install the matching version of bbstored on the >> other machine, including what'll probably be essentially a completely >> new /etc/box/bbstored.conf. You'd also of course have to modify the >> client /etc/box/bbackud.conf to point to the other machine by the >> correct name. And your dns, routing (including port forwarding), and >> firewalls would all have to be configured correctly. And probably >> other things that I'm forgetting at the moment. > > It's just a TCP/IP connection on port 2201. No magic, not trickyness. > > I suggest the following approach: > > * Configure the server as normal, except use 'localhost' as the address > for bbstored-config > > * Configure the client as normal, except use 'localhost' as the address > of the server in bbackupd-config > > * Use ssh to tunnel with appropriate users and keys set up. Note that > the ssh user does not have to be the same one as bbstored is running > under, it just needs to be able to make a TCP/IP connection to port 2201 > on localhost. > > ssh -L 2201:servers-real-address:2201 user at servers-real-address This is exactly what I've done. The result has previously been that when I started bbackupquery (to test the setup), I got a TLSReadFailed (7/34) immediately. As I have no logs to see what's happening (not root on server), I had no clue why. Trying just now, I could do a couple of 'ls' and 'cd' before it appeared. So, the conclusion is that I got the certs and the tunnel set up properly. Excellent. When I cd into my top dir, everything is fine. I cannot " get backup.file dummy.file" in bbackupqurey, without getting "Error occured during fetching of file". If I do a "cd subdir", I works, but when typing "ls" afterwards, I get the TLSReadFailed (7/34) again. How to debug this without the logs? > * You'll need to make sure that the tunnel is open when bbackupd wants > to connect. May I suggest a cunning script using the SyncAllowScript, > which is always run just before a sync starts by bbackupd. I'll look into that. From boxbackup at fluffy.co.uk Sun May 21 06:08:02 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sat, 20 May 2006 22:08:02 -0700 (PDT) Subject: [Box Backup] Windows Client 568 Integrity Info Message-ID: <20060521050802.70453.qmail@web60616.mail.yahoo.com> --0-1710516535-1148188082=:68792 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Content-Id: Content-Disposition: inline Here's an update to the integrity info. So, FWIW, and likely not much, I signed the 568 product and include the signature below and attached. $ date ; export mfile="boxbackup-chris_general_568-backup-client-mingw32.zip"; for x in sha1sum md5sum "ls -s" "ls -sh"; do ${x} $mfile ; done ; gpg --no-secmem-warning --fingerprint 1C23B5F0 ; cat boxbackup-chris_general_568-backup-client-mingw32.zip.sig Sun May 21 01:01:07 EST 2006 8a746b4e9393d4bcdc33788d6d384caa422dd067 *boxbackup-chris_general_568-backup-client-mingw32.zip 8b273fca0afe9ca41febce5271f56856 *boxbackup-chris_general_568-backup-client-mingw32.zip 2636 boxbackup-chris_general_568-backup-client-mingw32.zip 2.6M boxbackup-chris_general_568-backup-client-mingw32.zip pub 3200R/1C23B5F0 2003-09-23 Key fingerprint = 5D13 7C5C D003 D806 8BD1 438A B396 EC3D 1C23 B5F0 uid E.W. Peter Jalajas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (MingW32) iQGlAwUARG/Z57OW7D0cI7XwAQLkSAyAprB2te8/aE1xQ3JIbbySephzgEC3GFPB kBgNhWrhcZzF1BCw0l/E/o/YltevWxxnChTE4gk1rb4QXjcIeiyrB8K7NegHLXvA GDDO8ekjhBBXIc0O7m3F5hHRSorRdE4kt/t95dCfEcUK7qkr21kSqwp7ZPynEsS1 OjU9u7atTHRQcWssuKMwSz8A4VEvbUEBDF6CEChwyCWiueqfch9xm4W/lGgRhIO7 fO4UDyGkw4pOnamFeAh+9NDcmKeQnRf+ctH/mQlNOG5pASSICmL8WSQmAYGi9vB4 jsYxrx4TrIQT//w7HuSmr1nVz3xJrgEhTFl4BQS4VwEfNXHJKeG9hy1v3Jihk4BR eTeHTHKMYGDhZP9y9ozquSJ+PGY43Y4f/pw5EZVnxBTIwzmbz05YIc/yIUjoVCWZ Gc4R2Lkq7Q3NFMflcyTBhZmf6ubJNE9wUPUdOcQW5wdao9Hf/xJpPZTNECFdlkXj prKiOx0pUAEzRKJDMG+6SVR+5uO64DYTZeS/Tq4m4X7n/x9aXTJwrg== =IpsI -----END PGP SIGNATURE----- --0-1710516535-1148188082=:68792 Content-Type: application/octet-stream; name="boxbackup-chris_general_568-backup-client-mingw32.zip.sig" Content-Transfer-Encoding: base64 Content-Description: 2864901707-boxbackup-chris_general_568-backup-client-mingw32.zip.sig Content-Disposition: attachment; filename="boxbackup-chris_general_568-backup-client-mingw32.zip.sig" LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0NClZlcnNpb246IEdudVBH IHYxLjQuMyAoTWluZ1czMikNCg0KaVFHbEF3VUFSRy9aNTdPVzdEMGNJN1h3 QVFMa1NBeUFwckIydGU4L2FFMXhRM0pJYmJ5U2VwaHpnRUMzR0ZQQg0Ka0Jn TmhXcmhjWnpGMUJDdzBsL0Uvby9ZbHRldld4eG5DaFRFNGdrMXJiNFFYamNJ ZWl5ckI4SzdOZWdITFh2QQ0KR0RETzhla2poQkJYSWMwTzdtM0Y1aEhSU29y UmRFNGt0L3Q5NWRDZkVjVUs3cWtyMjFrU3F3cDdaUHluRXNTMQ0KT2pVOXU3 YXRUSFJRY1dzc3VLTXdTejhBNFZFdmJVRUJERjZDRUNod3lDV2l1ZXFmY2g5 eG00Vy9sR2dSaElPNw0KZk80VUR5R2t3NHBPbmFtRmVBaCs5TkRjbUtlUW5S ZitjdEgvbVFsTk9HNXBBU1NJQ21MOFdTUW1BWUdpOXZCNA0KanNZeHJ4NFRy SVFULy93N0h1U21yMW5WejN4SnJnRWhURmw0QlFTNFZ3RWZOWEhKS2VHOWh5 MXYzSmloazRCUg0KZVRlSFRIS01ZR0RoWlA5eTlvenF1U0orUEdZNDNZNGYv cHc1RVpWbnhCVEl3em1iejA1WUljL3lJVWpvVkNXWg0KR2M0UjJMa3E3UTNO Rk1mbGN5VEJoWm1mNnViSk5FOXdVUFVkT2NRVzV3ZGFvOUhmL3hKcFBaVE5F Q0ZkbGtYag0KcHJLaU94MHBVQUV6UktKRE1HKzZTVlIrNXVPNjREWVRaZVMv VHE0bTRYN24veDlhWFRKd3JnPT0NCj1JcHNJDQotLS0tLUVORCBQR1AgU0lH TkFUVVJFLS0tLS0NCg== --0-1710516535-1148188082=:68792-- From boxbackup at fluffy.co.uk Mon May 22 05:49:15 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sun, 21 May 2006 21:49:15 -0700 (PDT) Subject: [Box Backup] How to move store to new machine In-Reply-To: <20060516210147.19262.qmail@web60619.mail.yahoo.com> Message-ID: <20060522044915.49337.qmail@web60611.mail.yahoo.com> > > Someday soon I'll script up a command line to confirm the regex is > working. Right after I update my clients...and update the wiki. > I added a wiki section on testing regex on Windows on Box Backup 0.10: http://boxbackup.hostworks.ca/index.php/Verify_and_restore_backups#How_to_Test_RegEx_on_Box_Backup_0.10_Windows_Client I hope someone finds it useful. Pete From boxbackup at fluffy.co.uk Mon May 22 11:15:22 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Mon, 22 May 2006 12:15:22 +0200 Subject: [Box Backup] Re: Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <20060519053800.10903.18471.Mailman@love.warhead.org.uk> References: <20060519053800.10903.18471.Mailman@love.warhead.org.uk> Message-ID: <44718F3A.5050200@kontrapunkt.com> Hello... Thank you for helping out. Chris: Thank you for pointing out the KeepAliveTime parameter which I did not know about. As you suggested, I have added this line to my bbackupd.conf file: KeepAliveTime = 30 I will post the results to the list. Per: > Are you using the KeepAlive stuff? No, I just found out about it. Added as seen above. > What quality is the connection to the datacenter? > Could there be short outages? Hmmm.... I dont really know what to answer here, I guess anything is possible. > If this happens even semi-reliably, can you put a debug build of the > client (and, if possible, the server) in, to get more details on what's > going on? Yes, I KeepAliveTime does not solve my problem I will do as suggest. > Is it always the same file that's causing the problem? No, but i do have rather big files here 300MB+ Thanks for all your help, Tobias From boxbackup at fluffy.co.uk Mon May 22 18:33:14 2006 From: boxbackup at fluffy.co.uk (Per Thomsen) Date: Mon, 22 May 2006 10:33:14 -0700 Subject: [Box Backup] Re: Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <44718F3A.5050200@kontrapunkt.com> References: <20060519053800.10903.18471.Mailman@love.warhead.org.uk> <44718F3A.5050200@kontrapunkt.com> Message-ID: <4471F5DA.8090801@reedtz.com> On 5/22/06 3:15 AM, Tobias Balle-Petersen wrote: > Hello... > > Thank you for helping out. > > Chris: > Thank you for pointing out the KeepAliveTime parameter which I did not > know about. > > As you suggested, I have added this line to my bbackupd.conf file: > KeepAliveTime = 30 > > I will post the results to the list. > > Per: >> What quality is the connection to the datacenter? >> Could there be short outages? > > Hmmm.... I dont really know what to answer here, I guess anything is > possible. Yeah, that was a bit of a vague question... Maybe you can do a long running ping-test, to ensure that you don't have any packet loss. >From the client machine: ping -n -c 1000 -i 1 If you have no packet loss, chances are that the link is OK, and probably isn't an issue. > > >> If this happens even semi-reliably, can you put a debug build of the >> client (and, if possible, the server) in, to get more details on what's >> going on? > > Yes, I KeepAliveTime does not solve my problem I will do as suggest. OK. Will wait for the logs. > > >> Is it always the same file that's causing the problem? > > No, but i do have rather big files here 300MB+ OK. Anything over 2G? Just wondering about the large-file support. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Tue May 23 22:32:43 2006 From: boxbackup at fluffy.co.uk (patrick machielse) Date: Tue, 23 May 2006 23:32:43 +0200 Subject: [Box Backup] Locations not backed up when location name contains space Message-ID: Hi all, This location is never backed up: Volumes-Opslag-backup test- { Path = /Volumes/Opslag/backup test } It seems that bbackupd (I'm using 0.10) will not backup any location that has a space character in its name. If I change the name to 'Volumes-Opslag-backup~test-' (or any other name without spaces) bbackup will accept the location and perform the backup. bbackupd does not produce an error or a warning on the console. I'm on Mac OS X 10.4.6 patrick From boxbackup at fluffy.co.uk Tue May 23 23:42:41 2006 From: boxbackup at fluffy.co.uk (Per Thomsen) Date: Tue, 23 May 2006 15:42:41 -0700 Subject: [Box Backup] Locations not backed up when location name contains space In-Reply-To: References: Message-ID: <44738FE1.9010007@reedtz.com> On 5/23/06 2:32 PM, patrick machielse wrote: > Hi all, > > This location is never backed up: > > Volumes-Opslag-backup test- > { > Path = /Volumes/Opslag/backup test > } I tried this on Linux (0.09) and you're right. It appears to be a bug in the parsing code for the config file, but a cursory look at the (0.10) code didn't reveal anything to my untrained eye. It seems like the parser takes everything, including spaces, into the value part of the data structure that holds the configuration info. I'll build a debug version of 0.10 later today, and will test a few different things, to see if I can see where the problem is occurring. Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Wed May 24 07:10:27 2006 From: boxbackup at fluffy.co.uk (Per Thomsen) Date: Tue, 23 May 2006 23:10:27 -0700 Subject: [Box Backup] Locations not backed up when location name contains space In-Reply-To: <44738FE1.9010007@reedtz.com> References: <44738FE1.9010007@reedtz.com> Message-ID: <4473F8D3.10309@reedtz.com> On 5/23/06 3:42 PM, Per Thomsen wrote: > On 5/23/06 2:32 PM, patrick machielse wrote: > >> Hi all, >> >> This location is never backed up: >> >> Volumes-Opslag-backup test- >> { >> Path = /Volumes/Opslag/backup test >> } >> > I tried this on Linux (0.09) and you're right. It appears to be a bug in > the parsing code for the config file, but a cursory look at the (0.10) > code didn't reveal anything to my untrained eye. It seems like the > parser takes everything, including spaces, into the value part of the > data structure that holds the configuration info. > > I'll build a debug version of 0.10 later today, and will test a few > different things, to see if I can see where the problem is occurring. > Having a backup location with spaces in the name works for me in 0.10: ================================= bbackupd.conf: . . BackupLocations { home-pthomsen { Path = /home/pthomsen } test { Path = /tmp/test dir } dir 2 { Path = /tmp/dir 2 } } . . . ================================= # ls -l /tmp/dir\ 2 total 4 -rw-r--r-- 1 root root 0 May 23 22:34 blech # =================================== >From bbackupquery . . query > ls 00000002 -d---- home-pthomsen 00000003 -d---- test 00002698 -d---- dir 2 query > cd "dir 2" query > ls 0000269b f----- blech query > quit ==================================== Do you see the problematic BackupLocation in the top-level of the store (ls from the bbackupquery prompt)? Can you post the relevant section of bbackupd.conf (BackupLocations), and the output of 'bbackupquery ls quit'? Thanks, Per -- Per Reedtz Thomsen | Reedtz Consulting, LLC | F: 209 883 4119 V: 209 883 4102 | pthomsen at reedtz.com | C: 209 996 9561 GPG ID: 1209784F | Yahoo! Chat: pthomsen | AIM: pthomsen From boxbackup at fluffy.co.uk Wed May 24 12:46:50 2006 From: boxbackup at fluffy.co.uk (patrick machielse) Date: Wed, 24 May 2006 13:46:50 +0200 Subject: [Box Backup] Locations not backed up when location name contains space [solved] In-Reply-To: <20060524110001.1960.94300.Mailman@love.warhead.org.uk> References: <20060524110001.1960.94300.Mailman@love.warhead.org.uk> Message-ID: Op 24-mei-2006, om 13:00 heeft Per Thomsen het volgende geschreven: > Do you see the problematic BackupLocation in the top-level of the > store > (ls from the bbackupquery prompt)? > > Can you post the relevant section of bbackupd.conf (BackupLocations), > and the output of 'bbackupquery ls quit'? Thanks for the feedback Per. It turns out that it was my mistake after all (or rather: to begin with). I'm controlling bbackupquery from a custom application and I wasn't properly quoting the location name... Your comments made me re-examine my assumptions about my own code. After that, finding the bug was easy ;-) A humbler, patrick machielse From boxbackup at fluffy.co.uk Thu May 25 12:33:07 2006 From: boxbackup at fluffy.co.uk (Imre Gergely) Date: Thu, 25 May 2006 14:33:07 +0300 Subject: [Box Backup] specify source ip Message-ID: <447595F3.1020901@astral.ro> hi i was wondering if one can specify the source ip address to the client (in the config), when connecting to a server? i mean the ip address with which the client will connect to the given server. thx. From boxbackup at fluffy.co.uk Thu May 25 17:24:49 2006 From: boxbackup at fluffy.co.uk (Stefan Norlin) Date: Thu, 25 May 2006 18:24:49 +0200 Subject: [Box Backup] specify source ip References: <447595F3.1020901@astral.ro> Message-ID: <005b01c68017$bea64a50$0101050a@stenorhem> > i was wondering if one can specify the source ip address to the client (in > the > config), when connecting to a server? i mean the ip address with which the > client will connect to the given server. I suppose you mean if it is possible to specify which "source" IP address to bind to on a box with multiple interfaces? That is currently not possible. Regards, Stefab From boxbackup at fluffy.co.uk Fri May 26 12:40:38 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Fri, 26 May 2006 13:40:38 +0200 Subject: [Box Backup] Re: Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 Message-ID: <4476E936.3090801@kontrapunkt.com> Hello... After having set KeepAliveTime = 30 in the client conf (This is a parameter is only used for the client right?) the backup ran without errors for a few days. Now, however, its back to its old ways as seen in the log files below. I have now changed to KeepAliveTime = 60. I will let it run like this for the weekend. On the client, I don't think I have seen this before (from the log below): SSL err during Write: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry The log says "reset state and waiting to retry...", but bbackupd NEVER retries?? Thanks, Tobias Client: May 24 18:22:27 yoiko bbackupd[374]: Send StoreFile(0x1992a8,0x414836a07a5c0,0x12ff1342c142275d,0x0,"BB.oph?ngningsplan.indd") May 24 18:22:27 yoiko bbackupd[374]: Sending stream, size uncertain May 24 18:22:29 yoiko bbackupd[374]: Receive Success(0x1992aa) May 24 18:22:29 yoiko bbackupd[374]: Send StoreFile(0x1992a8,0x41482f543ae00,0xb8f8eaaf0dc2384c,0x0,"version17 OK.indd") May 24 18:22:29 yoiko bbackupd[374]: Sending stream, size uncertain May 24 18:25:09 yoiko bbackupd[374]: Receive Success(0x1992ab) May 24 18:25:09 yoiko bbackupd[374]: Send StoreFile(0x1992a8,0x4148353614ec0,0x26aa104c3bf83579,0x0,"version18 OK.indd") May 24 18:25:09 yoiko bbackupd[374]: Sending stream, size uncertain May 24 18:27:54 yoiko bbackupd[374]: Send SetClientStoreMarker(0x4148b37adf680) May 24 18:27:54 yoiko bbackupd[374]: SSL err during Write: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry May 24 18:27:54 yoiko bbackupd[374]: Exception caught (Connection TLSWriteFailed (Probably a network issue between client and server.) 7/33), reset state and waiting to retry... Server: May 24 18:22:27 maigo bbstored[375]: Receive CreateDirectory(0x195729,0x4148935f275c0,OPAQUE) May 24 18:22:27 maigo bbstored[375]: Receive CreateDirectory(0x195729,0x4148935f275c0,OPAQUE) May 24 18:22:27 maigo bbstored[375]: Receiving stream, size 49 May 24 18:22:27 maigo bbstored[375]: Send Success(0x1992a8) May 24 18:22:27 maigo bbstored[375]: Send Success(0x1992a8) May 24 18:22:27 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x41482fab03ac0,0xcda9eb427c7c9c49,0x0,OPAQUE) May 24 18:22:27 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x41482fab03ac0,0xcda9eb427c7c9c49,0x0,OPAQUE) May 24 18:22:27 maigo bbstored[375]: Receiving stream, size uncertain May 24 18:22:27 maigo bbstored[375]: Send Success(0x1992a9) May 24 18:22:27 maigo bbstored[375]: Send Success(0x1992a9) May 24 18:22:27 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x414836a07a5c0,0x12ff1342c142275d,0x0,OPAQUE) May 24 18:22:27 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x414836a07a5c0,0x12ff1342c142275d,0x0,OPAQUE) May 24 18:22:28 maigo bbstored[375]: Receiving stream, size uncertain May 24 18:22:29 maigo bbstored[375]: Send Success(0x1992aa) May 24 18:22:29 maigo bbstored[375]: Send Success(0x1992aa) May 24 18:22:29 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x41482f543ae00,0xb8f8eaaf0dc2384c,0x0,OPAQUE) May 24 18:22:29 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x41482f543ae00,0xb8f8eaaf0dc2384c,0x0,OPAQUE) May 24 18:22:29 maigo bbstored[375]: Receiving stream, size uncertain May 24 18:25:09 maigo bbstored[375]: Send Success(0x1992ab) May 24 18:25:09 maigo bbstored[375]: Send Success(0x1992ab) May 24 18:25:09 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x4148353614ec0,0x26aa104c3bf83579,0x0,OPAQUE) May 24 18:25:09 maigo bbstored[375]: Receive StoreFile(0x1992a8,0x4148353614ec0,0x26aa104c3bf83579,0x0,OPAQUE) May 24 18:25:09 maigo bbstored[375]: Receiving stream, size uncertain May 24 18:25:47 maigo bbstored[375]: Connection statistics for BACKUP-A: IN=1516485463 OUT=150212036 TOTAL=1666697499\n May 24 18:25:47 maigo bbstored[375]: in server child, exception Connection TLSReadFailed (Probably a network issue between client and server.) (7/34) -- terminating child From boxbackup at fluffy.co.uk Fri May 26 13:14:31 2006 From: boxbackup at fluffy.co.uk (Felix E. Klee) Date: Fri, 26 May 2006 14:14:31 +0200 Subject: [Box Backup] Berkeley DB: no Message-ID: <87wtc9aspk.wl%felix.klee@inka.de> I'm trying to build boxbackup 0.10, but I cannot get "configure" to find the Berkeley database: $ ls /opt/BerkeleyDB.4.4/ bin docs include lib $ ls /opt/BerkeleyDB.4.4/include/ db.h db_cxx.h $ ./configure --with-bdb-dir=/opt/BerkeleyDB.4.4 [...] checking db.h presence in /opt/BerkeleyDB.4.4/include... yes checking for db.h... no configure: no Berkeley DB version or higher found [...] Large files: no Berkeley DB: no Readline: yes Extended attributes: yes What may be the reason for the problem, and how do I solve it? -- Felix E. Klee From boxbackup at fluffy.co.uk Fri May 26 14:06:20 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Fri, 26 May 2006 14:06:20 +0100 Subject: [Box Backup] Berkeley DB: no In-Reply-To: <87wtc9aspk.wl%felix.klee@inka.de> References: <87wtc9aspk.wl%felix.klee@inka.de> Message-ID: <1148648781.8361.0.camel@avenin.ebourne.me.uk> On Fri, 2006-05-26 at 14:14 +0200, Felix E. Klee wrote: > I'm trying to build boxbackup 0.10, but I cannot get "configure" to find > the Berkeley database: > ... > What may be the reason for the problem, and how do I solve it? Send your config.log gzip/bzipped. Cheers, Martin. From boxbackup at fluffy.co.uk Fri May 26 14:19:53 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 26 May 2006 13:19:53 +0000 (GMT) Subject: [Box Backup] Re: Errors on backup - Client: 7/33, Server: 7/34 - OS X 10.4 - BB 0.10 In-Reply-To: <4476E936.3090801@kontrapunkt.com> References: <4476E936.3090801@kontrapunkt.com> Message-ID: Hi Tobias, > The log says "reset state and waiting to retry...", but bbackupd NEVER > retries?? Please could you remind me in a few weeks? I'm in the process of getting the existing unit tests up and running on Windows, then I can try to write a new one for this condition. What is your UpdateStoreInterval set to? And do you have automatic backups enabled? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Fri May 26 15:12:56 2006 From: boxbackup at fluffy.co.uk (Felix E. Klee) Date: Fri, 26 May 2006 16:12:56 +0200 Subject: [Box Backup] Berkeley DB: no In-Reply-To: <1148648781.8361.0.camel@avenin.ebourne.me.uk> References: <87wtc9aspk.wl%felix.klee@inka.de> <1148648781.8361.0.camel@avenin.ebourne.me.uk> Message-ID: <87versc1sn.wl%felix.klee@inka.de> At Fri, 26 May 2006 14:06:20 +0100, Martin Ebourne wrote: > Send your config.log [...] That's the keyword. :-) I just looked at it, and found the error: configure:5784: checking for db.h configure:5819: g++ -o conftest -g -O2 -I/opt/BerkeleyDB.4.4/include -L/opt/BerkeleyDB.4.4/lib conftest.cc -lz -ledit >&5 configure:5822: $? = 0 configure:5824: ./conftest ./conftest: error while loading shared libraries: libedit.so.0: cannot open shared object file: No such file or directory configure:5827: $? = 127 After running "ldconfig", it failed again, but this time because it couldn't find the freshly installed Berkeley DB library. So, I edited "/etc/ld.so.conf", ran "ldconfig" again, and finally "configure" finds everything. -- Felix E. Klee From boxbackup at fluffy.co.uk Fri May 26 15:35:34 2006 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Fri, 26 May 2006 16:35:34 +0200 Subject: [Box Backup] Boxbackup Wiki abuse? Message-ID: <44771236.3000509@trexler.at> Hi, I just made a minor update on the following Wiki Page http://boxbackup.hostworks.ca/index.php/Checking_the_storage_space_used_on_the_server#On_the_server and found the following text containing hundreds of links at the bottom of the edit box. If you look at the html-source these hidden links seem to be on every page in the wiki... br Wolfgang
[We are delicate. We do not delete your content.] [l_sp1] [http://ringtonemaker.blogs.eurosport.com/ ringtone maker] [http://verizonringtone.forumco.com/ verizon ringtone] [http://uscellularringtone.forumco.com US Cellular Ringtone] [http://blog.investing.com/bcbgshoes/ bcbg shoes] [http://www.buddyprofile.com/viewprofile.php?username=waterfordcrystal waterford crystal] [http://www.buddyprofile.com/viewprofile.php?username=swarovskicrystal swarovski crystal bead] [http://www.buddyprofile.com/viewprofile.php?username=mesotheliomalawsuits mesothelioma lawsuits] [http://www.buddyprofile.com/viewprofile.php?username=mesotheliomasymptoms mesothelioma symptoms] [http://www.buddyprofile.com/viewprofile.php?username=mesotheliomadiag mesothelioma diagnosis] ... ... ... [deleted hundreds of these lines...] From boxbackup at fluffy.co.uk Fri May 26 15:41:07 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 26 May 2006 14:41:07 +0000 (GMT) Subject: [Box Backup] Boxbackup Wiki abuse? In-Reply-To: <44771236.3000509@trexler.at> References: <44771236.3000509@trexler.at> Message-ID: Hi Wolfgang, > I just made a minor update on the following Wiki Page > http://boxbackup.hostworks.ca/index.php/Checking_the_storage_space_used_on_the_server#On_the_server > and found the following text containing hundreds of links at the bottom > of the edit box. If you look at the html-source these hidden links seem > to be on every page in the wiki... Yes, I've seen it too, I think we all have. The question is what we can do about it, without closing our wiki to new registrations, which makes it much less useful. I haven't hacked mediawiki but perhaps we can make it request a password from new users, that they can get from our website or from the mailing list? Otherwise we could just live with it, since it doesn't actually damage our content. Like spam, it may be a fact of life in open communities. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Fri May 26 15:48:08 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Fri, 26 May 2006 15:48:08 +0100 Subject: [Box Backup] Boxbackup Wiki abuse? In-Reply-To: References: <44771236.3000509@trexler.at> Message-ID: <1148654888.8361.5.camel@avenin.ebourne.me.uk> On Fri, 2006-05-26 at 14:41 +0000, Chris Wilson wrote: > Hi Wolfgang, > > > I just made a minor update on the following Wiki Page > > http://boxbackup.hostworks.ca/index.php/Checking_the_storage_space_used_on_the_server#On_the_server > > and found the following text containing hundreds of links at the bottom > > of the edit box. If you look at the html-source these hidden links seem > > to be on every page in the wiki... > > Yes, I've seen it too, I think we all have. The question is what we can do > about it, without closing our wiki to new registrations, which makes it > much less useful. > > I haven't hacked mediawiki but perhaps we can make it request a password > from new users, that they can get from our website or from the mailing > list? > > Otherwise we could just live with it, since it doesn't actually damage our > content. Like spam, it may be a fact of life in open communities. You could give the password either in a graphic (quite common now) or even just in the text - bots can't read! If mediawiki allows this it would be well worthwhile. Cheers, Martin. From boxbackup at fluffy.co.uk Fri May 26 17:31:53 2006 From: boxbackup at fluffy.co.uk (Brian Burton) Date: Fri, 26 May 2006 12:31:53 -0400 Subject: [Box Backup] Berkeley DB: no In-Reply-To: <87wtc9aspk.wl%felix.klee@inka.de> References: <87wtc9aspk.wl%felix.klee@inka.de> Message-ID: <44772D79.8080207@users.sourceforge.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Felix E. Klee wrote: > $ ./configure --with-bdb-dir=/opt/BerkeleyDB.4.4 > [...] > checking db.h presence in /opt/BerkeleyDB.4.4/include... yes > checking for db.h... no > configure: no Berkeley DB version or higher found > [...] > Large files: no > Berkeley DB: no > Readline: yes > Extended attributes: yes > > What may be the reason for the problem, and how do I solve it? This may not be what is happening for you, but I encountered a similar issue a few days ago. Looking in the config.log I saw that the real problem was I didn't have LD_LIBRARY_PATH set properly so the test program compiled by configure failed to run. I added the bdb lib dir to /etc/ls.so.conf and ran ld.config and then configure was able to find bdb without problems. All the best, ++Brian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQEVAwUBRHcteTxRyEoJfXIFAQIdBwf8CLIkfdOpOIAn3RMPZy1L6CvIbBTHHfRJ EZqorZ3v9cXKcWPCIAKz4PKcKyO0s08ekWksFWBWo3d6SFS6iL6BclI4c78RRkYZ TV3K5YkHvrwfBa+kOJ2ro/A/YqdAnacePTQZ4JehT22tUkOCWNEA79RP3Hp5gh7s P2rGt30Wn7O4xZbDBbfxzS5LZ/6Vg3MPXzfpWSMvmglOQmD3NgdYP+78edOwezmB 12JZF8R/khAn5/CaZy2IkAOyNBLIM5l0MGgaE3S7wQzyyrsal1yURT0UaLSLSkFR MNi9fU4zNLeR7OQalSypjWLcz1MOHveeU0hCqgI6g0IWHSVZkaul2w== =FmPH -----END PGP SIGNATURE----- From boxbackup at fluffy.co.uk Sat May 27 17:32:36 2006 From: boxbackup at fluffy.co.uk (Felix E. Klee) Date: Sat, 27 May 2006 18:32:36 +0200 Subject: [Box Backup] BoxBackup services? Message-ID: <87lksno2cb.wl%felix.klee@inka.de> Is there any provider that offers access to a backup server which runs BoxBackup? -- Felix E. Klee From boxbackup at fluffy.co.uk Sat May 27 21:51:22 2006 From: boxbackup at fluffy.co.uk (Valhalla) Date: Sat, 27 May 2006 22:51:22 +0200 Subject: [Box Backup] Easiest Linux version for BackupBox In-Reply-To: <20060527183701.678.49560.Mailman@love.warhead.org.uk> Message-ID: <446A36BB00BBA949@vsmtp3.tin.it> (added by postmaster@virgilio.it) Hi!I don't know very well Linux.. Witch version do you think I could install for create a backup server in a Windows LAN?? (the easiest version of Linux with all modules for use and manage BackupBox...) Thanks From boxbackup at fluffy.co.uk Sun May 28 02:36:28 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 28 May 2006 01:36:28 +0000 (GMT) Subject: [Box Backup] Easiest Linux version for BackupBox In-Reply-To: <446A36BB00BBA949@vsmtp3.tin.it> (added by postmaster@virgilio.it) References: <446A36BB00BBA949@vsmtp3.tin.it> (added by postmaster@virgilio.it) Message-ID: Hi Valhalla, On Sat, 27 May 2006, Valhalla wrote: > Hi!I don't know very well Linux.. Witch version do you think I could install > for create a backup server in a Windows LAN?? (the easiest version of Linux > with all modules for use and manage BackupBox...) I think Box Backup server should run fine on all the major distributions, or else we'd need to fix it. So it comes down to choice between the other facilities that the distributions offer. If you're familiar with any of the mainstream distributions, I'd suggest that you use that. Otherwise, I'd suggest Debian for a server. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 28 02:29:17 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 28 May 2006 01:29:17 +0000 (GMT) Subject: [Box Backup] BoxBackup services? In-Reply-To: <87lksno2cb.wl%felix.klee@inka.de> References: <87lksno2cb.wl%felix.klee@inka.de> Message-ID: Hi Felix, On Sat, 27 May 2006, Felix E. Klee wrote: > Is there any provider that offers access to a backup server which runs > BoxBackup? I would gladly do so. Please contact me off-list to discuss your needs. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 28 14:05:09 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sun, 28 May 2006 06:05:09 -0700 (PDT) Subject: [Box Backup] BoxBackup services? In-Reply-To: Message-ID: <20060528130509.22785.qmail@web60618.mail.yahoo.com> Hi Chris and Ben, Would it be appropriate to start a wiki page of those of us offering services? I'd be glad to build it in the image of: http://www.ubuntu.com/support/marketplace Thanks, Pete --- Chris Wilson wrote: > Hi Felix, > > On Sat, 27 May 2006, Felix E. Klee wrote: > > > Is there any provider that offers access to a backup server which > runs > > BoxBackup? > > I would gladly do so. Please contact me off-list to discuss your > needs. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Sun May 28 14:27:02 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sun, 28 May 2006 06:27:02 -0700 (PDT) Subject: [Box Backup] Easiest Linux version for BackupBox In-Reply-To: <446A36BB00BBA949@vsmtp3.tin.it> (added by postmaster@virgilio.it) Message-ID: <20060528132702.90587.qmail@web60611.mail.yahoo.com> Well, your first option should be to use what the main developer, Ben Summers, is deploying on, OpenBSD. I am using Box Backup 0.10 on the bleeding edge Ubuntu Dapper 6.06 betas (a Debian derivative): http://www.ubuntu.com/news/dapperrc I was running Box Backup 0.09 on Ubuntu Breezy 5.10 for several months: http://www.ubuntu.com/download The only problem with Ubuntu is that they are quite strict about not including closed software (Acrobat, Flash, Java, mp3, MS fonts, etc), but at the same time, the community has made it very easy to add those things to your system: https://wiki.ubuntu.com/RestrictedFormats (EasyUbuntu and Automatix are awesome automaters of installing these things:http://easyubuntu.freecontrib.org/ http://www.ubuntuforums.org/showthread.php?t=114251 http://www.ubuntuforums.org/showthread.php?t=138405 Not sure of their status at the moment.) If you are running it simply as a server (no gui) and not a desktop, then that should not be a concern. Instructions for installing on a Debian derivative (please submit corrections): http://boxbackup.hostworks.ca/index.php/Compiling_and_Installing_on_Debian_GNU/Linux Good luck! Pete --- Valhalla wrote: > Hi!I don't know very well Linux.. Witch version do you think I could > install > for create a backup server in a Windows LAN?? (the easiest version of > Linux > with all modules for use and manage BackupBox...) > > > Thanks > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Sun May 28 14:27:32 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 28 May 2006 13:27:32 +0000 (GMT) Subject: [Box Backup] BoxBackup services? In-Reply-To: <20060528130509.22785.qmail@web60618.mail.yahoo.com> References: <20060528130509.22785.qmail@web60618.mail.yahoo.com> Message-ID: Hi Pete, > Would it be appropriate to start a wiki page of those of us offering > services? I'd be glad to build it in the image of: > http://www.ubuntu.com/support/marketplace Fine by me, but I'm not the ultimate arbiter, Ben is. I hope that if it's an informational Wiki page (not advertising) and does not show favouritism to any one provider (e.g. list in alphabetical order, or date added), then it should be fine. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 28 14:50:59 2006 From: boxbackup at fluffy.co.uk (dave bamford) Date: Sun, 28 May 2006 14:50:59 +0100 Subject: [Box Backup] Web interface for bbackupquery Message-ID: <4479AAC3.6000108@logical-progress.com> Here is a very simple web page to browse your backed up files from a web page. You need a web server with PHP running on the client, and the bbackupquery executable and associated dlls if needed should be in the same folder as the web page. Only tried on a windows client running 0.10. Dave Bamford > > > > box backup query > > > //program to execute backup list command > if (!isset($_GET['wd'])){ > echo exec('bbackupquery "list -t" exit', $retval); > } > else { > echo exec('bbackupquery "list -t \"'.$_GET['wd'].'\"" exit', $retval); > } > $i=1; > While ($retval[$i] != "Session finished.") { > if ($i==9) { > echo '
href="index.php?wd='.substr($_GET['wd'],0,strrpos($_GET['wd'],"/")).'">..'; > } > if (substr($retval[$i],10,1) == 'd'){ > echo '
href="index.php?wd='.$_GET['wd'].'/'.substr($retval[$i],36).'">'.substr($retval[$i],36).''; > } > else { > echo "
".$retval[$i]; > } > $i++; > } > ?> > From boxbackup at fluffy.co.uk Sun May 28 17:13:36 2006 From: boxbackup at fluffy.co.uk (Felix E. Klee) Date: Sun, 28 May 2006 18:13:36 +0200 Subject: [Box Backup] BoxBackup services? In-Reply-To: References: <87lksno2cb.wl%felix.klee@inka.de> Message-ID: <87ejyenn4f.wl%felix.klee@inka.de> At Sun, 28 May 2006 01:29:17 +0000 (GMT), Chris Wilson wrote: > > Is there any provider that offers access to a backup server which > > runs BoxBackup? > > I would gladly do so. Please contact me off-list to discuss your > needs. Well, there's a crux that I didn't mention: The files that I want to backup are part of an overlay file system. This means: some files are private, others are shared (virtual server infrastructure): http://www.escapebox.net/modus.html Naturally, I only want to backup the private files (ca. 50MB compared to 2GB, fluctuating). The problem is that, due to technical reasons, it's not possible to mount all shared files into their own partition. There's a little tool, though, called "ismine" that tells one if a file is private file or a shared file. Is it possible to have "boxbackup" use "ismine" to determine whether a file should be backed up or not? -- Felix E. Klee From boxbackup at fluffy.co.uk Sun May 28 19:13:05 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 28 May 2006 19:13:05 +0100 Subject: [Box Backup] BoxBackup services? In-Reply-To: References: <20060528130509.22785.qmail@web60618.mail.yahoo.com> Message-ID: <614835A7-13C7-481D-AB3B-960237475CD9@fluffy.co.uk> On 28 May 2006, at 14:27, Chris Wilson wrote: > Hi Pete, > >> Would it be appropriate to start a wiki page of those of us >> offering services? I'd be glad to build it in the image of: >> http://www.ubuntu.com/support/marketplace > > Fine by me, but I'm not the ultimate arbiter, Ben is. I hope that > if it's an informational Wiki page (not advertising) and does not > show favouritism to any one provider (e.g. list in alphabetical > order, or date added), then it should be fine. Sounds good to me. It'd certainly help overcome one of the biggest problems for an individual user, which is finding a suitable server. Ben From boxbackup at fluffy.co.uk Sun May 28 19:17:19 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 28 May 2006 19:17:19 +0100 Subject: [Box Backup] Overlay file system (was: BoxBackup services?) In-Reply-To: <87ejyenn4f.wl%felix.klee@inka.de> References: <87lksno2cb.wl%felix.klee@inka.de> <87ejyenn4f.wl%felix.klee@inka.de> Message-ID: On 28 May 2006, at 17:13, Felix E. Klee wrote: > At Sun, 28 May 2006 01:29:17 +0000 (GMT), > Chris Wilson wrote: >>> Is there any provider that offers access to a backup server which >>> runs BoxBackup? >> >> I would gladly do so. Please contact me off-list to discuss your >> needs. > > Well, there's a crux that I didn't mention: The files that I want to > backup are part of an overlay file system. This means: some files are > private, others are shared (virtual server infrastructure): > > http://www.escapebox.net/modus.html > > Naturally, I only want to backup the private files (ca. 50MB > compared to > 2GB, fluctuating). The problem is that, due to technical reasons, > it's > not possible to mount all shared files into their own partition. > There's a little tool, though, called "ismine" that tells one if a > file > is private file or a shared file. > > Is it possible to have "boxbackup" use "ismine" to determine whether a > file should be backed up or not? It would be a quick modification to the code to do this, and you could use the existing infrastructure for file selection. (See the ExcludeList class.) However, surely you know which bits of the file system you have modified and can therefore point bbackupd at those critical bits? Ben From boxbackup at fluffy.co.uk Sun May 28 23:41:58 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 28 May 2006 22:41:58 +0000 (GMT) Subject: [Box Backup] BoxBackup services? In-Reply-To: <87ejyenn4f.wl%felix.klee@inka.de> References: <87lksno2cb.wl%felix.klee@inka.de> <87ejyenn4f.wl%felix.klee@inka.de> Message-ID: Hi Felix, On Sun, 28 May 2006, Felix E. Klee wrote: > Well, there's a crux that I didn't mention: The files that I want to > backup are part of an overlay file system. This means: some files are > private, others are shared (virtual server infrastructure): > > http://www.escapebox.net/modus.html > > Naturally, I only want to backup the private files (ca. 50MB compared to > 2GB, fluctuating). The problem is that, due to technical reasons, it's > not possible to mount all shared files into their own partition. > There's a little tool, though, called "ismine" that tells one if a file > is private file or a shared file. > > Is it possible to have "boxbackup" use "ismine" to determine whether a > file should be backed up or not? Not at the moment, I'm afraid, but it wouldn't be too hard to add such a feature to the client. If Ben said he was willing to accept the result into the Box trunk, then I'd be willing to try to implement it. No promises, though. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Mon May 29 08:48:28 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Mon, 29 May 2006 08:48:28 +0100 Subject: [Box Backup] BoxBackup services? In-Reply-To: References: <87lksno2cb.wl%felix.klee@inka.de> <87ejyenn4f.wl%felix.klee@inka.de> Message-ID: <5E423BDA-D3FF-4272-9DD7-BBBE2B9AF7DE@fluffy.co.uk> On 28 May 2006, at 23:41, Chris Wilson wrote: > Hi Felix, > > On Sun, 28 May 2006, Felix E. Klee wrote: > >> Well, there's a crux that I didn't mention: The files that I want to >> backup are part of an overlay file system. This means: some files >> are >> private, others are shared (virtual server infrastructure): >> >> http://www.escapebox.net/modus.html >> >> Naturally, I only want to backup the private files (ca. 50MB >> compared to >> 2GB, fluctuating). The problem is that, due to technical reasons, >> it's >> not possible to mount all shared files into their own partition. >> There's a little tool, though, called "ismine" that tells one if a >> file >> is private file or a shared file. >> >> Is it possible to have "boxbackup" use "ismine" to determine >> whether a >> file should be backed up or not? > > Not at the moment, I'm afraid, but it wouldn't be too hard to add > such a feature to the client. If Ben said he was willing to accept > the result into the Box trunk, then I'd be willing to try to > implement it. No promises, though. I think it would have to be something a bit more generic. I suspect we'd want to add that feature previously discussed where an external script would approve all files before they're uploaded. But not to run the script for every file, because that would be horrendous, so you'd want to communicate over stdin/stdout. IOStreamGetLine and LocalProcessStream might be good classes to use, and you could reduce the impact on the rest of the code by using a class derviced from ExcludeList. Ben From boxbackup at fluffy.co.uk Mon May 29 16:54:06 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Mon, 29 May 2006 08:54:06 -0700 (PDT) Subject: [Box Backup] BoxBackup services? In-Reply-To: <614835A7-13C7-481D-AB3B-960237475CD9@fluffy.co.uk> Message-ID: <20060529155406.66765.qmail@web60618.mail.yahoo.com> Here's a first draft: http://boxbackup.hostworks.ca/index.php/Box_Backup_Marketplace Comments welcome! --- Ben Summers wrote: > > On 28 May 2006, at 14:27, Chris Wilson wrote: > > > Hi Pete, > > > >> Would it be appropriate to start a wiki page of those of us > >> offering services? I'd be glad to build it in the image of: > >> http://www.ubuntu.com/support/marketplace > Sounds good to me. It'd certainly help overcome one of the biggest > problems for an individual user, which is finding a suitable server. > > Ben > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Mon May 29 21:47:52 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 29 May 2006 20:47:52 +0000 (GMT) Subject: [Box Backup] BoxBackup services? In-Reply-To: <20060529155406.66765.qmail@web60618.mail.yahoo.com> References: <20060529155406.66765.qmail@web60618.mail.yahoo.com> Message-ID: Hi Pete, > Here's a first draft: > http://boxbackup.hostworks.ca/index.php/Box_Backup_Marketplace > Comments welcome! Looks good, thanks. I've added my own details. I'm a bit worried that if we have a lot of providers, the page will become too long and we might have to split it, or just provide a company name and link to a page on their site describing their Box Backup services. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 30 02:17:36 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Mon, 29 May 2006 18:17:36 -0700 (PDT) Subject: [Box Backup] BoxBackup services? In-Reply-To: Message-ID: <20060530011736.80098.qmail@web60611.mail.yahoo.com> Thanks, Chris. Yeah, we'll deal with that good news appropriately one way or the other then. (Note to therapist: address feelings of server inadequacy.) --- Chris Wilson wrote: > Hi Pete, > > > Here's a first draft: > > http://boxbackup.hostworks.ca/index.php/Box_Backup_Marketplace > > Comments welcome! > > Looks good, thanks. I've added my own details. I'm a bit worried that > if > we have a lot of providers, the page will become too long and we > might > have to split it, or just provide a company name and link to a page > on > their site describing their Box Backup services. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed May 31 05:27:29 2006 From: boxbackup at fluffy.co.uk (Derek Lewis) Date: Tue, 30 May 2006 20:27:29 -0800 Subject: [Box Backup] Boxbackup usage Message-ID: <006f01c6846a$897a6ee0$6401a8c0@derek> I plan on using a server for a dual purpose file server and backup system. The desired backup system would be a single hot swap drive that would be used to store files offline. This seems like a job for rsynch, though I am interested in the encryption and automation features built into boxbackup. Can boxbackup be run on the same server with the file system to be backed up? Does boxbackup handle hot swap drives? If so, does it allow for spanning a backup session across several drives? From boxbackup at fluffy.co.uk Wed May 31 06:30:06 2006 From: boxbackup at fluffy.co.uk (Peter Porter) Date: Tue, 30 May 2006 23:30:06 -0600 Subject: [Box Backup] Win32 Issues Message-ID: <447D29DE.3000507@brightavenue.com> I've been using BoxBackup happily for a few years now, and recently upgraded my server to v0.10 from v0.09 (thanks to an 'emerge -vu world' on Gentoo without noticing the Box version change). I've never had a problem building on a Linux system. I rebuilt Linux clients with v0.10 and they all work fine. I was previously running v0.09 win32 binaries without a problem, but I'm having issues compiling in Cygwin with the new v0.10 binaries -- and moving to another server. I can use my compiled bbackupquery.exe on the machine where I've compiled, but I get an error on the target machine: ---------------------------------------------------------------------------------------------- server01 box $ bbackupquery -c bbackupd.conf Box Backup Query Tool v0.10, (c) Ben Summers and contributors 2003-2006 Using configuration file bbackupd.conf Connecting to store... Handshake with store... Login to store... Login complete. Type "help" for a list of commands. query > Exception: Common OSFileError (Error accessing a file. Check permissions.) (1/9) No random device -- additional seeding of random number generator not performed. Failed to read from console: error 6 ---------------------------------------------------------------------------------------------- Any ideas as to what I'm missing, or pointing me toward a decent faq for win32/Cygwin -- specifically how to install a compiled package, and what libraries would be required (from a base Cygwin install) would be much appreciated! Thanks, Peter From boxbackup at fluffy.co.uk Wed May 31 08:41:18 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Wed, 31 May 2006 08:41:18 +0100 Subject: [Box Backup] Boxbackup usage In-Reply-To: <006f01c6846a$897a6ee0$6401a8c0@derek> References: <006f01c6846a$897a6ee0$6401a8c0@derek> Message-ID: <1149061278.10959.12.camel@avenin.ebourne.me.uk> On Tue, 2006-05-30 at 20:27 -0800, Derek Lewis wrote: > Can boxbackup be run on the same server with the file system to be backed > up? Yes that works fine, I do that on one of my machines (purely because box handles deleted files and backup retrieval much better than rsync). > Does boxbackup handle hot swap drives? If so, does it allow for spanning a > backup session across several drives? It doesn't specifically handle them, so it will behave the same as any other unix program. If you take the fs away it will probably die and have to be restarted again later. It's not possible to span the backup store across several drives (apart form the built in raid where they need to all be accessible). Cheers, Martin. From boxbackup at fluffy.co.uk Wed May 31 09:08:03 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 31 May 2006 09:08:03 +0100 Subject: [Box Backup] Boxbackup usage In-Reply-To: <1149061278.10959.12.camel@avenin.ebourne.me.uk> References: <006f01c6846a$897a6ee0$6401a8c0@derek> <1149061278.10959.12.camel@avenin.ebourne.me.uk> Message-ID: On 31 May 2006, at 08:41, Martin Ebourne wrote: > On Tue, 2006-05-30 at 20:27 -0800, Derek Lewis wrote: >> Can boxbackup be run on the same server with the file system to be >> backed >> up? > > Yes that works fine, I do that on one of my machines (purely > because box > handles deleted files and backup retrieval much better than rsync). Although if they're on the same hard disc, you don't protect against hardware failure or other whole disc problems. > >> Does boxbackup handle hot swap drives? If so, does it allow for >> spanning a >> backup session across several drives? > > It doesn't specifically handle them, so it will behave the same as any > other unix program. If you take the fs away it will probably die and > have to be restarted again later. > > It's not possible to span the backup store across several drives > (apart > form the built in raid where they need to all be accessible). Use hardware RAID if you want to handle discs in interesting ways. The userland RAID cannot rebuilt currently, if a disc dies you can only recover data. Ben From boxbackup at fluffy.co.uk Wed May 31 09:22:34 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 31 May 2006 08:22:34 +0000 (GMT) Subject: [Box Backup] Win32 Issues In-Reply-To: <447D29DE.3000507@brightavenue.com> References: <447D29DE.3000507@brightavenue.com> Message-ID: Hi Peter, > I've been using BoxBackup happily for a few years now, and recently > upgraded my server to v0.10 from v0.09 (thanks to an 'emerge -vu world' > on Gentoo without noticing the Box version change). I've never had a > problem building on a Linux system. I rebuilt Linux clients with v0.10 > and they all work fine. > > I was previously running v0.09 win32 binaries without a problem, but I'm > having issues compiling in Cygwin with the new v0.10 binaries -- and > moving to another server. Sorry, but I don't think BB 0.10 works properly on Cygwin. A number of the unit tests fail. Please consider using the Windows native client, which you can find at [http://boxbackup.hostworks.ca/index.php/0.10#Windows_Client]. You will probably have to change your configuration file, since all paths should use "\" as the path separator, and "/cygdrive/c" should be changed to "C:\". Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 31 15:48:19 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 31 May 2006 16:48:19 +0200 Subject: [Box Backup] Compiling debug build of bbackupd on OS X In-Reply-To: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> References: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> Message-ID: <447DACB3.7090508@kontrapunkt.com> Hello.... I'm trying to resolve some problems I'm having with box backup by compiling a debug build of bbackupd as suggested on this list. I have extracted a fresh copy of the 0.10 sources. The wiki says: cd boxbackup-0.0 cd bin/bbstoreaccounts make In directory /Users/kpadmin/Desktop/boxbackup-0.10/bin/bbackupd I get: make: *** No targets specified and no makefile found. Stop. How do I compile? Thanks, Tobias From boxbackup at fluffy.co.uk Wed May 31 15:49:52 2006 From: boxbackup at fluffy.co.uk (Peter Porter) Date: Wed, 31 May 2006 08:49:52 -0600 Subject: [Box Backup] Win32 Issues In-Reply-To: References: <447D29DE.3000507@brightavenue.com> Message-ID: <447DAD10.7090903@brightavenue.com> I've tried the Windows native client with the same lack of avail. I get a handshake error, and I am certain that the certificates and key used are the same. My session looks like this: server01 box3 $ ./bbackupquery.exe -c bbackupd.conf Box Backup Query Tool v0.10_rc0.1_win32, (c) Ben Summers 2003, 2004 Using configuration file bbackupd.conf Connecting to store... Handshake with store... Exception: Connection Protocol_HandshakeFailed (7/48) No random device -- additional seeding of random number generator not performed. What unit tests fail on Cygwin? It seems as though bbackupquery works, it simply has a problem reading from the console. If I provide commands as arguments, I can see that they've executed. Doesn't a build on Cygwin with MinGW result in the native Win32 build? I built following the instructions in "win32_build_on_cygwin_using_mingw.txt", it would seem to me that I am building the native Win32 client. When I try to do a standard configure and build under Cygwin, I get errors that it cannot determine mount points on my system. -Peter Chris Wilson wrote: > Hi Peter, > >> I've been using BoxBackup happily for a few years now, and recently >> upgraded my server to v0.10 from v0.09 (thanks to an 'emerge -vu >> world' on Gentoo without noticing the Box version change). I've >> never had a problem building on a Linux system. I rebuilt Linux >> clients with v0.10 and they all work fine. >> >> I was previously running v0.09 win32 binaries without a problem, but >> I'm having issues compiling in Cygwin with the new v0.10 binaries -- >> and moving to another server. > > Sorry, but I don't think BB 0.10 works properly on Cygwin. A number of > the unit tests fail. > > Please consider using the Windows native client, which you can find at > [http://boxbackup.hostworks.ca/index.php/0.10#Windows_Client]. > > You will probably have to change your configuration file, since all > paths should use "\" as the path separator, and "/cygdrive/c" should > be changed to "C:\". > > Cheers, Chris. -- Peter Porter ******Bright Avenue, managed web for new horizons******* *http://www.brightavenue.com (866) 200-3263 peter at brightavenue.com From boxbackup at fluffy.co.uk Wed May 31 15:52:15 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 31 May 2006 15:52:15 +0100 Subject: [Box Backup] Compiling debug build of bbackupd on OS X In-Reply-To: <447DACB3.7090508@kontrapunkt.com> References: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> <447DACB3.7090508@kontrapunkt.com> Message-ID: <14F9FE83-2AE4-43EA-A663-6DE417F9A563@fluffy.co.uk> On 31 May 2006, at 15:48, Tobias Balle-Petersen wrote: > Hello.... > > I'm trying to resolve some problems I'm having with box backup by > compiling a debug build of bbackupd as suggested on this list. I > have extracted a fresh copy of the 0.10 sources. > > The wiki says: > cd boxbackup-0.0 > cd bin/bbstoreaccounts > make > > > In directory /Users/kpadmin/Desktop/boxbackup-0.10/bin/bbackupd I get: > make: *** No targets specified and no makefile found. Stop. > > > > How do I compile? Run ./configure in the distribution root first. Ben From boxbackup at fluffy.co.uk Wed May 31 16:07:12 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 31 May 2006 17:07:12 +0200 Subject: [Box Backup] Re.: Compiling debug build of bbackupd on OS X In-Reply-To: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> References: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> Message-ID: <447DB120.4070101@kontrapunkt.com> Hello.... I ran ./configure in the top directory to create makefiles in the dir/* directories. Now when I try to run make in bin/bbackupd I get: Makefile:9: *** missing separator. Stop. Any ideas? Thanks From boxbackup at fluffy.co.uk Wed May 31 16:14:42 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 31 May 2006 16:14:42 +0100 Subject: [Box Backup] Re.: Compiling debug build of bbackupd on OS X In-Reply-To: <447DB120.4070101@kontrapunkt.com> References: <20060531110001.25174.64194.Mailman@love.warhead.org.uk> <447DB120.4070101@kontrapunkt.com> Message-ID: <361D1D8B-5481-44AA-9E20-994C69AC0579@fluffy.co.uk> On 31 May 2006, at 16:07, Tobias Balle-Petersen wrote: > Hello.... > > I ran ./configure in the top directory to create makefiles in the > dir/* directories. Now when I try to run make in bin/bbackupd I get: > > Makefile:9: *** missing separator. Stop. > > > Any ideas? As instructed by ./configure, use bsdmake not make on OS X. Ben From boxbackup at fluffy.co.uk Wed May 31 16:51:25 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 31 May 2006 15:51:25 +0000 (GMT) Subject: [Box Backup] Win32 Issues In-Reply-To: <447DAD10.7090903@brightavenue.com> References: <447D29DE.3000507@brightavenue.com> <447DAD10.7090903@brightavenue.com> Message-ID: Hi Peter, I think there may be some confusion here. When you referred to "compiling in Cygwin with the new v0.10 binaries," did you mean: 1. following the instructions in "win32_build_on_cygwin_using_mingw.txt" (resulting in a native build); or 2. just "./configure; make" (resulting in a cygwin build)? > I've tried the Windows native client with the same lack of avail. I get > a handshake error, and I am certain that the certificates and key used > are the same. The expected behaviour depends on which build you actually made. If you built from my branch, revision 568, following the instructions in win32_build_on_cygwin_using_mingw.txt, then your build should behave in exactly the same way as mine. If not, then please identify what you did differently. If you put the same keys and configuration (modulo path separators) onto a Unix client, does it work? > What unit tests fail on Cygwin? It seems as though bbackupquery works, > it simply has a problem reading from the console. If I provide commands > as arguments, I can see that they've executed. Sorry, I don't remember exactly. You can run the tests yourself with "./runtests ALL debug". I'm not too interested in Cygwin as a platform now that we have a native build for Windows, and I don't know whether anyone else is. > Doesn't a build on Cygwin with MinGW result in the native Win32 build? No, the native build is cross-compiled and does not depend on Cygwin any more once it's compiled. It just uses cygwin as a convenient way to get the compiler, shell, perl, etc. installed on the machine. You can also do the native build using MSVC without installing Cygwin. > I built following the instructions in > "win32_build_on_cygwin_using_mingw.txt", it would seem to me that I am > building the native Win32 client. Yes, that is a native build, not a Cygwin build, assuming that you follow the instructions exactly. > When I try to do a standard configure and build under Cygwin, I get > errors that it cannot determine mount points on my system. That's probably because nobody is testing on Cygwin and it got broken. I think the standard 0.10 sources did compile on Cygwin, but I'm not 100% sure. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 31 18:07:31 2006 From: boxbackup at fluffy.co.uk (vordoo) Date: Wed, 31 May 2006 10:07:31 -0700 (PDT) Subject: [Box Backup] bbackupquery problem - cannot "cd" and browse Message-ID: <20060531170732.34447.qmail@web30415.mail.mud.yahoo.com> Hi all, BackupQuery v0.10 Debian sarge 2.4.27-2-386 GNU/Linux I get my backup dir OK, But I cannot "cd" and browse the backup tree. Directory 'home' not found. #bbackupquery query > ls 00000002 -d---- home query > cd home Directory 'home' not found query > list -r I get the full backedup file tree/list with old and deleted marked files. I have tried to change the backup directory in /etc/box/bbackupd.conf to a new one, but got the same result. Appreciate any guidance where to go from here. Thnx, Jeff __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com