From boxbackup at fluffy.co.uk Thu May 1 07:20:00 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 01 May 2008 08:20:00 +0200 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: References: <48136012.4010608@homann.se> <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> Message-ID: <48196110.6030902@homann.se> Chris Wilson wrote: > This really should not be happening. Is this all data that you copied from > the remote system, i.e. no remnants of your old corrupt local backup? Is > it possible that the local copy got corrupted while it was being rsynced? Yes, it is possible, even likely, that the archive somehow got corrupted while rsynced. > Also, were you running compare regularly on your files? When did you last > run it? It's possible that these errors were in your store for a while but > never got detected because compare wasn't run. Never run compare, I'm afraid. Looking in the mail logs for this list I did have problems with som files when I started using boxbackup. Can't remember if it got resolved. > How exactly did your local filesystem get corrupted? Was it user error or > a hardware failure? Could bad hardware have been slowly corrupting the > local copy? I think the controller HW got messed up, the two disks that it handles got corrupted. > Do you have compressed encrypted files over 2GB in the remote repository? No. > When you say 0xbfd9 is recreated, do you know by what? If it's by > bbstoreaccounts check fix, could you tell me what it outputs when it does > this? Yes, by running bbstoreaccounts check fix. I don't have a log right now, but in short. In phase 2, check diectories... a couple of hundred "File ID xxx has different container ID, probably moved". In phase 4, fix unattached objects "Recretating missing directory bfd9" amd the followed by thousands(?) of "Object xyz is unattached". After phase 5 and 6 finish without problems, I get "3225 errors found". The same thig repeats if I remove the bfd9 file from the archive and run bbstoreaccounst again. I hope this helps, or i can send yu more information later today. That soime of the fiels went missing I can live with, but I want to get as much as possible. During my operationms, the size of the archive has gone from 211 GB down to about 130 GB, is that bbstoreaccounts check fix work? -- Magnus Homann From boxbackup at fluffy.co.uk Thu May 1 23:34:20 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 1 May 2008 23:34:20 +0100 (BST) Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: <48196110.6030902@homann.se> References: <48136012.4010608@homann.se> <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> Message-ID: Hi Magnus, On Thu, 1 May 2008, Magnus Homann wrote: > Chris Wilson wrote: > > > This really should not be happening. Is this all data that you copied from > > the remote system, i.e. no remnants of your old corrupt local backup? Is it > > possible that the local copy got corrupted while it was being rsynced? > > Yes, it is possible, even likely, that the archive somehow got corrupted while > rsynced. [...] > > How exactly did your local filesystem get corrupted? Was it user error or a > > hardware failure? Could bad hardware have been slowly corrupting the local > > copy? > > I think the controller HW got messed up, the two disks that it handles got > corrupted. I'm afraid if your remote copy is badly corrupted then all bets are off. About the best I can do is to make it possible to skip files with errors during restore. Box Backup is designed for efficient backups over the Internet, perhaps it might be wise to run it directly to the remote server in future? Together with regular compares, that should minimise the chances of getting corrupt data in your store, and alert you quickly to problems before they become fatal. > > When you say 0xbfd9 is recreated, do you know by what? If it's by > > bbstoreaccounts check fix, could you tell me what it outputs when it > > does this? > > Yes, by running bbstoreaccounts check fix. > > I don't have a log right now, but in short. > > In phase 2, check diectories... a couple of hundred "File ID xxx has different > container ID, probably moved". > > In phase 4, fix unattached objects "Recretating missing directory bfd9" amd > the followed by thousands(?) of "Object xyz is unattached". > > After phase 5 and 6 finish without problems, I get "3225 errors found". > > The same thig repeats if I remove the bfd9 file from the archive and run > bbstoreaccounst again. > > I hope this helps, or i can send yu more information later today. I'm very surprised that it would recreate the directory in a way that was corrupt. I'd like to investigate this. Are you available this weekend? I'd be happy to spend some time debugging this interactively with you. > That soime of the fiels went missing I can live with, but I want to get > as much as possible. During my operationms, the size of the archive has > gone from 211 GB down to about 130 GB, is that bbstoreaccounts check fix > work? It could be that it deleted some corrupt files from the store. I'm quite surprised that it would do that, though. Perhaps if you could resync from the remote repository and run bbstoreaccounts check fix again to get a complete log of the output? Or else run "bbstoreaccounts check" (without fix) on the remote copy? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 2 07:12:13 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Fri, 02 May 2008 07:12:13 +0100 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: References: <48136012.4010608@homann.se> <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> Message-ID: <481AB0BD.80002@netinertia.co.uk> Chris Wilson wrote: > Box Backup is designed for efficient backups over the Internet, perhaps it > might be wise to run it directly to the remote server in future? Together > with regular compares, that should minimise the chances of getting corrupt > data in your store, and alert you quickly to problems before they become > fatal. Do any of the Linux distributions have anything like FreeBSD's periodic(8)? This thread has prompted me to start writing a periodic script that I will include with the FreeBSD port, that will run once a month and run "bbackupquery -q 'compare -a' quit" (with the -a flag being changeable, e.g. if someone wanted to use -aq instead). Maybe in the install docs we could recommend adding something like this to root's crontab. James From boxbackup at fluffy.co.uk Fri May 2 07:57:32 2008 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Fri, 02 May 2008 07:57:32 +0100 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: <481AB0BD.80002@netinertia.co.uk> References: <48136012.4010608@homann.se> <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> <481AB0BD.80002@netinertia.co.uk> Message-ID: <481ABB5C.1030208@hickinbottom.com> Yes - there are usually directories such as the following: /etc/cron.daily /etc/cron.weekly /etc/cron.monthly You can just drop executable scripts or binaries in there and they'll be run with /approximately/ that frequency (no guarantees). Any output from those scripts/binaries is emailed to a responsible adult on the system (usually root). This is the case on at least Red Hat Enterprise and Gentoo - I can't readily check others. Stuart James O'Gorman wrote: > Chris Wilson wrote: >> Box Backup is designed for efficient backups over the Internet, >> perhaps it might be wise to run it directly to the remote server in >> future? Together with regular compares, that should minimise the >> chances of getting corrupt data in your store, and alert you quickly >> to problems before they become fatal. > > Do any of the Linux distributions have anything like FreeBSD's > periodic(8)? This thread has prompted me to start writing a periodic > script that I will include with the FreeBSD port, that will run once a > month and run "bbackupquery -q 'compare -a' quit" (with the -a flag > being changeable, e.g. if someone wanted to use -aq instead). > > Maybe in the install docs we could recommend adding something like > this to root's crontab. > > James > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Fri May 2 11:17:20 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Fri, 2 May 2008 11:17:20 +0100 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: <481ABB5C.1030208@hickinbottom.com> References: <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> <481AB0BD.80002@netinertia.co.uk> <481ABB5C.1030208@hickinbottom.com> Message-ID: <20080502101720.GA9281@netinertia.co.uk> On Fri, May 02, 2008 at 07:57:32AM +0100, Stuart Hickinbottom wrote: > Yes - there are usually directories such as the following: > > /etc/cron.daily > /etc/cron.weekly > /etc/cron.monthly > > You can just drop executable scripts or binaries in there and they'll be run > with /approximately/ that frequency (no guarantees). Any output from those > scripts/binaries is emailed to a responsible adult on the system (usually > root). > > This is the case on at least Red Hat Enterprise and Gentoo - I can't readily > check others. Marvellous. I'll try and install Fedora and maybe another distro in a VM over the weekend and adapt my FreeBSD script to the Linux way. I'll commit it to contrib/ when it's done. James From boxbackup at fluffy.co.uk Fri May 2 14:26:23 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Fri, 02 May 2008 15:26:23 +0200 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: References: <48136012.4010608@homann.se> <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> Message-ID: <481B167F.90802@homann.se> Good news and bad news. I got it rsynched and worked around the exceptions with a little help of bash scripting (scary...) restoring directories one by one, skipping the offending ones. So, after a couple of days work I got all files restored back to something that looked complete. Started Samba, synched with some files on my laptop and modifed the soft/hard limmts to remove all old and deleted files from the archive (shrinking from 211 GB to about 40 GB) during night. In the morning, my disk was corrupt again, and most of the files and most of the backup stored locally was gone. Back to square -1. I have now bought a new internal disk, in case that's the culprit, and also bought an USB disk which as we speak is at my friends house filling up with the 211 GB of his copy of the backup store. The idea is to bring it back, copy it's content to my disk, remove the USB disk from the computer and start all over. At least I now know how to work around some of the exception. I believe a 'force' flag to restore would be an excellent idea, so that it is possible to continue restoring after one of the stored files is corrupted. I was stupid enough to not take a separate backup of the newly restored file to another disk last night, but smart/lucky enough to sync the most important files to my laptop. I'm off for the weekend, I'll let you know what happens Magnus From boxbackup at fluffy.co.uk Fri May 2 15:36:44 2008 From: boxbackup at fluffy.co.uk (Richard Hurt) Date: Fri, 2 May 2008 10:36:44 -0400 Subject: [Box Backup] Certificates & Keys Question Message-ID: <712ba87c0805020736w5d663c2ej5411df8674b45478@mail.gmail.com> I am trying to make sure my systems are secure and I would like to check my assumptions and perhaps update the Wiki with the final information. Please let me know where my assumptions go wrong and if I'm missing something. Thanx! Richard ====================== SNIP ========================= Box Backup uses SSL to communicate between the server and the client. It also uses AES to encrypt the data on the server. These technologies rely on various files (aka Keys) some of which need to be protected more than others. -csr.pem - Security Level: LOW This is merely a certificate request file and is only used once. After getting your certificate this file can|should be deleted. -cert.pem - Security Level: MEDIUM This is the clients SSL certificate and it the client to securely communicate with the server. This file is unique to each client and should be marginally protected. If a bad guy had this file he could copy your encrypted data from the server but wouldn't be able to use it. -key.pem - Security Level: ??? I'm not quite sure what this file is used for. Is it another SSL key or something to do with the AES encryption. What happens if a bad guy gets a hold of this file? What damage could he do? -FileEncKeys.raw - Security Level: HIGHEST This is the master AES key for your data and is used to encrypt your data before sending it to the server. This file is unique to each client and should be protected at all costs and stored off-site in a secure location. Without this file your data is useless. If a bad guy gets this file all bets are off and you are sunk. If you lose this file you are sunk. Everything can be replaced, except this file. serverCA.pem - Security Level: MEDIUM This is the servers SSL certificate and it allows the client to security communicate with the server. This file is common to all clients and should be marginally protected. If a bad guy had this file he could run a Man-In-The-Middle attack and impersonate your Box Backup server thereby capturing your encrypted data. From boxbackup at fluffy.co.uk Fri May 2 16:20:47 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Fri, 02 May 2008 17:20:47 +0200 Subject: [Box Backup] Certificates & Keys Question In-Reply-To: <712ba87c0805020736w5d663c2ej5411df8674b45478@mail.gmail.com> References: <712ba87c0805020736w5d663c2ej5411df8674b45478@mail.gmail.com> Message-ID: <1209741647.9239.24.camel@localhost.localdomain> fre, 02 05 2008 kl. 10:36 -0400, skrev Richard Hurt: > I am trying to make sure my systems are secure and I would like to > check my assumptions and perhaps update the Wiki with the final > information. Please let me know where my assumptions go wrong and if > I'm missing something. >=20 > Thanx! > Richard >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D SNIP = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D >=20 > Box Backup uses SSL to communicate between the server and the client. > It also uses AES to encrypt the data on the server. These > technologies rely on various files (aka Keys) some of which need to be > protected more than others. >=20 > -csr.pem > - Security Level: LOW > This is merely a certificate request file and is only used once. > After getting your certificate this file can|should be deleted. >=20 > -cert.pem > - Security Level: MEDIUM > This is the clients SSL certificate and it the client to securely > communicate with the server. This file is unique to each client and > should be marginally protected. If a bad guy had this file he could > copy your encrypted data from the server but wouldn't be able to use > it. The bad guy would need =EF=BB=BF-key.pem as well in order t= o do this. Without key.pem, he would theoretically have a small chance to be able to encrypt data on the SSL session, but he would not be able to decrypt them. And without both files, the SSL session cannot be set up. Security level for this file should be LOW, since it is public and can be regenerated at the server. >=20 > -key.pem > - Security Level: ??? > I'm not quite sure what this file is used for. Is it another SSL > key or something to do with the AES encryption. What happens if a bad > guy gets a hold of this file? What damage could he do? This is the client's private key, as opposed to =EF=BB=BF-cert.pem, which is the public part that is presented to the other side when communicating. The private key is necessary to decrypt incoming communications. A bad guy would be able to use it in conjunction with the public =EF=BB=BF-cert.pem to decrypt t= he SSL session, but not the actual data, which are encrypted using =EF=BB=BF-FileEncKeys.raw. Security level should be MEDIUM to HIGH. This file is private and is used to secure your communications, and establish identity. >=20 > -FileEncKeys.raw > - Security Level: HIGHEST > This is the master AES key for your data and is used to encrypt > your data before sending it to the server. This file is unique to > each client and should be protected at all costs and stored off-site > in a secure location. Without this file your data is useless. If a > bad guy gets this file all bets are off and you are sunk. If you lose > this file you are sunk. Everything can be replaced, except this file. >=20 > serverCA.pem > - Security Level: MEDIUM > This is the servers SSL certificate and it allows the client to > security communicate with the server. This file is common to all > clients and should be marginally protected. If a bad guy had this > file he could run a Man-In-The-Middle attack and impersonate your Box > Backup server thereby capturing your encrypted data. The server's private key would also be needed - you cannot run a MITM based on the certificate alone. As with the =EF=BB=BF-cert.= pem, this file is public. It is used, among other things, to establish the server's identity. Security can be LOW, since this is a public file, which the server can re-issue at will. Disclaimer: This is based entirely on my own reading of the SSL/TLS specifications. Others may well be more informed ;) Bjarne From boxbackup at fluffy.co.uk Fri May 2 18:10:09 2008 From: boxbackup at fluffy.co.uk (Richard Hurt) Date: Fri, 2 May 2008 13:10:09 -0400 Subject: [Box Backup] Wiki Account Request Message-ID: <712ba87c0805021010i26688b14x2a7dbd5f3e18719d@mail.gmail.com> Trac Admins, I would like a Wiki account so that I may add a page about the different key files and how to store | protect | deal with them. Thanx! Richard From boxbackup at fluffy.co.uk Fri May 2 21:00:53 2008 From: boxbackup at fluffy.co.uk (Richard Hurt) Date: Fri, 2 May 2008 16:00:53 -0400 Subject: [Box Backup] Certificates & Keys Question In-Reply-To: <1209741647.9239.24.camel@localhost.localdomain> References: <712ba87c0805020736w5d663c2ej5411df8674b45478@mail.gmail.com> <1209741647.9239.24.camel@localhost.localdomain> Message-ID: <712ba87c0805021300i1f94f6b9h1891bd2d87aee769@mail.gmail.com> T0ssIEkndmUgYWRkZWQgIG5ldyBwYWdlIHRvIHRoZSB3aWtpCihodHRwczovL3d3dy5ib3hiYWNr dXAub3JnL3RyYWMvd2lraS9NYW5hZ2luZ0tleXNBbmRDZXJ0aWZpY2F0ZXMpIGFuZAphZGRlZCBh IGxpbmsgdG8gaXQgYXQgdGhlIGJvdHRvbSBvZiB0aGUgRkFRLiAgSSBkaWRuJ3QgcmVhbGx5IGtu b3cKd2hhdCB0byBjYWxsIG15IG5ldyBwYWdlIHNvIEkganVzdCBtYWRlIHNvbWV0aGluZyB1cC4g IEFsc28gSSBjb3VsZG4ndApnZXQgdGhlIEZBUSBwYWdlIHRvIGZvbGxvdyB0aGUgQ2FtZWxDYXNl IHdpa2kgbGlua2luZy4gIEkgdGhpbmsgdGhlcmUKaXMgc29tZSBmdW5reSBzdHVmZiBnb2luZyBv biBpbiB0aGF0IHBhZ2UgYnV0IEkgZG9uJ3QgaGF2ZSBlbm91Z2ggVHJhYwpleHBlcmllbmNlIHRv IG1ha2UgaXQgd29yayBjb3JyZWN0bHkuCgpMYXRlci4uLgogIFJpY2hhcmQKCk9uIEZyaSwgTWF5 IDIsIDIwMDggYXQgMTE6MjAgQU0sIEJqYXJuZSBDYXJsc2VuIDxiamFybmVAbWFpbDJuZXQuZGs+ IHdyb3RlOgo+IGZyZSwgMDIgMDUgMjAwOCBrbC4gMTA6MzYgLTA0MDAsIHNrcmV2IFJpY2hhcmQg SHVydDoKPgo+ID4gSSBhbSB0cnlpbmcgdG8gbWFrZSBzdXJlIG15IHN5c3RlbXMgYXJlIHNlY3Vy ZSBhbmQgSSB3b3VsZCBsaWtlIHRvCj4gPiBjaGVjayBteSBhc3N1bXB0aW9ucyBhbmQgcGVyaGFw cyB1cGRhdGUgdGhlIFdpa2kgd2l0aCB0aGUgZmluYWwKPiA+IGluZm9ybWF0aW9uLiAgUGxlYXNl IGxldCBtZSBrbm93IHdoZXJlIG15IGFzc3VtcHRpb25zIGdvIHdyb25nIGFuZCBpZgo+ID4gSSdt IG1pc3Npbmcgc29tZXRoaW5nLgo+ID4KPiA+IFRoYW54IQo+ID4gICBSaWNoYXJkCj4gPgo+ID4g PT09PT09PT09PT09PT09PT09PT09PSAgU05JUCAgPT09PT09PT09PT09PT09PT09PT09PT09PQo+ ID4KPiA+IEJveCBCYWNrdXAgdXNlcyBTU0wgdG8gY29tbXVuaWNhdGUgYmV0d2VlbiB0aGUgc2Vy dmVyIGFuZCB0aGUgY2xpZW50Lgo+ID4gSXQgYWxzbyB1c2VzIEFFUyB0byBlbmNyeXB0IHRoZSBk YXRhIG9uIHRoZSBzZXJ2ZXIuICBUaGVzZQo+ID4gdGVjaG5vbG9naWVzIHJlbHkgb24gdmFyaW91 cyBmaWxlcyAoYWthIEtleXMpIHNvbWUgb2Ygd2hpY2ggbmVlZCB0byBiZQo+ID4gcHJvdGVjdGVk IG1vcmUgdGhhbiBvdGhlcnMuCj4gPgo+ID4gPGFjY291bnQgbnVtYmVyPi1jc3IucGVtCj4gPiAg IC0gU2VjdXJpdHkgTGV2ZWw6IExPVwo+ID4gICAgIFRoaXMgaXMgbWVyZWx5IGEgY2VydGlmaWNh dGUgcmVxdWVzdCBmaWxlIGFuZCBpcyBvbmx5IHVzZWQgb25jZS4KPiA+IEFmdGVyIGdldHRpbmcg eW91ciBjZXJ0aWZpY2F0ZSB0aGlzIGZpbGUgY2FufHNob3VsZCBiZSBkZWxldGVkLgo+ID4KPiA+ IDxhY2NvdW50IG51bWJlcj4tY2VydC5wZW0KPiA+ICAgLSBTZWN1cml0eSBMZXZlbDogTUVESVVN Cj4gPiAgICAgVGhpcyBpcyB0aGUgY2xpZW50cyBTU0wgY2VydGlmaWNhdGUgYW5kIGl0IHRoZSBj bGllbnQgdG8gc2VjdXJlbHkKPiA+IGNvbW11bmljYXRlIHdpdGggdGhlIHNlcnZlci4gIFRoaXMg ZmlsZSBpcyB1bmlxdWUgdG8gZWFjaCBjbGllbnQgYW5kCj4gPiBzaG91bGQgYmUgbWFyZ2luYWxs eSBwcm90ZWN0ZWQuICBJZiBhIGJhZCBndXkgaGFkIHRoaXMgZmlsZSBoZSBjb3VsZAo+ID4gY29w eSB5b3VyIGVuY3J5cHRlZCBkYXRhIGZyb20gdGhlIHNlcnZlciBidXQgd291bGRuJ3QgYmUgYWJs ZSB0byB1c2UKPiA+IGl0Lgo+Cj4gVGhlIGJhZCBndXkgd291bGQgbmVlZCDvu788YWNjb3VudCBu dW1iZXI+LWtleS5wZW0gYXMgd2VsbCBpbiBvcmRlciB0byBkbwo+IHRoaXMuIFdpdGhvdXQga2V5 LnBlbSwgaGUgd291bGQgdGhlb3JldGljYWxseSBoYXZlIGEgc21hbGwgY2hhbmNlIHRvIGJlCj4g YWJsZSB0byBlbmNyeXB0IGRhdGEgb24gdGhlIFNTTCBzZXNzaW9uLCBidXQgaGUgd291bGQgbm90 IGJlIGFibGUgdG8KPiBkZWNyeXB0IHRoZW0uIEFuZCB3aXRob3V0IGJvdGggZmlsZXMsIHRoZSBT U0wgc2Vzc2lvbiBjYW5ub3QgYmUgc2V0IHVwLgo+IFNlY3VyaXR5IGxldmVsIGZvciB0aGlzIGZp bGUgc2hvdWxkIGJlIExPVywgc2luY2UgaXQgaXMgcHVibGljIGFuZCBjYW4KPiBiZSByZWdlbmVy YXRlZCBhdCB0aGUgc2VydmVyLgo+Cj4KPiA+Cj4gPiA8YWNjb3VudCBudW1iZXI+LWtleS5wZW0K PiA+ICAgLSBTZWN1cml0eSBMZXZlbDogPz8/Cj4gPiAgICAgSSdtIG5vdCBxdWl0ZSBzdXJlIHdo YXQgdGhpcyBmaWxlIGlzIHVzZWQgZm9yLiAgSXMgaXQgYW5vdGhlciBTU0wKPiA+IGtleSBvciBz b21ldGhpbmcgdG8gZG8gd2l0aCB0aGUgQUVTIGVuY3J5cHRpb24uICBXaGF0IGhhcHBlbnMgaWYg YSBiYWQKPiA+IGd1eSBnZXRzIGEgaG9sZCBvZiB0aGlzIGZpbGU/ICBXaGF0IGRhbWFnZSBjb3Vs ZCBoZSBkbz8KPgo+IFRoaXMgaXMgdGhlIGNsaWVudCdzIHByaXZhdGUga2V5LCBhcyBvcHBvc2Vk IHRvIO+7vzxhY2NvdW50Cj4gbnVtYmVyPi1jZXJ0LnBlbSwgd2hpY2ggaXMgdGhlIHB1YmxpYyBw YXJ0IHRoYXQgaXMgcHJlc2VudGVkIHRvIHRoZQo+IG90aGVyIHNpZGUgd2hlbiBjb21tdW5pY2F0 aW5nLiBUaGUgcHJpdmF0ZSBrZXkgaXMgbmVjZXNzYXJ5IHRvIGRlY3J5cHQKPiBpbmNvbWluZyBj b21tdW5pY2F0aW9ucy4gQSBiYWQgZ3V5IHdvdWxkIGJlIGFibGUgdG8gdXNlIGl0IGluCj4gY29u anVuY3Rpb24gd2l0aCB0aGUgcHVibGljIO+7vzxhY2NvdW50IG51bWJlcj4tY2VydC5wZW0gdG8g ZGVjcnlwdCB0aGUKPiBTU0wgc2Vzc2lvbiwgYnV0IG5vdCB0aGUgYWN0dWFsIGRhdGEsIHdoaWNo IGFyZSBlbmNyeXB0ZWQKPiB1c2luZyDvu788YWNjb3VudCBudW1iZXI+LUZpbGVFbmNLZXlzLnJh dy4KPiBTZWN1cml0eSBsZXZlbCBzaG91bGQgYmUgTUVESVVNIHRvIEhJR0guIFRoaXMgZmlsZSBp cyBwcml2YXRlIGFuZCBpcwo+IHVzZWQgdG8gc2VjdXJlIHlvdXIgY29tbXVuaWNhdGlvbnMsIGFu ZCBlc3RhYmxpc2ggaWRlbnRpdHkuCj4KPiA+Cj4gPiA8YWNjb3VudCBudW1iZXI+LUZpbGVFbmNL ZXlzLnJhdwo+ID4gICAtIFNlY3VyaXR5IExldmVsOiBISUdIRVNUCj4gPiAgICAgVGhpcyBpcyB0 aGUgbWFzdGVyIEFFUyBrZXkgZm9yIHlvdXIgZGF0YSBhbmQgaXMgdXNlZCB0byBlbmNyeXB0Cj4g PiB5b3VyIGRhdGEgYmVmb3JlIHNlbmRpbmcgaXQgdG8gdGhlIHNlcnZlci4gIFRoaXMgZmlsZSBp cyB1bmlxdWUgdG8KPiA+IGVhY2ggY2xpZW50IGFuZCBzaG91bGQgYmUgcHJvdGVjdGVkIGF0IGFs bCBjb3N0cyBhbmQgc3RvcmVkIG9mZi1zaXRlCj4gPiBpbiBhIHNlY3VyZSBsb2NhdGlvbi4gIFdp dGhvdXQgdGhpcyBmaWxlIHlvdXIgZGF0YSBpcyB1c2VsZXNzLiAgSWYgYQo+ID4gYmFkIGd1eSBn ZXRzIHRoaXMgZmlsZSBhbGwgYmV0cyBhcmUgb2ZmIGFuZCB5b3UgYXJlIHN1bmsuICBJZiB5b3Ug bG9zZQo+ID4gdGhpcyBmaWxlIHlvdSBhcmUgc3Vuay4gIEV2ZXJ5dGhpbmcgY2FuIGJlIHJlcGxh Y2VkLCBleGNlcHQgdGhpcyBmaWxlLgo+ID4KPiA+IHNlcnZlckNBLnBlbQo+ID4gICAtIFNlY3Vy aXR5IExldmVsOiBNRURJVU0KPiA+ICAgVGhpcyBpcyB0aGUgc2VydmVycyBTU0wgY2VydGlmaWNh dGUgYW5kIGl0IGFsbG93cyB0aGUgY2xpZW50IHRvCj4gPiBzZWN1cml0eSBjb21tdW5pY2F0ZSB3 aXRoIHRoZSBzZXJ2ZXIuICBUaGlzIGZpbGUgaXMgY29tbW9uIHRvIGFsbAo+ID4gY2xpZW50cyBh bmQgc2hvdWxkIGJlIG1hcmdpbmFsbHkgcHJvdGVjdGVkLiAgSWYgYSBiYWQgZ3V5IGhhZCB0aGlz Cj4gPiBmaWxlIGhlIGNvdWxkIHJ1biBhIE1hbi1Jbi1UaGUtTWlkZGxlIGF0dGFjayBhbmQgaW1w ZXJzb25hdGUgeW91ciBCb3gKPiA+IEJhY2t1cCBzZXJ2ZXIgdGhlcmVieSBjYXB0dXJpbmcgeW91 ciBlbmNyeXB0ZWQgZGF0YS4KPgo+IFRoZSBzZXJ2ZXIncyBwcml2YXRlIGtleSB3b3VsZCBhbHNv IGJlIG5lZWRlZCAtIHlvdSBjYW5ub3QgcnVuIGEgTUlUTQo+IGJhc2VkIG9uIHRoZSBjZXJ0aWZp Y2F0ZSBhbG9uZS4gQXMgd2l0aCB0aGUg77u/PGFjY291bnQgbnVtYmVyPi1jZXJ0LnBlbSwKPiB0 aGlzIGZpbGUgaXMgcHVibGljLiBJdCBpcyB1c2VkLCBhbW9uZyBvdGhlciB0aGluZ3MsIHRvIGVz dGFibGlzaCB0aGUKPiBzZXJ2ZXIncyBpZGVudGl0eS4gU2VjdXJpdHkgY2FuIGJlIExPVywgc2lu Y2UgdGhpcyBpcyBhIHB1YmxpYyBmaWxlLAo+IHdoaWNoIHRoZSBzZXJ2ZXIgY2FuIHJlLWlzc3Vl IGF0IHdpbGwuCj4KPiBEaXNjbGFpbWVyOiBUaGlzIGlzIGJhc2VkIGVudGlyZWx5IG9uIG15IG93 biByZWFkaW5nIG9mIHRoZSBTU0wvVExTCj4gc3BlY2lmaWNhdGlvbnMuIE90aGVycyBtYXkgd2Vs bCBiZSBtb3JlIGluZm9ybWVkIDspCj4KPiBCamFybmUKPgo+IF9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fCj4gYm94YmFja3VwIG1haWxpbmcgbGlzdAo+IGJv eGJhY2t1cEBmbHVmZnkuY28udWsKPiBodHRwOi8vbGlzdHMud2FyaGVhZC5vcmcudWsvbWFpbG1h bi9saXN0aW5mby9ib3hiYWNrdXAKPgo= From boxbackup at fluffy.co.uk Fri May 2 21:45:56 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Fri, 2 May 2008 21:45:56 +0100 Subject: [Box Backup] Certificates & Keys Question In-Reply-To: <712ba87c0805021300i1f94f6b9h1891bd2d87aee769@mail.gmail.com> References: <712ba87c0805020736w5d663c2ej5411df8674b45478@mail.gmail.com> <1209741647.9239.24.camel@localhost.localdomain> <712ba87c0805021300i1f94f6b9h1891bd2d87aee769@mail.gmail.com> Message-ID: <20080502204556.GA51847@netinertia.co.uk> On Fri, May 02, 2008 at 04:00:53PM -0400, Richard Hurt wrote: > OK, I've added new page to the wiki > (https://www.boxbackup.org/trac/wiki/ManagingKeysAndCertificates) and > added a link to it at the bottom of the FAQ. I didn't really know > what to call my new page so I just made something up. Also I couldn't > get the FAQ page to follow the CamelCase wiki linking. I think there > is some funky stuff going on in that page but I don't have enough Trac > experience to make it work correctly. The FAQ page is using reStructuredText ( https://www.boxbackup.org/trac/wiki/WikiRestructuredText ). To add a Trac link, use the following: .. trac:: TracPage James From boxbackup at fluffy.co.uk Sat May 3 14:43:58 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Sat, 03 May 2008 15:43:58 +0200 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: <20080502101720.GA9281@netinertia.co.uk> References: <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> <481AB0BD.80002@netinertia.co.uk> <481ABB5C.1030208@hickinbottom.com> <20080502101720.GA9281@netinertia.co.uk> Message-ID: <481C6C1E.4030008@trexler.at> James O'Gorman schrieb: > On Fri, May 02, 2008 at 07:57:32AM +0100, Stuart Hickinbottom wrote: >> Yes - there are usually directories such as the following: >> >> /etc/cron.daily >> /etc/cron.weekly >> /etc/cron.monthly >> >> You can just drop executable scripts or binaries in there and they'll be run >> with /approximately/ that frequency (no guarantees). Any output from those >> scripts/binaries is emailed to a responsible adult on the system (usually >> root). >> >> This is the case on at least Red Hat Enterprise and Gentoo - I can't readily >> check others. > > Marvellous. I'll try and install Fedora and maybe another distro in a VM > over the weekend and adapt my FreeBSD script to the Linux way. I'll > commit it to contrib/ when it's done. Good idea, but I would suggest "compare -aq" as default, as "compare -a" will do a full download and therefore generate a lot of traffic. This is probably not wanted weekly/monthly in most cases. Wolfgang From boxbackup at fluffy.co.uk Sat May 3 16:49:57 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Sat, 03 May 2008 16:49:57 +0100 Subject: [Box Backup] Restoring a corrupted archive? In-Reply-To: <481C6C1E.4030008@trexler.at> References: <48181DDA.2050004@homann.se> <481842AD.4080101@homann.se> <48188C20.6050302@homann.se> <4818E265.2020107@homann.se> <48196110.6030902@homann.se> <481AB0BD.80002@netinertia.co.uk> <481ABB5C.1030208@hickinbottom.com> <20080502101720.GA9281@netinertia.co.uk> <481C6C1E.4030008@trexler.at> Message-ID: <481C89A5.3030604@netinertia.co.uk> Wolfgang Trexler wrote: > Good idea, but I would suggest "compare -aq" as default, as "compare -a" > will do a full download and therefore generate a lot of traffic. This is > probably not wanted weekly/monthly in most cases. I did wonder about that. It's figured it'd be good to have the full compare run, but you're right, the default should probably be checksum only. James From boxbackup at fluffy.co.uk Sun May 4 06:53:32 2008 From: boxbackup at fluffy.co.uk (Jason) Date: Sun, 4 May 2008 01:53:32 -0400 Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hello again. So, I ran "bbstoreaccounts check " just to see what happened, and it took a while, though less than a day. The output seemed reasonable, so I ran "bbstoreaccounts check fix", and it's still running. It's been going for probably 10 days now. That account stores about 20 gig of data, and it is being stored on an encrypted drive so there's encryption overhead, but 10 days still seems like quite a bit. It keeps spitting out hex values as it goes, although it's about 1 every 13 seconds or so now when I seem to think it was 1 every 2-3 seconds when I started it. I didn't record the data from the previous run (without "fix"), or pay a lot of attention to how quickly it was going in the beginning, but do these numbers sound right? Is there any way to guess how much longer it's going to take? I don't mind the waiting so much as the not knowing the progress. Thanks for you time, Jason On Mon, Apr 21, 2008 at 5:27 PM, Chris Wilson wrote: > Hi Jason, > > > > I've been using boxbackup for a while now, but unfortunately I haven't > > been paying as much attention to my logs as I should have. It appears > > I've been getting this message in the logs for quite some time now, so > > I'm not sure what happened in order to cause it to start. > > > > Apr 21 10:13:33 [bbstored/hk] while housekeeping account 00000003, > > exception BackupStore InvalidBackupStoreFilename (4/3) -- aborting > > housekeeping run for this account > > > > I looked for more information on this exception in trac, the > > exceptions list on the website, and in the mailing list archives but > > didn't find any information on what map be happening. I even took a > > brief look at the code which throws this exception, but it wasn't > > immediately obvious to me what was going on. > > > > So, can anyone tell me what could be causing this > > Probably corruption of the filesystem that the store is on. > > > > and how I might fix > > it? > > bbstoreaccounts check fix > > > > This is on the bbstored server, and I've got a few other clients that > > connect to this server and they have identical boxbackup configuration > > files, I believe. So perhaps it has something to do with some of the > > files that are actually being backed up? > > I hope not, but if it reappears let us know. > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Sun May 4 13:37:31 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 4 May 2008 13:37:31 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi Jason, On Sun, 4 May 2008, Jason wrote: > So, I ran "bbstoreaccounts check " just to see what happened, > and it took a while, though less than a day. The output seemed > reasonable, so I ran "bbstoreaccounts check fix", and it's > still running. It's been going for probably 10 days now. That account > stores about 20 gig of data, and it is being stored on an encrypted > drive so there's encryption overhead, but 10 days still seems like quite > a bit. It keeps spitting out hex values as it goes, although it's about > 1 every 13 seconds or so now when I seem to think it was 1 every 2-3 > seconds when I started it. > > I didn't record the data from the previous run (without "fix"), or pay > a lot of attention to how quickly it was going in the beginning, but > do these numbers sound right? Is there any way to guess how much > longer it's going to take? I don't mind the waiting so much as the > not knowing the progress. That sounds very unreasonable to me. I'd expect it to be doing at least 1 MB/second, or no more than 6 hours for the whole 20GB account. Could you give an example of the hex output? If there are any lines that don't just have hex numbers, those would be more helpful. You can safely interrupt and restart the check fix process. Alternatively, can I get a copy of the store directory to test and profile it locally? It shouldn't expose any sensitive information as it's all encrypted. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Mon May 5 16:57:33 2008 From: boxbackup at fluffy.co.uk (Jason) Date: Mon, 5 May 2008 11:57:33 -0400 Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: When I run the command without "fix" I get this: Check store account ID 00000003 Phase 1, check objects... Phase 2, check directories... File ID 1a66aa has different container ID, probably moved File ID 1a66ae has different container ID, probably moved File ID 1a66ac has different container ID, probably moved ,,, File ID 18898e has different container ID, probably moved File ID 186165 has different container ID, probably moved File ID 15b759 has different container ID, probably moved Directory ID 14987e has wrong container ID. Phase 3, check root... Phase 4, fix unattached objects... Object 18df9e is unattached. Object 18df9f is unattached. Object 18dfa0 is unattached. ,,, Object 1ab162 is unattached. Object 1ab163 is unattached. Object 1ab164 is unattached. Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... NOTE: Soft limit for account changed to ensure housekeeping doesn't delete files on next run NOTE: Hard limit for account changed to ensure housekeeping doesn't delete files on next run 104352 errors found NOTE: No changes to the store account have been made. Run again with fix option to fix these errors You should now use bbackupquery on the client machine to examine the store. When running with "fix" I get: NOTE: Will fix errors encountered during checking. Check store account ID 00000003 Phase 1, check objects... Phase 2, check directories... File ID 1a66aa has different container ID, probably moved File ID 1a66ae has different container ID, probably moved File ID 1a66ac has different container ID, probably moved ... File ID 18898e has different container ID, probably moved File ID 186165 has different container ID, probably moved File ID 15b759 has different container ID, probably moved Directory ID 14987e has wrong container ID. Phase 3, check root... Phase 4, fix unattached objects... Object 18df9e is unattached. Object 18df9f is unattached. Object 18dfa0 is unattached. ... Object 18f4e6 is unattached. Object 18f4e7 is unattached. Object 18f4e8 is unattached. And it's still going. I started it yesterday after getting your message. On Sun, May 4, 2008 at 8:37 AM, Chris Wilson wrote: > Hi Jason, > > > On Sun, 4 May 2008, Jason wrote: > > > So, I ran "bbstoreaccounts check " just to see what happened, > > and it took a while, though less than a day. The output seemed > > reasonable, so I ran "bbstoreaccounts check fix", and it's > > still running. It's been going for probably 10 days now. That account > > stores about 20 gig of data, and it is being stored on an encrypted > > drive so there's encryption overhead, but 10 days still seems like quite > > a bit. It keeps spitting out hex values as it goes, although it's about > > 1 every 13 seconds or so now when I seem to think it was 1 every 2-3 > > seconds when I started it. > > > > I didn't record the data from the previous run (without "fix"), or pay > > a lot of attention to how quickly it was going in the beginning, but > > do these numbers sound right? Is there any way to guess how much > > longer it's going to take? I don't mind the waiting so much as the > > not knowing the progress. > > That sounds very unreasonable to me. I'd expect it to be doing at least 1 > MB/second, or no more than 6 hours for the whole 20GB account. > > Could you give an example of the hex output? If there are any lines that > don't just have hex numbers, those would be more helpful. You can safely > interrupt and restart the check fix process. > > Alternatively, can I get a copy of the store directory to test and profile > it locally? It shouldn't expose any sensitive information as it's all > encrypted. > > > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Mon May 5 17:08:16 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 5 May 2008 17:08:16 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi Jason, On Mon, 5 May 2008, Jason wrote: > When I run the command without "fix" I get this: > > Phase 4, fix unattached objects... > Object 18df9e is unattached. > Object 18df9f is unattached. > Object 18dfa0 is unattached. > ,,, > Object 1ab162 is unattached. > Object 1ab163 is unattached. > Object 1ab164 is unattached. [...] > When running with "fix" I get: > > Phase 4, fix unattached objects... > Object 18df9e is unattached. > Object 18df9f is unattached. > Object 18dfa0 is unattached. > ... > Object 18f4e6 is unattached. > Object 18f4e7 is unattached. > Object 18f4e8 is unattached. > > And it's still going. I started it yesterday after getting your message. So if I understand you correctly, it's in Phase 4 and printing out the same numbers that it printed without "fix", but much more slowly (because it's repairing the errors)? I think that gives me enough information to start digging, thanks. Out of curiosity, roughly how many "object is unattached" messages did you get in the check run (without fix), and how long is it taking to produce each one when fixing? (this should give us an estimate of how long the fix will take at the current rate). I'll get back to you when I know more about the problem, hopefully later today. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Mon May 5 17:26:44 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 5 May 2008 17:26:44 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi James, On Mon, 5 May 2008, Chris Wilson wrote: > Out of curiosity, roughly how many "object is unattached" messages did > you get in the check run (without fix), and how long is it taking to > produce each one when fixing? (this should give us an estimate of how > long the fix will take at the current rate). Sorry, I missed that part in your previous message where you answered one of these. If every single object in that range is missing a parent then it's about 120,000 objects, of whicit's finished fixing about 5,000 so far, in just over a day. That tells me that it's taking about 16 seconds each, and it's about 5% finished, so it will take 20-30 days to finish :-( I think the main problem is this code in BackupStoreCheck2.cpp: // The easiest way to do this is to verify it again. Not such a // bad penalty, because this really shouldn't be done very often. std::auto_ptr file(RaidFileRead::Open(mDiscSetNumber, filename)); BackupStoreFile::VerifyEncodedFileFormat(*file, &diffFromObjectID); Unfortunately this is done for every unattached file, in your case 120,000 of them. That's not compatible with my definition of "not very often". I think it means reading the entire contents of the file in each case. I don't know how many files you have in total, or what the encrypted filesystem is that you're running on, but I guess it's conceiveable that it could take days to read 20 GB of data, especially if seeks are involved. What sort of performance do you get, e.g. how long does it take to read a 1GB file from this filesystem? Is it CPU or I/O bound? If it's CPU, then you might find it much faster to copy the data off the encrypted partition before running check fix on it. Also, since Box Backup's backups are already encrypted by the client, why put it on an encrypted filesystem? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Mon May 5 19:17:49 2008 From: boxbackup at fluffy.co.uk (Greg Bolshaw) Date: Mon, 5 May 2008 19:17:49 +0100 Subject: [Box Backup] Backing up to mounted network storage Message-ID: Dear list I am considering mounting network storage (NFS or Samba) to be used as a Box Backup data store. The volume would be connected by 100Mb ethernet. Are there any extra precautions to be taken with such a setup? Also, would it be recommended to run the same setup but instead over a 10Mb VPN? Any advice much appreciated. Thanks Greg From boxbackup at fluffy.co.uk Mon May 5 21:11:26 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 5 May 2008 21:11:26 +0100 (BST) Subject: [Box Backup] Backing up to mounted network storage In-Reply-To: References: Message-ID: Hi Greg, On Mon, 5 May 2008, Greg Bolshaw wrote: > I am considering mounting network storage (NFS or Samba) to be used as a > Box Backup data store. The volume would be connected by 100Mb ethernet. > Are there any extra precautions to be taken with such a setup? Also, > would it be recommended to run the same setup but instead over a 10Mb > VPN? This setup is not recommended because: * You lose the bandwidth efficiency of Box Backup (which exists only between bbackupd and bbstored). Running over the VPN is particularly likely to have poor performance. * Many NAS devices, particularly cheap ones, are very slow * The necessary POSIX semantics for safe operation of bbstored, e.g. allowing renaming over existing files (atomic replace), may not work correctly, particularly on an SMB server. However, if you're willing to take the risks and compare regularly, it might work OK. bbstored was designed to be lightweight and to run on embedded systems, so if your NAS device is an open system and runs a POSIX operating system you may find that you can successfully run bbstored on the NAS itself. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Mon May 5 23:38:59 2008 From: boxbackup at fluffy.co.uk (Jason) Date: Mon, 5 May 2008 18:38:59 -0400 Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: # time cat 1_gig_file > /dev/null real 0m58.625s user 0m0.100s sys 0m6.720s I definitely think it's CPU bound because of the drive encryption. One of the CPUs is pegged by kernel processes when I run that cat. Unfortunately I don't have enough unencrypted free space on that box to copy the data there. How would I go about copying the data to another system and fixing it there? Do I just need to copy this one store and the boxbackup config files to the other machine? I store it on an encrypted filesystem because I already have a cluster of boxes dedicated to data storage and the whole thing is encrypted, so it was far easier for me to just put the boxbackup stores with all my other data instead of sectioning of an unexcrypted bit just for boxbackup. Thank you very much for your continued help with this. It really bolsters my confidence in boxbackup knowing that someone actually cares. Jason On Mon, May 5, 2008 at 12:26 PM, Chris Wilson wrote: > Hi James, > > > On Mon, 5 May 2008, Chris Wilson wrote: > > > Out of curiosity, roughly how many "object is unattached" messages did > > you get in the check run (without fix), and how long is it taking to > > produce each one when fixing? (this should give us an estimate of how > > long the fix will take at the current rate). > > Sorry, I missed that part in your previous message where you answered one > of these. If every single object in that range is missing a parent then > it's about 120,000 objects, of whicit's finished fixing about 5,000 so > far, in just over a day. That tells me that it's taking about 16 seconds > each, and it's about 5% finished, so it will take 20-30 days to finish :-( > > I think the main problem is this code in BackupStoreCheck2.cpp: > > // The easiest way to do this is to verify it again. Not such a > // bad penalty, because this really shouldn't be done very often. > > std::auto_ptr file(RaidFileRead::Open(mDiscSetNumber, > filename)); > BackupStoreFile::VerifyEncodedFileFormat(*file, &diffFromObjectID); > > Unfortunately this is done for every unattached file, in your case > 120,000 of them. That's not compatible with my definition of "not very > often". > > I think it means reading the entire contents of the file in each case. I > don't know how many files you have in total, or what the encrypted > filesystem is that you're running on, but I guess it's conceiveable that > it could take days to read 20 GB of data, especially if seeks are > involved. > > What sort of performance do you get, e.g. how long does it take to read a > 1GB file from this filesystem? Is it CPU or I/O bound? If it's CPU, then > you might find it much faster to copy the data off the encrypted partition > before running check fix on it. > > Also, since Box Backup's backups are already encrypted by the client, why > put it on an encrypted filesystem? > > > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Mon May 5 23:43:25 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 5 May 2008 23:43:25 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi Jason, On Mon, 5 May 2008, Jason wrote: > # time cat 1_gig_file > /dev/null > real 0m58.625s > user 0m0.100s > sys 0m6.720s > > I definitely think it's CPU bound because of the drive encryption. > One of the CPUs is pegged by kernel processes when I run that cat. OK, but you're getting about 18 MB/second which is much faster than bbstoreaccounts is running, so the problem isn't here. > Unfortunately I don't have enough unencrypted free space on that box > to copy the data there. How would I go about copying the data to > another system and fixing it there? Do I just need to copy this one > store and the boxbackup config files to the other machine? Yes, you could just copy the store files, bbstored.conf and raidfile.conf, but it may not make enough difference to be worthwhile. > I store it on an encrypted filesystem because I already have a cluster > of boxes dedicated to data storage and the whole thing is encrypted, so > it was far easier for me to just put the boxbackup stores with all my > other data instead of sectioning of an unexcrypted bit just for > boxbackup. OK. > Thank you very much for your continued help with this. It really > bolsters my confidence in boxbackup knowing that someone actually cares. No problem, I like to care and I'd like to fix this for you :-) I'm working on reproducing the problem and profiling bbstoreaccounts check fix at the moment. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Tue May 6 02:33:11 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 6 May 2008 02:33:11 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi Jason, OK, I found the problem. BackupStoreDirectory::CheckAndFix has an algorithm that's potentially exponential, and I can see it getting slower and slower as the bbstoreaccounts check fix process runs. The good news is that I understand the problem, the bad news is that it's not easy to fix, and your check fix will probably take more than 30 days to complete :-( Also, I'm going to be out of country for the next 3 weeks from Wednesday, but I'll have some long journeys to think about the problem and figure out how to fix it. Ben, if you have time to investigate BackupStoreDirectory::CheckAndFix it would be greatly appreciated, as you know how it's supposed to work and I don't (yet) :-) My gprof call graph for bbstoreaccounts check fix, when I manually removed a directory from the store that contained about 1000 files, looks like this: ----------------------------------------------- 0.05 1.06 1072/3679 BackupStoreCheck::InsertObjectIntoDirectory(long long, long long, bool) [7] 0.13 2.59 2607/3679 BackupStoreCheck::CheckDirectories() [5] [4] 49.7 0.18 3.65 3679 BackupStoreDirectory::CheckAndFix() [4] 0.04 0.94 608393/608394 std::set, std::allocator >::insert(long long const&) [13] 0.04 0.81 608393/608393 std::set, std::allocator >::insert(BackupStoreFilename const&) [16] 0.00 0.63 608393/609465 std::set, std::allocator >::find(long long const&) [19] 0.00 0.49 608393/608393 std::set, std::allocator >::find(BackupStoreFilename const&) [23] 0.00 0.24 3220/3220 std::set, std::allocator >::~set() [32] 0.00 0.13 3220/3221 std::set, std::allocator >::~set() [49] 0.06 0.00 1219393/1319188 BackupStoreDirectory::Entry::GetFlags() const [86] 0.04 0.00 608393/608393 BackupStoreDirectory::Entry::GetDependsNewer() const [107] 0.04 0.00 1216786/2461259 __gnu_cxx::__normal_iterator > >::operator++() [72] 0.04 0.00 1216786/1824110 BackupStoreDirectory::Entry::GetObjectID() const [91] 0.04 0.00 608393/608393 std::set, std::allocator >::end() const [112] 0.01 0.02 608393/609465 std::set, std::allocator >::end() const [122] 0.02 0.00 1227823/2515664 std::vector >::end() [90] 0.02 0.00 608393/608393 BackupStoreDirectory::Entry::GetDependsOlder() const [145] 0.02 0.00 1832537/3062008 bool __gnu_cxx::operator!= > > (__gnu_cxx::__normal_iterator > > const&, __gnu_cxx::__normal_iterator > > const&) [119] 0.00 0.01 3220/3221 std::set, std::allocator >::set() [164] 0.00 0.01 3220/3220 std::set, std::allocator >::set() [176] 0.01 0.00 1216786/1216786 BackupStoreDirectory::Entry::GetName() const [189] 0.01 0.00 5478144/6693968 __gnu_cxx::__normal_iterator > >::operator*() const [177] 0.00 0.00 619430/661726 std::vector >::begin() [256] 0.00 0.00 608393/608393 __gnu_cxx::__normal_iterator > > ::operator--() [408] 0.00 0.00 608393/609465 std::_Rb_tree_const_iterator::operator!= (std::_Rb_tree_const_iterator const&) const [403] 0.00 0.00 608393/608393 std::_Rb_tree_const_iterator:: operator!=(std::_Rb_tree_const_iterator const&) const [410] 0.00 0.00 3679/4751 bool __gnu_cxx::operator== > > (__gnu_cxx::__normal_iterator > > const&, __gnu_cxx::__normal_iterator > > const&) [535] ----------------------------------------------- If you're not used to reading gprof output then this link may help: http://sourceware.org/binutils/docs-2.18/gprof/Call-Graph.html#Call-Graph What this appears to mean to me is that, in 3000 calls to CheckAndFix, we did 600,000 insert() and find() operations on the two sets, idsEncountered and filenamesEncountered, and that took nearly 3 seconds out of the total 3.65 seconds used by CheckAndFix and its callees. I'm not yet sure whether most of those inserts happened on just one run of CheckAndFix, when a large number of objects had to be removed and therefore the loop had to be restarted multiple times; or if it's just that CheckAndFix does a lot of work and is called many times. One thought that occurred to me was that in either case, we could reduce the constant cost of find() and insert() by using TR1 unordered_set instead of std::set on platforms that support it. Another is that the IDs are usually encountered in reverse order, so we could give begin() as a hint to the insert() operator because the new element will often belong at the beginning of the list for the ID list (but not the filename list). However that only saves about 1/3 of the overhead at best. The third option would be to determine how many times we restart the search in these cases, and if it's significant, to rewrite the algorithm so that the search doesn't have to be restarted, for example by queueing all deletions until the end. I could also be entirely wrong and CheckAndFix might not really be the slow path in this case. I can give you complete gprof output if that helps, or the built binary and gmon.out file so you can run your own analysis. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Tue May 6 23:00:11 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 6 May 2008 23:00:11 +0100 (BST) Subject: [Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3) In-Reply-To: References: Message-ID: Hi Jason, I just committed a fix to bbstoreaccounts in trunk which should make fixing unattached files a lot faster (about 100x in my test with 5,000 files) and no longer O(n^2), at the cost of using more memory while running. If you're feeling brave and you have a backup copy of the account, please help me test it by downloading and building the latest trunk source and running the new release/bin/bbstoreaccounts to check and fix your account. I hope that this solves the issue for you, but please let me know how you get on. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Thu May 8 11:52:36 2008 From: boxbackup at fluffy.co.uk (Greg Bolshaw) Date: Thu, 8 May 2008 11:52:36 +0100 Subject: [Box Backup] Backing up to mounted network storage In-Reply-To: References: Message-ID: <4E2702B1-B80D-498B-B60E-7A1FA2691591@bolshaw.me.uk> On 5 May 2008, at 21:11, Chris Wilson wrote: > This setup is not recommended because: > > * You lose the bandwidth efficiency of Box Backup (which exists only > between bbackupd and bbstored). Running over the VPN is particularly > likely to have poor performance. > > * Many NAS devices, particularly cheap ones, are very slow > > * The necessary POSIX semantics for safe operation of bbstored, e.g. > allowing renaming over existing files (atomic replace), may not work > correctly, particularly on an SMB server. > > However, if you're willing to take the risks and compare regularly, it > might work OK. > > bbstored was designed to be lightweight and to run on embedded > systems, so > if your NAS device is an open system and runs a POSIX operating > system you > may find that you can successfully run bbstored on the NAS itself. Hi Chris Thanks for the detailed reply. I will give it a go in a test environment and see how the performance and reliability suffer. Out of interest, what would happen if the network storage became unavailable during a backup? Would the client know where to resume from? Thanks Greg From boxbackup at fluffy.co.uk Thu May 8 11:56:13 2008 From: boxbackup at fluffy.co.uk (Greg Bolshaw) Date: Thu, 8 May 2008 11:56:13 +0100 Subject: [Box Backup] Deleting from backup Message-ID: Dear list I have accidentally backed up some files that I don't want backing up. As they are quite large, is there any way to delete them from the backup permanently? Also, is it possible to automatically delete files from the backup after a number of days, rather than keeping them until the storage limit is reached? Thanks Greg From boxbackup at fluffy.co.uk Thu May 8 14:04:56 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 08 May 2008 15:04:56 +0200 Subject: [Box Backup] Deleting from backup In-Reply-To: References: Message-ID: <4822FA78.2010106@homann.se> Greg Bolshaw wrote: > Dear list > > I have accidentally backed up some files that I don't want backing up. > As they are quite large, is there any way to delete them from the backup > permanently? Yes, there is. WARNING! NO GUARANTEES! 1) Find with bbackupquery the object id of the file(s). 2) Stop the server. 3) Remove the file(s) from the storage. The algorithm to go from the object id to the storage file is a bit complex. 3a) If the object id is 0x00 to 0xff (example 0x12), the name is /o12.rfx 3b) If the object id is 0x100 to 0xffff (example 0x1234, the name is /12/o34.rfw 3c) If the object id is 0x10000 to 0xffffff (example 0x123456, the name is /34/12/o56.rfw (note reversal!) 3d) It is wise to check the file size on storage with the real file that should be removed. Theu don't match, but should be close. 4) Run bbstoreaccounts check fix. 5) Start the server. That's what I practiced during my attempts to recover a corrupt storage. Good luck! A 'delete ' would be good in bbackupquery. Magnus From boxbackup at fluffy.co.uk Thu May 8 19:16:13 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 8 May 2008 20:16:13 +0200 (CAT) Subject: [Box Backup] Backing up to mounted network storage In-Reply-To: <4E2702B1-B80D-498B-B60E-7A1FA2691591@bolshaw.me.uk> References: <4E2702B1-B80D-498B-B60E-7A1FA2691591@bolshaw.me.uk> Message-ID: Hi Greg, On Thu, 8 May 2008, Greg Bolshaw wrote: > Thanks for the detailed reply. I will give it a go in a test environment > and see how the performance and reliability suffer. Out of interest, > what would happen if the network storage became unavailable during a > backup? Would the client know where to resume from? It depends exactly what happens to bbstored and whether all outstanding requests to the NAS either succeed or fail, but if the NAS (and the mounting client) implement sensible POSIX semantics then it should resume successfully and automatically. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sat May 10 11:47:20 2008 From: boxbackup at fluffy.co.uk (Tobias Roth) Date: Sat, 10 May 2008 12:47:20 +0200 Subject: [Box Backup] Confusion with Regex Message-ID: <48257D38.1050303@fsck.ch> Hi I'm confused about the usage of regular expressions in the Exclude lines. The website says to look at re_format(7) and mentiones this example: ExcludeFilesRegex = *.(mp3|MP3)\$ But according to re_format(7), this is not a valid regular expression, because the * needs to be preceeded by an atom (also, why is the $ sign escaped? If it signifies end-of-line, it shouldn't be). And boxbackup confirms this, I get an error if I use: ExcludeFilesRegex = * So, the following seems to work: ExcludeFilesRegex = .* ExcludeDirsRegex = .* AlwaysIncludeFile = /boot/loader.conf AlwaysIncludeDir = /boot/profile AlwaysIncludeFilesRegex = /boot/profile/.* It backs up /boot/loader.conf and everything in dir /boot/profile/ Now, I would like to do something else: Include all files except those with a filename beginning with a dot. How would the exclude line look for that? ExcludeFilesRegex = ^\..* doesn't seem to work, it still includes all hidden files. Thanks, Tobias From boxbackup at fluffy.co.uk Sat May 10 21:06:53 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sat, 10 May 2008 22:06:53 +0200 (CAT) Subject: [Box Backup] Confusion with Regex In-Reply-To: <48257D38.1050303@fsck.ch> References: <48257D38.1050303@fsck.ch> Message-ID: Hi Tobias, On Sat, 10 May 2008, Tobias Roth wrote: > I'm confused about the usage of regular expressions in the Exclude > lines. The website says to look at re_format(7) and mentiones this > example: > > ExcludeFilesRegex = *.(mp3|MP3)\$ > > But according to re_format(7), this is not a valid regular expression, > because the * needs to be preceeded by an atom (also, why is the $ sign > escaped? If it signifies end-of-line, it shouldn't be). And boxbackup > confirms this, I get an error if I use: > > ExcludeFilesRegex = * I think you're right, the documentation is in a bit of a state regarding regular expressions, particularly because on some platforms you only get POSIX and on some you get PCRE, which is more powerful, and there's no easy way to tell which you have got in your build, and we're waiting until we've fixed that before rewriting the docs. > Now, I would like to do something else: Include all files except those > with a filename beginning with a dot. > > How would the exclude line look for that? ExcludeFilesRegex = ^\..* > doesn't seem to work, it still includes all hidden files. It has to match the full path if you start with ^, therefore this would not match anything at all, since all paths start with / on Unix and a drive letter on Windows. You probably want something like this: ExcludeFilesRegex = /\. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 11 13:30:38 2008 From: boxbackup at fluffy.co.uk (Greg Bolshaw) Date: Sun, 11 May 2008 13:30:38 +0100 Subject: [Box Backup] Deleting from backup In-Reply-To: <4822FA78.2010106@homann.se> References: <4822FA78.2010106@homann.se> Message-ID: <4520BEAC-81E6-48A6-9F90-D9CFAB94B1B7@bolshaw.me.uk> On 8 May 2008, at 14:04, Magnus Homann wrote: > That's what I practiced during my attempts to recover a corrupt > storage. Good luck! Thank you, that worked perfectly. > A 'delete ' would be good in bbackupquery. I agree, it would be great to have such a function. Ideally this would also allow for the removal of directories including subdirectories and files. Would this be difficult to implement? Is it possible to automatically delete files from the backup after a number of days, rather than keeping them until the storage limit is reached? I would like to create a backup scheme where the last 30 days of files are always available, rather than the last n GB. Thanks Greg From boxbackup at fluffy.co.uk Sun May 11 14:41:20 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Sun, 11 May 2008 15:41:20 +0200 (CAT) Subject: [Box Backup] Deleting from backup In-Reply-To: <4520BEAC-81E6-48A6-9F90-D9CFAB94B1B7@bolshaw.me.uk> References: <4822FA78.2010106@homann.se> <4520BEAC-81E6-48A6-9F90-D9CFAB94B1B7@bolshaw.me.uk> Message-ID: Hi Greg, On Sun, 11 May 2008, Greg Bolshaw wrote: >> A 'delete ' would be good in bbackupquery. > > I agree, it would be great to have such a function. Ideally this would > also allow for the removal of directories including subdirectories and > files. Would this be difficult to implement? Not really, the command already exists in the protocol (bbackupd uses it), but it only marks the files as deleted rather than actually deleting them. Only the store can actually, permanently delete files. > Is it possible to automatically delete files from the backup after a > number of days, rather than keeping them until the storage limit is > reached? I would like to create a backup scheme where the last 30 days > of files are always available, rather than the last n GB. Of course it's possible in theory but I'm afraid it's not implemented yet. Currently the store decides when to delete files based on its own algorithm, which just tries to keep the account below the soft limit. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Sun May 11 23:55:20 2008 From: boxbackup at fluffy.co.uk (Bill West) Date: Sun, 11 May 2008 18:55:20 -0400 Subject: [Box Backup] Cleanup after accidently hitting the 2GB problem Message-ID: Cleanup after accidently hitting the 2GB problem. Here's a quick note about how to clean up after hitting the 2GB file size problem. I do not normally have any files > 2GB, but an Apache 'access.log' file was not being rotated, and I ran into the problem. The 'access.log' file was > 2GB. I fixed the file rotation problem and got rid of the file that was > 2GB by running 'logrotate -f' twice, which compressed the large file and left it in 'access.log.2.gz', but still had a file named 'access.log'. Running /usr/local/bin/bbackup-daily on the client still generated the error message, apparently because attempting to diff the stored version of the file (>2GB) against the version on the client (<2GB) still caused a problem. Renaming 'access.log' to 'access_good.log' did not correct the situation. Apparently bbackup is detecting that the file is still the same file (I guess from the inode ???), and still trys to diff against the stored version. Copying the file to a temp directory and deleting it from the directory that bbackup was trying to store got rid of the error message. Copying it back to the original directory and original filename did not reintroduce the error. Looking at the output from bbackquery 'ls -odt', apparently the >2GB file is still in the store, marked as deleted, with the name 'access_good.log', but is not causing problems because nothing in the current backup is diff'd against it. I am running bbackupd version 0.09, and recent version of Debian Linux and Apache. The following operation we done with Apache not running. Relevant portions of syslog and bbackquery output are below. It took me several hours to figure this out. Maybe this will help someone else deal with this more quickly. Bill West BWI 887 Main Street, Suite D Monroe, Connecticut 06468-2800 877 567-7450, 203 261-6027 FAX 203 261-5061 info at bwi.com http://www.bwi.com BWI and RackPaq are registered trademarks of Bill West Incorporated Destwin is a trademark of DESTWIN, LLC. ---------------Extracted from syslog file on client, with comments -------------------------------------- ### Original error, after rotating logs so there is no file > 2GB May 11 17:28:06 shearwater bbackupd[3888]: Incoming connection from local (UNIX socket) May 11 17:28:06 shearwater bbackupd[3888]: Connection from command socket May 11 17:28:06 shearwater bbackupd[3888]: Beginning scan of local files May 11 17:28:06 shearwater bbackupd[3888]: Opening connection to server backup.bogus.net... May 11 17:28:06 shearwater bbackupd[3888]: Connection made, login successful May 11 17:28:07 shearwater bbackupd[3888]: Backup object failed, error when reading /www/logs/access.log May 11 17:28:07 shearwater bbackupd[3888]: Error code when uploading was (4/11), BackupStore BadBackupStoreFile May 11 17:28:07 shearwater bbackupd[3888]: Finished scan of local files May 11 17:28:07 shearwater bbackupd[3888]: File statistics: total file size uploaded 2066, bytes already on server 0, encoded size 1040 ### After 'mv -i access.log access_good.log' May 11 17:48:57 shearwater bbackupd[3888]: Incoming connection from local (UNIX socket) May 11 17:48:57 shearwater bbackupd[3888]: Connection from command socket May 11 17:48:57 shearwater bbackupd[3888]: Beginning scan of local files May 11 17:48:57 shearwater bbackupd[3888]: Opening connection to server backup.bogus.net... May 11 17:48:59 shearwater bbackupd[3888]: Connection made, login successful May 11 17:49:01 shearwater bbackupd[3888]: Backup object failed, error when reading /www/logs/access_good.log May 11 17:49:01 shearwater bbackupd[3888]: Error code when uploading was (4/11), BackupStore BadBackupStoreFile May 11 17:49:10 shearwater bbackupd[3888]: Finished scan of local files May 11 17:49:10 shearwater bbackupd[3888]: File statistics: total file size uploaded 6956, bytes already on server 0, encoded size 2104 ### After 'cp -p access_good.log /tmp/access_good.log; rm access_good.log' May 11 17:50:56 shearwater bbackupd[3888]: Incoming connection from local (UNIX socket) May 11 17:50:56 shearwater bbackupd[3888]: Connection from command socket May 11 17:50:56 shearwater bbackupd[3888]: Beginning scan of local files May 11 17:50:56 shearwater bbackupd[3888]: Opening connection to server backup.bogus.net... May 11 17:50:56 shearwater bbackupd[3888]: Connection made, login successful May 11 17:50:57 shearwater bbackupd[3888]: Finished scan of local files May 11 17:50:57 shearwater bbackupd[3888]: File statistics: total file size uploaded 0, bytes already on server 0, encoded size 0 ### After 'cp -p /tmp/access_good.log access.log' May 11 17:52:32 shearwater bbackupd[3888]: Incoming connection from local (UNIX socket) May 11 17:52:32 shearwater bbackupd[3888]: Connection from command socket May 11 17:52:32 shearwater bbackupd[3888]: Beginning scan of local files May 11 17:52:32 shearwater bbackupd[3888]: Opening connection to server backup.bogus.net... May 11 17:52:32 shearwater bbackupd[3888]: Connection made, login successful May 11 17:52:34 shearwater bbackupd[3888]: Finished scan of local files May 11 17:52:34 shearwater bbackupd[3888]: File statistics: total file size uploaded 588880, bytes already on server 0, encoded size 114423 ---------------------- output from bbackupquery, interactive mode ----------------------- query > ls -odt ------- snip irrelevant stuff --------------- 00018c0e f--o-- 2008-05-03T23:49:35 error.log 00018c0f f-Xo-- 2008-05-04T04:17:09 access_good.log 00018c18 f--o-- 2008-05-04T22:50:50 error.log 00018c19 f-Xo-- 2008-05-05T04:16:38 access_good.log 00018c2f f--o-- 2008-05-06T01:23:18 error.log 00018c30 f-Xo-- 2008-05-06T04:16:40 access_good.log 00018c45 f-Xo-- 2008-05-07T04:16:38 access_good.log 00018c54 f--o-- 2008-05-08T00:32:06 error.log 00018c55 f-Xo-- 2008-05-08T04:16:30 access_good.log 00018c6e f--o-- 2008-05-08T22:58:46 error.log 00018c6f f-X--- 2008-05-09T04:16:39 access_good.log 00018cb4 f--o-- 2008-05-11T03:01:36 error.log 00018ce7 f----- 2008-05-11T20:44:35 error.log 00018ce8 f----- 2008-05-11T20:44:14 access.log.2.gz 00018ce9 f----- 2008-05-11T20:46:18 access.log.1 00018cea f----- 2008-05-11T18:58:43 error.log.1 00018d03 f----- 2008-05-11T21:45:39 access.log query > quit -------------------------------------------------------------------------------------------- From boxbackup at fluffy.co.uk Tue May 13 11:58:16 2008 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 13 May 2008 11:58:16 +0100 Subject: [Box Backup] Deleting from backup Message-ID: <7DE20C13-E79F-476D-97EA-5E57C8A622AC@fluffy.co.uk> Chris Wilson wrote: > On Sun, 11 May 2008, Greg Bolshaw wrote: > >>> A 'delete ' would be good in bbackupquery. >> [...] >> Is it possible to automatically delete files from the backup after a >> number of days, rather than keeping them until the storage limit is >> reached? I would like to create a backup scheme where the last 30 >> days >> of files are always available, rather than the last n GB. > > Of course it's possible in theory but I'm afraid it's not > implemented yet. > Currently the store decides when to delete files based on its own > algorithm, which just tries to keep the account below the soft limit. There's a "delete as soon as possible" flag supported by the server, but not by bbackupquery. If there was some UI for setting this flag, it'd get deleted by the store on the next housekeeping run. Ben From boxbackup at fluffy.co.uk Wed May 14 15:19:56 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Wed, 14 May 2008 16:19:56 +0200 Subject: [Box Backup] New openssl packages fix predictable random number generator Message-ID: <482AF50C.8020400@trexler.at> This is a multi-part message in MIME format. --------------080605090102060801070101 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, does this debian-openssl bug affect boxbackup when running on a debian machine? ==> http://www.us.debian.org/security/2008/dsa-1571 If the boxbackup keys where generated on a debian machine, does that mean the server store is "crackable"? The debian advise is to change any keys generated on a debian machine, but given the concept of boxbackup a change of keys would demand to completely delete the store and newly backup the data. (Which will generate a lot of traffic and is not the same as the history is lost...) Am I right with my assumptions? best regards Wolfgang --------------080605090102060801070101 Content-Type: message/rfc822; name="[SECURITY] [DSA 1571-1] New openssl packages fix predictable random numbergenerator.eml" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename*0="[SECURITY] [DSA 1571-1] New openssl packages fix predictable"; filename*1=" random number generator.eml" Return-Path: Received: from zhaus.trexler.at ([unix socket]) by trexler.at (Cyrus v2.1.18-IPv6-Debian-2.1.18-5.1) with LMTP; Tue, 13 May 2008 14:28:29 +0200 X-Sieve: CMU Sieve 2.2 Received: from liszt.debian.org (liszt.debian.org [82.195.75.100]) by zhaus.trexler.at (8.13.8/8.13.8/Debian-3) with ESMTP id m4DCSTFf025949 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 13 May 2008 14:28:29 +0200 Received: from localhost (localhost [127.0.0.1]) by liszt.debian.org (Postfix) with QMQP id F26F813A5706; Tue, 13 May 2008 12:17:41 +0000 (UTC) Old-Return-Path: X-Original-To: debian-security-announce at lists.debian.org Delivered-To: lists-debian-security-announce at liszt.debian.org Received: from mail.enyo.de (mail.enyo.de [IPv6:2001:14b0:202:1::a7]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by liszt.debian.org (Postfix) with ESMTP id 5861C13A5660 for ; Tue, 13 May 2008 12:17:24 +0000 (UTC) Received: from deneb.vpn.enyo.de ([212.9.189.177] helo=deneb.enyo.de) by mail.enyo.de with esmtp id 1JvtHD-000669-HE for debian-security-announce at lists.debian.org; Tue, 13 May 2008 14:06:39 +0200 Received: from fw by deneb.enyo.de with local (Exim 4.69) (envelope-from ) id 1JvtHD-0002K2-31 for debian-security-announce at lists.debian.org; Tue, 13 May 2008 14:06:39 +0200 From: Florian Weimer To: debian-security-announce at lists.debian.org Date: Tue, 13 May 2008 14:06:39 +0200 Message-ID: <87od7az9v4.fsf at mid.deneb.enyo.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Debian: PGP check passed for security officers Subject: [SECURITY] [DSA 1571-1] New openssl packages fix predictable random number generator Priority: urgent Reply-To: debian-security at lists.debian.org Resent-Message-ID: Resent-From: debian-security-announce at lists.debian.org X-Mailing-List: archive/latest/245 X-Loop: debian-security-announce at lists.debian.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: Precedence: list Resent-Sender: debian-security-announce-request at lists.debian.org Resent-Date: Tue, 13 May 2008 12:17:41 +0000 (UTC) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - ------------------------------------------------------------------------ Debian Security Advisory DSA-1571-1 security at debian.org http://www.debian.org/security/ Florian Weimer May 13, 2008 http://www.debian.org/security/faq - ------------------------------------------------------------------------ Package : openssl Vulnerability : predictable random number generator Problem type : remote Debian-specific: yes CVE Id(s) : CVE-2008-0166 Luciano Bello discovered that the random number generator in Debian's openssl package is predictable. This is caused by an incorrect Debian-specific change to the openssl package (CVE-2008-0166). As a result, cryptographic key material may be guessable. This is a Debian-specific vulnerability which does not affect other operating systems which are not based on Debian. However, other systems can be indirectly affected if weak keys are imported into them. It is strongly recommended that all cryptographic key material which has been generated by OpenSSL versions starting with 0.9.8c-1 on Debian systems is recreated from scratch. Furthermore, all DSA keys ever used on affected Debian systems for signing or authentication purposes should be considered compromised; the Digital Signature Algorithm relies on a secret random value used during signature generation. The first vulnerable version, 0.9.8c-1, was uploaded to the unstable distribution on 2006-09-17, and has since propagated to the testing and current stable (etch) distributions. The old stable distribution (sarge) is not affected. Affected keys include SSH keys, OpenVPN keys, DNSSEC keys, and key material for use in X.509 certificates and session keys used in SSL/TLS connections. Keys generated with GnuPG or GNUTLS are not affected, though. A detector for known weak key material will be published at: (OpenPGP signature) Instructions how to implement key rollover for various packages will be published at: This web site will be continously updated to reflect new and updated instructions on key rollovers for packages using SSL certificates. Popular packages not affected will also be listed. In addition to this critical change, two other vulnerabilities have been fixed in the openssl package which were originally scheduled for release with the next etch point release: OpenSSL's DTLS (Datagram TLS, basically "SSL over UDP") implementation did not actually implement the DTLS specification, but a potentially much weaker protocol, and contained a vulnerability permitting arbitrary code execution (CVE-2007-4995). A side channel attack in the integer multiplication routines is also addressed (CVE-2007-3108). For the stable distribution (etch), these problems have been fixed in version 0.9.8c-4etch3. For the unstable distribution (sid) and the testing distribution (lenny), these problems have been fixed in version 0.9.8g-9. We recommend that you upgrade your openssl package and subsequently regenerate any cryptographic material, as outlined above. Upgrade instructions - -------------------- wget url will fetch the file for you dpkg -i file.deb will install the referenced file. If you are using the apt-get package manager, use the line for sources.list as given below: apt-get update will update the internal database apt-get upgrade will install corrected packages You may use an automated update by adding the resources from the footer to the proper configuration. Debian GNU/Linux 4.0 alias etch - ------------------------------- Source archives: http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3.dsc Size/MD5 checksum: 1099 5e60a893c9c3258669845b0a56d9d9d6 http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c.orig.tar.gz Size/MD5 checksum: 3313857 78454bec556bcb4c45129428a766c886 http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3.diff.gz Size/MD5 checksum: 55320 f0e457d6459255da86f388dcf695ee20 alpha architecture (DEC Alpha) http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_alpha.deb Size/MD5 checksum: 1025954 d82f535b49f8c56aa2135f2fa52e7059 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_alpha.deb Size/MD5 checksum: 4558230 399adb0f2c7faa51065d4977a7f3b3c4 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_alpha.deb Size/MD5 checksum: 2620892 0e5efdec0a912c5ae56bb7c5d5d896c6 http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_alpha.deb Size/MD5 checksum: 2561650 affe364ebcabc2aa33ae8b8c3f797b5e http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_alpha.udeb Size/MD5 checksum: 677172 5228d266c1fc742181239019dbad4c42 amd64 architecture (AMD x86_64 (AMD64)) http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_amd64.deb Size/MD5 checksum: 1654902 d8ad8dc51449cf6db938d2675789ab25 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_amd64.deb Size/MD5 checksum: 891102 2e97e35c44308a59857d2e640ddf141a http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_amd64.deb Size/MD5 checksum: 992248 82193ea11b0bc08c74a775039b855a05 http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_amd64.deb Size/MD5 checksum: 2178610 fb7c53e5f157c43753db31885ff68420 http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_amd64.udeb Size/MD5 checksum: 580250 7fb3d7fee129cc9a4fb21f5c471dfbab arm architecture (ARM) http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_arm.deb Size/MD5 checksum: 1537440 c5ab48e9bde49ba32648fb581b90ba18 http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_arm.udeb Size/MD5 checksum: 516576 84385b137c731de3b86824c17affa9f3 http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_arm.deb Size/MD5 checksum: 2049882 7ed60840eb3e6b26c6856dcaf5776b0c http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_arm.deb Size/MD5 checksum: 1011698 abfa887593089ac0f1cd4e31154897ee http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_arm.deb Size/MD5 checksum: 805912 a605625ea107252e9aebbc77902a63ed hppa architecture (HP PA RISC) http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_hppa.deb Size/MD5 checksum: 1585900 2cbe55764db351dc6c3c2d622aa90caf http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_hppa.deb Size/MD5 checksum: 2248328 664fb0992b786ce067a7d878056fc191 http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_hppa.deb Size/MD5 checksum: 1030782 21f445c541d5e5b7c16de1db9ee9d681 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_hppa.deb Size/MD5 checksum: 945144 c1092f3bb94d920d0beaa372c9cab04e http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_hppa.udeb Size/MD5 checksum: 631132 76339119275786b5e80a7a1b4cd26b71 i386 architecture (Intel ia32) http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_i386.deb Size/MD5 checksum: 2086512 eeef437fb87ad6687cd953d5951aa472 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_i386.deb Size/MD5 checksum: 5584696 6d364557c9d392bb90706e049860be66 http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_i386.deb Size/MD5 checksum: 1000832 ed5668305f1e4b4e4a22fbd24514c758 http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_i386.udeb Size/MD5 checksum: 554676 dbad0172c990359282884bac1d141034 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_i386.deb Size/MD5 checksum: 2717086 361fde071d18ccf93338134357ab1a61 ia64 architecture (Intel ia64) http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_ia64.udeb Size/MD5 checksum: 801748 05b29fc674311bd31fe945036a08abd5 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_ia64.deb Size/MD5 checksum: 1192192 56be85aceb4e79e45f39c4546bfecf4f http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_ia64.deb Size/MD5 checksum: 2593418 f9edaea0a86c1a1cea391f890d7ee70f http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_ia64.deb Size/MD5 checksum: 1569418 4b2cb04d13efabdddddbd0f6d3cefd9b http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_ia64.deb Size/MD5 checksum: 1071156 e1f487c4310ad526c071f7483de4cd1a mips architecture (MIPS (Big Endian)) http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_mips.deb Size/MD5 checksum: 1003816 f895a8bc714e9c373ee80f736b5af00b http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_mips.deb Size/MD5 checksum: 2262266 004484e816d4fe5ff03fe6d7df38d7b7 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_mips.deb Size/MD5 checksum: 1692606 e8273f5d123f892a81a155f14ba19b50 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_mips.deb Size/MD5 checksum: 875558 44074bce1cde4281c5abcf45817f429d http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_mips.udeb Size/MD5 checksum: 580130 b6b810d1c39164747e3ebc9df4903974 mipsel architecture (MIPS (Little Endian)) http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_mipsel.udeb Size/MD5 checksum: 566168 97963ca9b6ada94445fb25b3126655e9 http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_mipsel.deb Size/MD5 checksum: 992712 41c2bbe984553d693f21c3ec349ea465 http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_mipsel.deb Size/MD5 checksum: 2255558 3c63936cd511975291b4230bef1a2e3b http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_mipsel.deb Size/MD5 checksum: 860506 d580fbeed6efd734245ea7a7bed225bb http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_mipsel.deb Size/MD5 checksum: 1649300 3315d1406f995f5b6d2a4f958976a794 powerpc architecture (PowerPC) http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_powerpc.deb Size/MD5 checksum: 1002022 b2749639425c3a8ac493e072cfffb358 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_powerpc.deb Size/MD5 checksum: 895460 e15fbbbbcfe17e82bacc07f6febd9707 http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_powerpc.udeb Size/MD5 checksum: 585320 61488ea7f54b55a21f7147fe5bc3b0f0 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_powerpc.deb Size/MD5 checksum: 1728384 539ee1a3fe7d9b89034ebfe3c1091b6f http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_powerpc.deb Size/MD5 checksum: 2210792 82e9e27c6083a95c76c5817f9604178f s390 architecture (IBM S/390) http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_s390.udeb Size/MD5 checksum: 643008 4861c78ea63b6c3c08c22a0c5326d981 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_s390.deb Size/MD5 checksum: 1632976 01d289d460622382b59d07950305764f http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_s390.deb Size/MD5 checksum: 951404 d92bb390489bed0abff58f7a1ceade6b http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_s390.deb Size/MD5 checksum: 1014308 487c24f2af25797a857814af7c9c0d0b http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_s390.deb Size/MD5 checksum: 2193782 f1fe472c802e929a57bd8c8560bd3009 sparc architecture (Sun SPARC/UltraSPARC) http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8-dbg_0.9.8c-4etch3_sparc.deb Size/MD5 checksum: 4091340 970453ebfab8152c9c44ae210fbaa2a4 http://security.debian.org/pool/updates/main/o/openssl/libcrypto0.9.8-udeb_0.9.8c-4etch3_sparc.udeb Size/MD5 checksum: 539054 7be1258f74165c4b037e202d2048f8ce http://security.debian.org/pool/updates/main/o/openssl/openssl_0.9.8c-4etch3_sparc.deb Size/MD5 checksum: 1010536 6444d6cc6fd838c82716462aacd1cf84 http://security.debian.org/pool/updates/main/o/openssl/libssl-dev_0.9.8c-4etch3_sparc.deb Size/MD5 checksum: 2108000 ab0d0ccc72764a26b7767cace520b269 http://security.debian.org/pool/updates/main/o/openssl/libssl0.9.8_0.9.8c-4etch3_sparc.deb Size/MD5 checksum: 2126386 61ddc204ee650cdd0f2b56e358134e2b These files will probably be moved into the stable distribution on its next update. - --------------------------------------------------------------------------------- For apt-get: deb http://security.debian.org/ stable/updates main For dpkg-ftp: ftp://security.debian.org/debian-security dists/stable/updates/main Mailing list: debian-security-announce at lists.debian.org Package info: `apt-cache show ' and http://packages.debian.org/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSCmDjL97/wQC1SS+AQLZGgf8Dp7Rj1HmC4n0QowM9cRnzw24upFQ1bpq SbkU/NhkoLORcMnXsnVPL30bmtpXltjpWuKIuRGzudXBonXaZtX1N4rl9HDpN+gt AZJdxweSSmwQNyvOyPRKDVJ1w/YYiaJnSIDNks6NqSNYSEAb5L3bHBeHDTgLsWMW jYcF5GJSt8yG3GvA0FyFIPwJihr2YF/RmhpurGQf3XO6S94cDsdLtr/KOcdmdWze 39E+2h3L34HGIwVUgK9uY8Gv0DCPqhQZ4157CteFpQwQoKzFSxYApruCm4QcFxV+ BxuB/M9M5tPWrX1slffG+q3YHK0mDnB9d2JqSwQ5TD9kxTiwEEY8sQ== =lX6B -----END PGP SIGNATURE----- -- To UNSUBSCRIBE, email to debian-security-announce-REQUEST at lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster at lists.debian.org --------------080605090102060801070101-- From boxbackup at fluffy.co.uk Wed May 14 17:49:19 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Wed, 14 May 2008 18:49:19 +0200 Subject: [Box Backup] Help with include/exclude In-Reply-To: <482AF50C.8020400@trexler.at> References: <482AF50C.8020400@trexler.at> Message-ID: <482B180F.9080403@homann.se> Hi, I need som ehlp with exclude/include I have directories of the form: /path/dirA/ /path/dirB/ /path/dirC/ and so on. In each of these subdirecotries there are a few files that I want to back up, and two directories. /path/dirX/file1 /path/dirX/file2 /path/dirX/file3 /path/dirX/file4 /path/dirX/subdir1 /path/dirX/subdir2 Can someone suggest an efficient configuration for this? Magnus From boxbackup at fluffy.co.uk Wed May 14 18:19:04 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 14 May 2008 19:19:04 +0200 (CAT) Subject: [Box Backup] Help with include/exclude In-Reply-To: <482B180F.9080403@homann.se> References: <482AF50C.8020400@trexler.at> <482B180F.9080403@homann.se> Message-ID: Hi Magnus, On Wed, 14 May 2008, Magnus Homann wrote: > I have directories of the form: > > /path/dirA/ > /path/dirB/ > /path/dirC/ > > and so on. > > In each of these subdirecotries there are a few files that I want to > back up, and two directories. > > /path/dirX/file1 > /path/dirX/file2 > /path/dirX/file3 > /path/dirX/file4 > /path/dirX/subdir1 > /path/dirX/subdir2 > > Can someone suggest an efficient configuration for this? ExcludeFilesRegex = ^/path/(dirA|dirB|dirC)/ ExcludeDirsRegex = ^/path/(dirA|dirB|dirC)/ AlwaysIncludeFilesRegex = (file1|file2|file3|file4)$ AlwaysIncludeDirsRegex = ^/path/(dirA|dirB|dirC)/(subdir1|subdir2) AlwaysIncludeFilesRegex = ^/path/(dirA|dirB|dirC)/(subdir1|subdir2) Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 14 18:26:29 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 14 May 2008 19:26:29 +0200 (CAT) Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <482AF50C.8020400@trexler.at> References: <482AF50C.8020400@trexler.at> Message-ID: Hi Wolfgang, On Wed, 14 May 2008, Wolfgang Trexler wrote: > does this debian-openssl bug affect boxbackup when running on a debian > machine? ==> http://www.us.debian.org/security/2008/dsa-1571 > > If the boxbackup keys where generated on a debian machine, does that > mean the server store is "crackable"? > > The debian advise is to change any keys generated on a debian machine, but > given the concept of boxbackup a change of keys would demand to completely > delete the store and newly backup the data. (Which will generate a lot of > traffic and is not the same as the history is lost...) > > Am I right with my assumptions? Partly. It looks like clients running Debian may have generated poor keys for their accounts, which may make it possible for an attacker to log into their accounts. Similarly, a Debian server may have generated weak server keys that would allow an attacker to impersonate a server. However the data encryption key is probably not vulnerable to this attack and therefore your data should be safe even if your account is compromised. Changing certificate keys is not painful, but changing data encryption keys is. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 14 19:22:13 2008 From: boxbackup at fluffy.co.uk (Kenny Millington) Date: Wed, 14 May 2008 19:22:13 +0100 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: References: <482AF50C.8020400@trexler.at> Message-ID: <1210789333.7416.0.camel@shodan> Hi, > However the data encryption key is probably not vulnerable to this attack > and therefore your data should be safe even if your account is > compromised. So out of interest how is the data encryption key generated? Thanks, -- Kenny Millington Systems Developer kenny.millington at 3ait.co.uk 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ CoReg: 3866698 VATReg: 771388600 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. From boxbackup at fluffy.co.uk Wed May 14 22:05:43 2008 From: boxbackup at fluffy.co.uk (boxbackup at fluffy.co.uk) Date: Wed, 14 May 2008 14:05:43 -0700 (PDT) Subject: [Box Backup] Debian dev servers Message-ID: <47565.64.161.120.82.1210799143.squirrel@www.vancocomputing.net> Chris, the servers for BoxBackup development I promised are ready for you, but my e-mails to you offlist are not getting through. Could you e-mail or me offlist so that I can send you the server info? Thanks! -- Sean Van Couwenberghe Vanco Computing From boxbackup at fluffy.co.uk Wed May 14 22:16:13 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Wed, 14 May 2008 23:16:13 +0200 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210789333.7416.0.camel@shodan> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> Message-ID: <1210799773.9122.6.camel@localhost.localdomain> By letting OpenSSL generate 1024 random characters. It's the 'openssl rand -out -FileEncKeys.raw 1024' command that is used. Bjarne ons, 14 05 2008 kl. 19:22 +0100, skrev Kenny Millington: > Hi, > > > However the data encryption key is probably not vulnerable to this attack > > and therefore your data should be safe even if your account is > > compromised. > > So out of interest how is the data encryption key generated? > > > Thanks, From boxbackup at fluffy.co.uk Wed May 14 22:29:44 2008 From: boxbackup at fluffy.co.uk (Kenny Millington) Date: Wed, 14 May 2008 22:29:44 +0100 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210799773.9122.6.camel@localhost.localdomain> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> Message-ID: <1210800584.7416.7.camel@shodan> Hi, > By letting OpenSSL generate 1024 random characters. It's the 'openssl > rand -out -FileEncKeys.raw 1024' command that is used. Ah! That could be a problem then... (having checked by asking on #debian) "openssl rand ..." was also affected by problem reported in the Debian Security Advisory. This means that if any data encryption keys were generated on vulnerable hosts they need to be regenerated or the data cannot be considered secure (given the amount of entropy that would have been used). So, um, don't shoot the messenger! :o) -- Kenny Millington Systems Developer kenny.millington at 3ait.co.uk 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ CoReg: 3866698 VATReg: 771388600 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. From boxbackup at fluffy.co.uk Thu May 15 00:22:37 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Thu, 15 May 2008 01:22:37 +0200 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210800584.7416.7.camel@shodan> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> <1210800584.7416.7.camel@shodan> Message-ID: <1210807357.9122.40.camel@localhost.localdomain> No gun is in my hand - see! ;) You are absolutely correct. The author has chosen to seed the prng by running through the (insecure on Debian/-derivatives) function ssleay_rand_add a number of times, meaning that in the previous versions of OpenSSL, which had the mixing function commented out on those systems, the input was not mixed properly into the the state of the prng. That said, Box Backup uses the raw output from the prng, not the generated .pem files, so the FileEncKeys.raw should be considerably more secure - the raw data has a SHA-1 hash mixed in after all, and collisions in SHA-1 in this context only means that two keys have a finite and very small possibility of ending up the same, and they could conceivably do that even on a non-Debian system. A regeneration of FileEncKeys.raw after upgrading OpenSSL to the latest verson should be considered by all Debian and -derivative users though. As I read the Box Backup docs, this would mean that all earlier backups will be valueless, unless the old (possibly insecure) keys are kept, and you would presumably have to back up and clear the store, and then start a new backup cycle on the fresh store while keeping the old backups in another place. Oh my! Now it's me who am going to get shot... Bjarne ons, 14 05 2008 kl. 22:29 +0100, skrev Kenny Millington: > Hi, > > > By letting OpenSSL generate 1024 random characters. It's the 'openssl > > rand -out -FileEncKeys.raw 1024' command that is used. > > Ah! That could be a problem then... (having checked by asking on > #debian) "openssl rand ..." was also affected by problem reported in the > Debian Security Advisory. > > This means that if any data encryption keys were generated on vulnerable > hosts they need to be regenerated or the data cannot be considered > secure (given the amount of entropy that would have been used). > > So, um, don't shoot the messenger! :o) > From boxbackup at fluffy.co.uk Thu May 15 07:37:57 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Thu, 15 May 2008 08:37:57 +0200 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210807357.9122.40.camel@localhost.localdomain> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> <1210800584.7416.7.camel@shodan> <1210807357.9122.40.camel@localhost.localdomain> Message-ID: <482BDA45.9010800@homann.se> Bjarne Carlsen wrote: > As I read the Box Backup docs, this would mean that all earlier backups > will be valueless, unless the old (possibly insecure) keys are kept, and > you would presumably have to back up and clear the store, and then start > a new backup cycle on the fresh store while keeping the old backups in > another place. Interesting. If I just change the key without starint all over, will then boxbackuip conclude that all my files have changed, and upload a new version without any problems, or is the infrastructure around the files (pointers, lists, whatnots) also dependant on th ekeys. Can I then retrieve old files by just stopping dameons and copying back the old .*, starting bbstored an do a 'get - ' in bbackupquery? Magnus From boxbackup at fluffy.co.uk Thu May 15 09:27:44 2008 From: boxbackup at fluffy.co.uk (Kenny Millington) Date: Thu, 15 May 2008 09:27:44 +0100 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210807357.9122.40.camel@localhost.localdomain> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> <1210800584.7416.7.camel@shodan> <1210807357.9122.40.camel@localhost.localdomain> Message-ID: <1210840064.18401.76.camel@helios> Hi, Disclaimer: I'm about to talk about stuff I don't entirely understand... but it's a discussion right.. ;) > That said, Box Backup uses the raw output from the prng, not the > generated .pem files, so the FileEncKeys.raw should be considerably more > secure - the raw data has a SHA-1 hash mixed in after all, and > collisions in SHA-1 in this context only means that two keys have a > finite and very small possibility of ending up the same, and they could > conceivably do that even on a non-Debian system. Ok, so my (probably limited understanding) is that the weakness is due to poor random number generator seeding. Such that the seed is a very limited space (maybe just the PID). I'm guessing this from the fact that ssleay_rand_seed() only calls the bugged "ssleay_rand_add()" function. So if the seeding is poor/weak surely (or not) that would mean the seeds were guessable (maybe just PID) then surely it'd be trivial to iterate over all the possible seeds generating the streams of prng output until the generated output is able to decrypt data in the store? It's reported on the appropriate Debian wiki[1] page that the bugged libssl was only generating 2^15 unique keys - if this was exclusively due to poor prng seeding then I would expect boxbackup data encryption keys to have the same problem. (If that makes no sense I refer you to my disclaimer above... ;)) > A regeneration of FileEncKeys.raw after upgrading OpenSSL to the latest > verson should be considered by all Debian and -derivative users though. Indeed. > Oh my! Now it's me who am going to get shot... I think it's probably both of us... [1] http://wiki.debian.org/SSLkeys -- Kenny Millington Systems Developer kenny.millington at 3ait.co.uk 3aIT Limited - Official Corporate Sponsor of the British Bobsleigh Team 4-10 Barttelot Rd Horsham West Sussex RH12 1DQ CoReg: 3866698 VATReg: 771388600 T: +44 (0)870 881 5097 F: +44 (0)870 116 0793 Visit www.3aIT.co.uk for Design, Systems, Support Disclaimer: The information contained within this email is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying or distribution of this email is prohibited and may be unlawful. The content of this email represents the views of the individual and not necessarily 3aIT Limited. 3aIT Limited reserves the right to monitor the content of all emails in accordance with lawful business practice. Whilst every effort is made to ensure that attachments are free from computer viruses before transmission, 3aIT Limited does not accept any liability in respect of any virus that is not detected. From boxbackup at fluffy.co.uk Thu May 15 12:35:05 2008 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 15 May 2008 12:35:05 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data Message-ID: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> Right then. The big problem is that this breaks a Box Backup claim that you don't need to trust your server administrator, plus it makes the authentication method for getting access to your data useless. Since the keys used to encrypt your data and log into the server are weak, you should consider it exposed to the server administrator and anyone who may have logged in with forged certificates. I advise Box Backup users who generated their certificates/keys on affected Debian system to consider the security of their backups compromised. The server admin or anyone able to deduce the private key of a server or client certificates could have read your data. If the PRNG in your OpenSSL was insufficiently random, you need to: * Regenerate all affected certificates, which have been generated or signed on an affected system * Regenerate all the data keys (*-FileEncKeys.raw) * Destroy the data stored on your server to an appropriate level of security (overwrite with zeros at the least, more if you're paranoid) * Upload everything again * Take appropriate measures under the assumption that you have been storing your data in plain text on a public server without authentication. (ie, start from scratch, destroying all trace of the backed up data, and take other measures to mitigate the exposure of your secrets) You need only worry about the systems where: * The certificates were generated or signed * The .raw keys were generated * The client which backed up data If your server has this flaw, but no key material or signing was done on it, you're fine (as I understand it -- if the PRNG affects certificate exchanges for authentication then there's a problem). If your certs are weak but the .raw keys are fine, assume that your data has not been read, but that an attacker logged in and corrupted your backups. Destroy the data and start again. If your certs are fine but a client's .raw file isn't OR an affected client backed up data, just destroy data for that client and restart with that client. Assume that client's data has been exposed to the server admin, but not the outside world. Note when I say the "certs are fine" I mean that all cryptographic operations were done on an unaffected machine, including the generation of the client certificate keys before signing elsewhere. I suppose we better post something about this on the home page, after someone else has checked these statements. Ben From boxbackup at fluffy.co.uk Thu May 15 12:39:46 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Thu, 15 May 2008 13:39:46 +0200 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210840064.18401.76.camel@helios> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> <1210800584.7416.7.camel@shodan> <1210807357.9122.40.camel@localhost.localdomain> <1210840064.18401.76.camel@helios> Message-ID: <1210851587.9122.70.camel@localhost.localdomain> > It's reported on the appropriate Debian wiki[1] page that the bugged > libssl was only generating 2^15 unique keys - if this was exclusively > due to poor prng seeding then I would expect boxbackup data encryption > keys to have the same problem. > > (If that makes no sense I refer you to my disclaimer above... ;)) Box Backup does not _really_ fall under the bad spell: - The key-generation with the rand function is not easily repeatable. An attacker will have to determine his attack vector not from a fixed set of generated keys, but from the underlying keystream. While the stream itself may be repeatable and may not be too long, (indeed most keystreams are repeatable), the starting point within the stream needs to be determined, and that is no trivial task. - The data are vulnerable only at a few points: - In transit, where getting hold of them is not excactly trivial, and where they are easily protected by regenerating certificates - On the server, requiring access to the server, together with knowledge of which account holds the vulnerable data - The keys do not neccessarily have to be unique - just unknown to an attacker - he will not be able to separate strongly protected data from weakly protected, so he will have quite a job set up for him. The way I see it, we are defending against the usual suspects here: The malicious server operator, anyone from the outside able to gain privileges on the server and finally outright theft of the whole server. The man in the middle is not interesting, because of the easiness with which you can regenerate and re-issue your certificates. Those are culprits, we have been dealing with since time immemorial... Bjarne From boxbackup at fluffy.co.uk Thu May 15 14:01:24 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Thu, 15 May 2008 15:01:24 +0200 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> Message-ID: <1210856484.9122.103.camel@localhost.localdomain> tor, 15 05 2008 kl. 12:35 +0100, skrev Ben Summers: > I advise Box Backup users who generated their certificates/keys on > affected Debian system to consider the security of their backups > compromised. The server admin or anyone able to deduce the private > key > of a server or client certificates could have read your data. There are several scenarios that need to be taken into consideration: * Alice generates her *.crt and *.FileEncKeys.raw on a secure system, but Bob signs the *.crt on a compromised system. In this case Alice's certificates are to be considered compromised, and are to be re-issued using a secure box for signing, but her data are secure. To a certain extent even secure from tampering by Mike spoofing Alice in the compromise period, since Mike would only have been able to read data which would be of no use to him, and write data, but not encrypted to Alice's key. Mike could not modify data which were already in place, (they were, and are, encrypted to Alice's key). This would seem to be the most prolific scenario, given that backup vendors cater mostly to Windows systems that subsequently push their backups to a possibly insecure Debian/-derivative box. * Alice generates her *.crt and *.FileEncKeys.raw on an insecure system. Whether or not Bob's system was secure at the time is of no consequence to the matter, the certificates and backups are to be considered compromised. Probably the second most prolific scenario - that of backing up servers to servers. * Alice lets Bob generate both *.crt and *.FileEncKeys.raw, sign the *.crt and send the whole shebang back to Alice. This means that Alice has implicit trust in Bob. Not a preferred scenario, but it exists out there! The security of both certificates and keys are blown no matter what the status of the two systems are, since this violates the basic rule that Alice should not trust Bob. Bjarne From boxbackup at fluffy.co.uk Thu May 15 19:20:43 2008 From: boxbackup at fluffy.co.uk (James O'Gorman) Date: Thu, 15 May 2008 19:20:43 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> Message-ID: <482C7EFB.9060701@netinertia.co.uk> Ben Summers wrote: [snip] > I suppose we better post something about this on the home page, after > someone else has checked these statements. As a semi-proactive measure I have updated the website with Ben's statements. If anyone wishes to correct or update any part of it, please e-mail me directly (preferably with a patch generated from "svn diff" :-) ). James From boxbackup at fluffy.co.uk Thu May 15 20:58:03 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Thu, 15 May 2008 21:58:03 +0200 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <482C7EFB.9060701@netinertia.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <482C7EFB.9060701@netinertia.co.uk> Message-ID: <482C95CB.1050709@trexler.at> James O'Gorman schrieb: > Ben Summers wrote: > [snip] >> I suppose we better post something about this on the home page, after >> someone else has checked these statements. > > As a semi-proactive measure I have updated the website with Ben's > statements. If anyone wishes to correct or update any part of it, please > e-mail me directly (preferably with a patch generated from "svn diff" > :-) ). I asked the debian maintainer of boxbackup Reinhard Tartler to post something in the debian SSL-Bug wiki, what Reinhard already did. Many thanks (btw)! see http://wiki.debian.org/SSLkeys Wolfgang From boxbackup at fluffy.co.uk Fri May 16 16:02:54 2008 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Fri, 16 May 2008 17:02:54 +0200 Subject: [Box Backup] Sugested filesystem? Message-ID: <482DA21E.3080404@kontrapunkt.com> Hello... I'm running boxbackup 0.11. I am setting up a new BB store on Debian machine. I'm wondering what filesystem I should choose for the BB store files. Any suggestions? Thanks, Tobias Balle-Petersen From boxbackup at fluffy.co.uk Fri May 16 18:28:53 2008 From: boxbackup at fluffy.co.uk (Brent Saner) Date: Fri, 16 May 2008 13:28:53 -0400 Subject: [Box Backup] Sugested filesystem? In-Reply-To: <482DA21E.3080404@kontrapunkt.com> References: <482DA21E.3080404@kontrapunkt.com> Message-ID: <482DC455.506@acetechgroup.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 Tobias Balle-Petersen wrote: I'm wondering what filesystem I should choose for the BB store | files. Any suggestions? depends on the size you plan on the store's files to be, but usually ext3 is great as it's a pretty wide-read standard. if you don't need or want journaling, ext2 is also great. ext4 is in the works, but is still beta. if you want speed, ReiserFS is pretty great, except it's DARN hard to do disaster recovery on. usually, though, you don't need the speed performance for backups anyways since it's limited by the data link and the client's I/O as well. i'd say ext3 is your best bet. - -- I wish ZORK had co-op mode. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFILcRV8u2Zh4MtlQoRAy99AJ0aZTZ13glPtmgKtSv6Q3LRhzy31ACfc9qt vWXJJ4U2r+eB5lPij5v/1Sc= =IfdL -----END PGP SIGNATURE----- From boxbackup at fluffy.co.uk Fri May 16 17:42:30 2008 From: boxbackup at fluffy.co.uk (dave bamford) Date: Fri, 16 May 2008 17:42:30 +0100 Subject: [Box Backup] Sugested filesystem? In-Reply-To: <482DA21E.3080404@kontrapunkt.com> References: <482DA21E.3080404@kontrapunkt.com> Message-ID: <482DB976.6040000@logical-progress.com> Tobias Balle-Petersen wrote: > Hello... > > I'm running boxbackup 0.11. I am setting up a new BB store on Debian > machine. I'm wondering what filesystem I should choose for the BB > store files. Any suggestions? > > Thanks, > Tobias Balle-Petersen > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > Tobias I use jfs on my Debin stores as part of a raid1 array. Regards Dave Bamford From boxbackup at fluffy.co.uk Fri May 16 20:30:47 2008 From: boxbackup at fluffy.co.uk (Louis-Dominique Dubeau) Date: Fri, 16 May 2008 15:30:47 -0400 Subject: [Box Backup] lazy and snapshot together, file formats and files changing often Message-ID: <1210966247.7008.71.camel@bodhi> Hello everyone, I'm currently testing and evaluating boxbackup. I have three questions: 1. Is it possible to set up boxbackup to have a client perform both lazy backups and snapshots of a single set of files, to a single backup store? e.g. Boxbackup would generally be in lazy mode except that once a day it would perform a snapshot. The only thing changing between the two kinds of backups would be the lazy vs snapshot mode. All other parameters would remain the same. 2. It is possible to access the backup store outside of boxbackup? What I mean is that if the store was a collection of .tar.gz files encrypted with gnupg, then I would just need gnupg, tar and gunzip to peek at the contents of the store. Is the backup store created by boxbackup like this or does it have some sort of boxbackup-specific format? 3. As a small quick and dirty test, I've set boxbackup to perform lazy backups my ~./mozilla directory. I figured there are enough changes in there to trigger lazy backups. Anyway, I found that some of the files in the cache maintained by firefox are not backed up even after several hours. Then I thought that maybe files which change too often are never backed up. The way I understand lazy backups, boxbackup waits until a file has remained unchanged for some time before it backs it up. So if the file changes too often maybe that time period never elapses and the file is never backed up. So can it happen that a file which is frequently changing never gets picked up by a lazy backup? Must I force a snapshot at some point? Thanks, Louis From boxbackup at fluffy.co.uk Fri May 16 21:51:13 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 16 May 2008 21:51:13 +0100 (BST) Subject: [Box Backup] lazy and snapshot together, file formats and files changing often In-Reply-To: <1210966247.7008.71.camel@bodhi> References: <1210966247.7008.71.camel@bodhi> Message-ID: Hi Louis, On Fri, 16 May 2008, Louis-Dominique Dubeau wrote: > I'm currently testing and evaluating boxbackup. I have three questions: > > 1. Is it possible to set up boxbackup to have a client perform both > lazy backups and snapshots of a single set of files, to a single > backup store? > > e.g. Boxbackup would generally be in lazy mode except that once a day it > would perform a snapshot. The only thing changing between the two kinds > of backups would be the lazy vs snapshot mode. All other parameters > would remain the same. Yes that's absolutely fine. The only difference between lazy and snapshot modes is that in lazy mode, bbackupd automatically performs a sync at regular intervals, whereas in snapshot mode, it only does so when asked. You can run in lazy mode and request snapshot syncs whenever you want. > 2. It is possible to access the backup store outside of boxbackup? > > What I mean is that if the store was a collection of .tar.gz files > encrypted with gnupg, then I would just need gnupg, tar and gunzip to > peek at the contents of the store. Is the backup store created by > boxbackup like this or does it have some sort of boxbackup-specific > format? Not really, because the individual files are encrypted and stored in a format that allows them to be diffed while encrypted, and there is no standard for such a format that I'm aware of, there exist no tools other than Box Backup that can process these files. Having said that, it wouldn't be hard to modify bbackupquery to read and decrypt the encrypted files itself, without using the network. Encrypted directories would be a little harder. > 3. As a small quick and dirty test, I've set boxbackup to perform lazy > backups my ~./mozilla directory. I figured there are enough changes > in there to trigger lazy backups. > > Anyway, I found that some of the files in the cache maintained by > firefox are not backed up even after several hours. Then I thought that > maybe files which change too often are never backed up. The way I > understand lazy backups, boxbackup waits until a file has remained > unchanged for some time before it backs it up. So if the file changes > too often maybe that time period never elapses and the file is never > backed up. > > So can it happen that a file which is frequently changing never gets > picked up by a lazy backup? Must I force a snapshot at some point? No, there is a MaxUploadWait parameter which specifies after how long the client will give up waiting for the file to be quiescent and back it up anyway. However, it's more likely to be in an inconsistent state when this happens, as the application may not have finished writing to it. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 16 22:53:19 2008 From: boxbackup at fluffy.co.uk (Louis-Dominique Dubeau) Date: Fri, 16 May 2008 17:53:19 -0400 Subject: [Box Backup] lazy and snapshot together, file formats and files changing often In-Reply-To: References: <1210966247.7008.71.camel@bodhi> Message-ID: <1210974799.7008.110.camel@bodhi> On Fri, 2008-05-16 at 21:51 +0100, Chris Wilson wrote: > Hi Louis, Hi. > Yes that's absolutely fine. The only difference between lazy and snapshot > modes is that in lazy mode, bbackupd automatically performs a sync at > regular intervals, whereas in snapshot mode, it only does so when asked. > You can run in lazy mode and request snapshot syncs whenever you want. I am not clear on how I can achieve this. I tried *not* changing my .conf file (that is, my conf is still set for lazy backups) and issuing: $ sudo bbackupctl sync But I still see files missing from the store when I check with bbackupquery. However, if I create a new config to do a snapshot, tell the daemon to reload its config and then issue bbackupctl sync, then the snapshot goes through. So if I want to do a sync once every day, I need every day to switch the configuration from lazy to snapshot, issue a sync command, and then switch the configuration back from snapshot to lazy? > No, there is a MaxUploadWait parameter which specifies after how long the > client will give up waiting for the file to be quiescent and back it up > anyway. However, it's more likely to be in an inconsistent state when this > happens, as the application may not have finished writing to it. Ooops. I missed that parameter. I'll just have to figure out how I want to handle the possibility of inconsistency. The way I currently do backups, I log out from KDE, create an LVM snapshot of my home, log back into KDE and use some home-baked glue script to start rsync to backup from the LVM snapshot instead of my actual home. When the backup is done I destroy the snapshot. Thanks, Louis From boxbackup at fluffy.co.uk Fri May 16 23:17:30 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 16 May 2008 23:17:30 +0100 (BST) Subject: [Box Backup] lazy and snapshot together, file formats and files changing often In-Reply-To: <1210974799.7008.110.camel@bodhi> References: <1210966247.7008.71.camel@bodhi> <1210974799.7008.110.camel@bodhi> Message-ID: Hi Louis, On Fri, 16 May 2008, Louis-Dominique Dubeau wrote: > > The only difference between lazy and snapshot modes is that in lazy > > mode, bbackupd automatically performs a sync at regular intervals, > > whereas in snapshot mode, it only does so when asked. You can run in > > lazy mode and request snapshot syncs whenever you want. > > I am not clear on how I can achieve this. I tried *not* changing my > .conf file (that is, my conf is still set for lazy backups) and issuing: > > $ sudo bbackupctl sync > > But I still see files missing from the store when I check with > bbackupquery. Snapshot mode does not force all files to be backed up. The same rules about MinimumFileAge, MaxUploadWait, etc. are still observed. You could argue that this is a bug. > However, if I create a new config to do a snapshot, tell the daemon to > reload its config and then issue bbackupctl sync, then the snapshot goes > through. So if I want to do a sync once every day, I need every day to > switch the configuration from lazy to snapshot, issue a sync command, > and then switch the configuration back from snapshot to lazy? Not at all, you can and should leave it in lazy mode. Probably all that happened is that the files became old enough while you went through the process of changing the config that they were backed up anyway. > I'll just have to figure out how I want to handle the possibility of > inconsistency. The way I currently do backups, I log out from KDE, > create an LVM snapshot of my home, log back into KDE and use some > home-baked glue script to start rsync to backup from the LVM snapshot > instead of my actual home. When the backup is done I destroy the > snapshot. You could do something similar by configuring the bbackupd NotifyScript to create a snapshot before starting the sync, delete it afterwards, and back up the snapshot instead of your home. Lazy backups could still be inconsistent, but if you log yourself out before running the snapshot sync, it would be consistent. If you do it this way, then at least your backups are all crash-consistent, in other words what you have is the exact state that the files would be in if the system crashed at that point. Since most applications are designed to at least maintain crash-consistency (in order to recover after a system crash) this should be consistent enough for most purposes. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Sat May 17 00:06:51 2008 From: boxbackup at fluffy.co.uk (Louis-Dominique Dubeau) Date: Fri, 16 May 2008 19:06:51 -0400 Subject: [Box Backup] lazy and snapshot together, file formats and files changing often In-Reply-To: References: <1210966247.7008.71.camel@bodhi> <1210974799.7008.110.camel@bodhi> Message-ID: <1210979211.7008.116.camel@bodhi> On Fri, 2008-05-16 at 23:17 +0100, Chris Wilson wrote: > Snapshot mode does not force all files to be backed up. The same rules > about MinimumFileAge, MaxUploadWait, etc. are still observed. You could > argue that this is a bug. I would say it is strange. I would definitely expect bbackupctl sync to force a all files to be backed up. > Not at all, you can and should leave it in lazy mode. Probably all that > happened is that the files became old enough while you went through the > process of changing the config that they were backed up anyway. Actually, I misspoke. I switched to snapshot mode *and* I set parameters like MinimumFileAge, etc. to 0. Thanks, Louis From boxbackup at fluffy.co.uk Sat May 17 13:54:36 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Sat, 17 May 2008 13:54:36 +0100 Subject: [Box Backup] Sugested filesystem? In-Reply-To: <482DB976.6040000@logical-progress.com> References: <482DA21E.3080404@kontrapunkt.com> <482DB976.6040000@logical-progress.com> Message-ID: <5BCA4EEA-E8D7-4752-93D3-81B76F97FBFC@mbrown.co.uk> > > I use jfs on my Debin stores as part of a raid1 array. Resierfs on Raid 5 here seems to work fine. Matt From boxbackup at fluffy.co.uk Sat May 17 14:10:44 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Sat, 17 May 2008 15:10:44 +0200 Subject: [Box Backup] Sugested filesystem? In-Reply-To: <482DA21E.3080404@kontrapunkt.com> References: <482DA21E.3080404@kontrapunkt.com> Message-ID: <1211029844.9122.117.camel@localhost.localdomain> fre, 16 05 2008 kl. 17:02 +0200, skrev Tobias Balle-Petersen: > I'm wondering what filesystem I should choose for the BB store > files. Any suggestions? Reiserfs on Raid-1 works for me.. Bjarne From boxbackup at fluffy.co.uk Mon May 19 12:52:18 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Mon, 19 May 2008 12:52:18 +0100 Subject: [Box Backup] Connection fails during backup to remote server *Resolved* In-Reply-To: References: <081A821F-F1D2-40F4-A8A8-996900BEF1B0@mbrown.co.uk> <057998E1-1C1B-4297-83ED-090E9A69B448@mbrown.co.uk> Message-ID: Hi Chris, > If you could test RC3 for me when it comes out (hopefully in a few > days) > that would be fantastic, and a much better use of your time :-) Is there an RC3 to test ? if so I have a couple of test sites I can roll it out to. Also what was the SSL Keep Alive TIme option I need to enable ? I am seeing this issue with another server we backup, I am going to roll out the patched code for the retry fix in snapshot mode. Regards Matt From boxbackup at fluffy.co.uk Mon May 19 13:07:44 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Mon, 19 May 2008 13:07:44 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <1210856484.9122.103.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <1210856484.9122.103.camel@localhost.localdomain> Message-ID: <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> Hi, > There are several scenarios that need to be taken into consideration: Just as a question, I have a number of clients who use Ubuntu Dapper 6.06 LTS who were unaffected as I understand by the SSL issue. All the .raw keys were generated on these hosts - so I guess these are ok, however the CA was an affected server - so this being used to create the serverCA.pem and sign all the clients csr.pem files was an issue. What I have done in this instance is update the SSL on the affected server, recreate the CA and re-sign all existing clients csr.pem files and re-issue a new serverCA.pem .. Is this enough ? I am taking the assumption as no-one but me has access to the store, and I create and admin all certs, setup the clients etc - that the risk of someone accessing the data is minimal ? In addition as the .raw keys were never an issue, the only weakness in this instance was the actual store allowing weak SSL connections ? I am really hoping to avoid re-sync'ing all the data, as it would take days ... Regards Matt From boxbackup at fluffy.co.uk Mon May 19 13:12:34 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Mon, 19 May 2008 13:12:34 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <1210856484.9122.103.camel@localhost.localdomain> <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> Message-ID: <62785B8A-71C1-495A-B290-B9227EDA3074@mbrown.co.uk> Hi, > Hi, > >> There are several scenarios that need to be taken into consideration: > > Just as a question, I have a number of clients who use Ubuntu Dapper > 6.06 LTS who were unaffected as I understand by the SSL issue. > > All the .raw keys were generated on these hosts - so I guess these > are ok, however the CA was an affected server - so this being used > to create the serverCA.pem and sign all the clients csr.pem files > was an issue. > > What I have done in this instance is update the SSL on the affected > server, recreate the CA and re-sign all existing clients csr.pem > files and re-issue a new serverCA.pem .. > > Is this enough ? > > I am taking the assumption as no-one but me has access to the store, > and I create and admin all certs, setup the clients etc - that the > risk of someone accessing the data is minimal ? > > In addition as the .raw keys were never an issue, the only weakness > in this instance was the actual store allowing weak SSL connections ? > > I am really hoping to avoid re-sync'ing all the data, as it would > take days ... > > Regards > > Matt P.S I should also add the store was running an affected Distro ... From boxbackup at fluffy.co.uk Mon May 19 13:47:14 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Mon, 19 May 2008 14:47:14 +0200 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <1210856484.9122.103.camel@localhost.localdomain> <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> Message-ID: <1211201234.8938.6.camel@localhost.localdomain> man, 19 05 2008 kl. 13:07 +0100, skrev Matt Brown: > What I have done in this instance is update the SSL on the affected > server, recreate the CA and re-sign all existing clients csr.pem > files > and re-issue a new serverCA.pem .. You've done exactly what was needed. As your .raw keys were generated on a secure system, your data are safe. You should consider the transfer itself compromised, (which is no big deal since the transferred data were encrypted). Bjarne From boxbackup at fluffy.co.uk Mon May 19 13:49:33 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Mon, 19 May 2008 14:49:33 +0200 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <62785B8A-71C1-495A-B290-B9227EDA3074@mbrown.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <1210856484.9122.103.camel@localhost.localdomain> <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> <62785B8A-71C1-495A-B290-B9227EDA3074@mbrown.co.uk> Message-ID: <1211201373.8938.9.camel@localhost.localdomain> man, 19 05 2008 kl. 13:12 +0100, skrev Matt Brown: > P.S I should also add the store was running an affected Distro ... No matter, as long as that distor did not generate the keys used. The store does not do any crypto on the data - only on the certificates for SSL transmission. Bjarne From boxbackup at fluffy.co.uk Mon May 19 14:23:57 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Mon, 19 May 2008 14:23:57 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <1211201373.8938.9.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <1210856484.9122.103.camel@localhost.localdomain> <29FEE333-B475-45C6-9AED-520AD2BC011C@mbrown.co.uk> <62785B8A-71C1-495A-B290-B9227EDA3074@mbrown.co.uk> <1211201373.8938.9.camel@localhost.localdomain> Message-ID: Hi, Bjarne, > No matter, as long as that distor did not generate the keys used. The > store does not do any crypto on the data - only on the certificates > for > SSL transmission. Thanks for feedback, i'm so glad all the keys were generated on a non- affected system :-) Matt From boxbackup at fluffy.co.uk Mon May 19 14:36:33 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Mon, 19 May 2008 09:36:33 -0400 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> Message-ID: <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> Thanks for your timely instructions, Ben. They helped a lot. This issue really hurt me this week. This week, it's Debian, next week, who knows. Presuming that this won't be the last time we need to update keys, and maybe it's good security policy anyway, does anyone out there know of a way to slow down brute force attacks on our Box Backup servers? I'm thinking along the lines of using tricks like port knocking, like, put a KnockSequence=12345,13456,14567,2201 option in bbackupd.conf) or csf/lfd (http://www.configserver.com/cp/csf.html) which blocks source IPs after a number of failed logins, or limits logins per hour, like, put LoginFailuresLimitPerClientPerHour=10 LoginFailuresBlockClientDuration=6H (M=minutes, H=hours, D=days) MaxLoginsPerClientPerHour=2 options in bbstored.conf). If these ideas aren't totally silly for some reason, I can put them on the Features Request page--let me know. Or is there something I can do on my Ubuntu/Debian OS firewall? Can Box Backup be plugged into something like csf/lfd? Thanks, Pete From boxbackup at fluffy.co.uk Mon May 19 15:02:39 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Mon, 19 May 2008 15:02:39 +0100 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> Message-ID: <4C6F2CBB-7B0A-42CB-B7AD-C2030DB6508D@mbrown.co.uk> Hi Pete, > This week, it's Debian, next week, who knows. Presuming that this > won't be the last time we need to update keys, and maybe it's good > security policy anyway, does anyone out there know of a way to slow > down brute force attacks on our Box Backup servers? I tend to use Fail2Ban to block brute force and failed logins - mainly for SSH (but can be customised for other applications). I have now implemented a FireWall policy that only allows connections from our clients IP addresses. HTH Matt From boxbackup at fluffy.co.uk Mon May 19 15:40:14 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Mon, 19 May 2008 16:40:14 +0200 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> Message-ID: <1211208014.8938.16.camel@localhost.localdomain> man, 19 05 2008 kl. 09:36 -0400, skrev Peter Jalajas, GigaLock Backup Services: > Or is there something I can do on my Ubuntu/Debian OS firewall? Can > Box Backup be plugged into something like csf/lfd? We use ShoreWall, which can be set to deny routes - and I mean deny the route, not just drop the packets - to attacking hosts, DenyHosts, which can be configured to watch ports besides SSH for brute-force attempts and put the offending hosts in /etc/hosts.deny and finally PortSentry, which will deny routes to port-scanning hosts. While this may sound like overkill, I am a firm believer in defence in depth. Bjarne From boxbackup at fluffy.co.uk Mon May 19 21:21:42 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Mon, 19 May 2008 16:21:42 -0400 Subject: [Box Backup] Advice for users of Debian-derived systems affected by the OpenSSL fiasco -- assume compromise of all data In-Reply-To: <1211208014.8938.16.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> Message-ID: <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> Thanks for the input everyone. fail2ban looks like the lightest solution. Shorewall, denyhosts, and portsentry are good well-known tools, although I think more complicated. Snort came up on my research this morning as another well-known, but complicated, option. Another interesting simple-looking trick that might work, derived from: http://kevin.vanzonneveld.net/techblog/article/block_brute_force_attacks_with_iptables/ would be some variant of: sudo iptables -A INPUT -i eth0 -p tcp --dport 2201 -m state --state NEW -m recent --set --name BoxBackup sudo iptables -A INPUT -i eth0 -p tcp --dport 2201 -m state --state NEW -m recent --update --seconds 60 --hitcount 8 --rttl --name BoxBackup -j DROP I've read this morning that ssh is quite resistant to brute-force attacks because of the delay between password prompts (not sure if that applies to publickey logins). Does Box Backup have a similar feature to insert a delay between login attempts from a client? If so, is it configurable? And if not, should I put it on the Feature Request page (something like, DelayBetweenClientLoginAttempts=60S (S=Seconds, M=Minutes, H=Hours)? Although I'd hate to have to add and manage a whole 'nother application on my systems simply to prevent brute force attacks against Box Backup, they're probably worth it in the long-run. But it'd be simpler, and I guess thus safer and cleaner, to have this little feature within Box Backup itself. Thanks again, everyone, Pete From boxbackup at fluffy.co.uk Tue May 20 10:42:09 2008 From: boxbackup at fluffy.co.uk (Reinhard Tartler) Date: Tue, 20 May 2008 11:42:09 +0200 Subject: [Box Backup] Re: Bug#479145: boxbackup-client: Complaints about =?utf-8?Q?=C2=ABfile?= of unknown =?utf-8?Q?type=C2=BB?= (socket) In-Reply-To: <87wsmbbz6i.fsf@xoog.err.no> (Tollef Fog Heen's message of "Sat, 03 May 2008 09:54:13 +0200") References: <87wsmbbz6i.fsf@xoog.err.no> Message-ID: <87r6bx71n2.fsf@faui44a.informatik.uni-erlangen.de> (please keep the CC line intact) Tollef Fog Heen writes: > Hi, > > it seems like boxbackup-client doesn't know how to ignore sockets and > so I get lines in daemon log like: > > May 2 23:07:44 xoog Box Backup (bbackupd)[3512]: WARNING: Ignoring file = of unknown type: /var/run/postgresql/.s.PGSQL.5432 > May 2 23:07:44 xoog Box Backup (bbackupd)[3512]: WARNING: Ignoring file = of unknown type: /var/run/postgresql/.s.PGSQL.5433 > > This causes boxbackup-client to send email about =C2=ABfiles not being > backed up=C2=BB, which is quite annoying. Indeed. I had a closer look at the source code, and I think the relevant part of the code is in BackupDaemon::NotifyUnsupportedFileType: virtual void NotifyUnsupportedFileType( const BackupClientDirectoryRecord* pDirRecord, const std::string& rLocalPath) { BOX_WARNING("Ignoring file of unknown type: " << rLocalPath= ); } Now, as a workaround, you could of course add an explicit ignore in your boxbackup configuration to work around this issue. I'm not sure if this is the 'correct' way to solve this problem. TBH, I'd rather expect boxbackup to 'more or less' silently ignore this. I'm therefore inclined to change that line of code from BOX_WARNING to BOX_INFO. What do the boxbackup developers think of this? --=20 Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 From boxbackup at fluffy.co.uk Wed May 21 12:09:59 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Wed, 21 May 2008 13:09:59 +0200 Subject: [Box Backup] Feature Request In-Reply-To: <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> Message-ID: <1211368199.8938.55.camel@localhost.localdomain> While the workaround to transfer data via an USB harddrive, which is described in the wiki is nice, this workaround actually breaks the security of Box Backup in relation to clients. Box Backup is a system where the server is not trusted and should never be concerned with the clients' keys. I propose a switch in bbackupd to back up the client's files directly to a connected USB-device in encrypted form in order to perform the first transfer to the server. In this way, the server and its operators will never see client-data in unencrypted form, nor have access to the xxx-FileEncKeys.raw. What say you, developers? Bjarne From boxbackup at fluffy.co.uk Wed May 21 15:43:24 2008 From: boxbackup at fluffy.co.uk (Magnus Homann) Date: Wed, 21 May 2008 16:43:24 +0200 Subject: [Box Backup] Feature Request In-Reply-To: <1211368199.8938.55.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> <1211368199.8938.55.camel@localhost.localdomain> Message-ID: <4834350C.6010602@homann.se> Bjarne Carlsen wrote: > While the workaround to transfer data via an USB harddrive, which is > described in the wiki is nice, this workaround actually breaks the > security of Box Backup in relation to clients. > > Box Backup is a system where the server is not trusted and should never > be concerned with the clients' keys. Agree on that. If I understadn the Wiki you take the USB disk with user files in clear text and carry it over to the server? Wouldn't it be easier to setup a temporary server at (or close to) the client machine, and then copy the storage driectory that bbstored created (through the use of USB disk or whatnot)? Magnus From boxbackup at fluffy.co.uk Wed May 21 15:52:02 2008 From: boxbackup at fluffy.co.uk (Bjarne Carlsen) Date: Wed, 21 May 2008 16:52:02 +0200 Subject: [Box Backup] Feature Request In-Reply-To: <4834350C.6010602@homann.se> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> <1211368199.8938.55.camel@localhost.localdomain> <4834350C.6010602@homann.se> Message-ID: <1211381522.8938.63.camel@localhost.localdomain> ons, 21 05 2008 kl. 16:43 +0200, skrev Magnus Homann: > Agree on that. If I understadn the Wiki you take the USB disk with user > files in clear text and carry it over to the server? Yes. > > Wouldn't it be easier to setup a temporary server at (or close to) the > client machine, and then copy the storage driectory that bbstored > created (through the use of USB disk or whatnot)? > As backup vendor, I would very much like to have the backup daemon itself do the job. Setting up servers "all over the place" disagrees with my natural laziness ;) Bjarne From boxbackup at fluffy.co.uk Thu May 22 08:48:40 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 22 May 2008 17:48:40 +1000 Subject: [Box Backup] Feature Request In-Reply-To: <1211381522.8938.63.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> <1211368199.8938.55.camel@localhost.localdomain> <4834350C.6010602@homann.se> <1211381522.8938.63.camel@localhost.localdomain> Message-ID: > As backup vendor, I would very much like to have the backup daemon > itself do the job. Setting up servers "all over the place" disagrees > with my natural laziness ;) Which is exactly why I wrote up the wiki how I did :) The scenario I often partake in is that a new client has more data than I'm prepared to send over the wire, so I take my hdd, copy their files, then seed their backup on my own server. Sure, I guess I could take a laptop running bbstored on it, but then it's something extra to carry. Timothy. From boxbackup at fluffy.co.uk Thu May 22 10:28:30 2008 From: boxbackup at fluffy.co.uk (dave bamford) Date: Thu, 22 May 2008 10:28:30 +0100 Subject: [Box Backup] Feature Request In-Reply-To: <1211368199.8938.55.camel@localhost.localdomain> References: <2AA32699-E84F-4055-AD1E-2A52542B1A5E@fluffy.co.uk> <74d01c7a0805190636t28c7297dy5152ba24dc8f6b9c@mail.gmail.com> <1211208014.8938.16.camel@localhost.localdomain> <74d01c7a0805191321u7a588d47q597d61030fb6f3f9@mail.gmail.com> <1211368199.8938.55.camel@localhost.localdomain> Message-ID: <48353CBE.6080104@logical-progress.com> Bjarne Carlsen wrote: >While the workaround to transfer data via an USB harddrive, which is >described in the wiki is nice, this workaround actually breaks the >security of Box Backup in relation to clients. > >Box Backup is a system where the server is not trusted and should never >be concerned with the clients' keys. > >I propose a switch in bbackupd to back up the client's files directly to >a connected USB-device in encrypted form in order to perform the first >transfer to the server. In this way, the server and its operators will >never see client-data in unencrypted form, nor have access to the >xxx-FileEncKeys.raw. > > > Great idea, some clients take a week or so to upload the initial backup. So once the "seed" backup is done the USB drive is physically taken to the server and the encrypted files are transferred as is, then backups continue in the normal way. I have done this sucessfully with other backup software Dave Bamford. From boxbackup at fluffy.co.uk Tue May 27 17:19:42 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Tue, 27 May 2008 17:19:42 +0100 Subject: [Box Backup] Odd Box Issue Message-ID: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> Hi, I have a number of sites which I use Box with to backup to a central store - however one of them has stopped working :( What appears to happen is the connection is opened, and the files start to sync - when Box finds a changed file to upload, it hangs .... I have run a check on the store for errors: /usr/local/bin/bbstoreaccounts check 7 fix NOTICE: Will fix errors encountered during checking. Checking store account ID 0x00000007... Phase 1, check objects... Phase 2, check directories... Phase 3, check root... Phase 4, fix unattached objects... Phase 5, fix unrecovered inconsistencies... Phase 6, regenerate store info... NOTICE: Finished checking store account ID 0x00000007: no errors found This appears to be fine, and running bbackup -VTD shows a lot of verbose output and show the time for the file eta as 0s after a while a timeout occurs advising: caught exception: Connection Protocol_Timeout (Probably a network issue between client and server.) (7/41) All clients are using the same version and all work except this site which stopped a few days ago .... Any pointers where I can go from here ? Ta Matt Brown From boxbackup at fluffy.co.uk Tue May 27 17:38:58 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Tue, 27 May 2008 17:38:58 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> Message-ID: <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> > > This appears to be fine, and running bbackup -VTD shows a lot of > verbose output and show the time for the file eta as 0s after a > while a timeout occurs advising: > > caught exception: Connection Protocol_Timeout (Probably a network > issue between client and server.) (7/41) > > All clients are using the same version and all work except this site > which stopped a few days ago .... > > Any pointers where I can go from here ? Here is the output from bbackupd -VTD 17:31:04 Receive Success(0x1e55f) 17:31:04 /etc/clamav/clamd.conf: will not upload (no reason to upload, mod time is 1204898784000000 versus sync period 0 to 1242664240014206) 17:31:04 Synchronised file: /etc/clamav/clamd.conf 17:31:04 /etc/clamav/clamav-base.init: will not upload (no reason to upload, mod time is 1168075219000000 versus sync period 0 to 1242664240014206) 17:31:04 Synchronised file: /etc/clamav/clamav-base.init 17:31:04 /etc/clamav/freshclam.conf: will not upload (no reason to upload, mod time is 1211196585000000 versus sync period 0 to 1242664240014206) 17:31:04 Synchronised file: /etc/clamav/freshclam.conf 17:31:04 Scanning directory: /etc/clamav/onupdateexecute.d 17:31:04 Send ListDirectory(0x1e563,0xffff,0xc,true) 17:31:04 Receive Success(0x1e563) 17:31:04 Scanning directory: /etc/clamav/onerrorexecute.d 17:31:04 Send ListDirectory(0x1e564,0xffff,0xc,true) 17:31:04 Receive Success(0x1e564) 17:31:04 Scanning directory: /etc/clamav/virusevent.d 17:31:04 Send ListDirectory(0x56bb7,0xffff,0xc,true) 17:31:04 Receive Success(0x56bb7) 17:31:04 Scanning directory: /etc/box 17:31:04 Send ListDirectory(0x1e565,0xffff,0xc,true) 17:31:04 Receive Success(0x1e565) 17:31:04 /etc/box/bbackupd.conf: will upload (modified since last sync) 17:31:04 Uploading complete file: /etc/box/bbackupd.conf 17:31:04 Send StoreFile(0x1e565,0x44e38cea68fc0,0x8dbf71bc59981ca, 0,bbackupd.conf) 17:31:04 Read 4096 bytes at 4096, 1515 remain, eta 0s 17:31:04 Read 1515 bytes at 5611, 0 remain, eta 0s This is where everything grinds to a halt, however it has over time got past this file and moved on - but as I only changed this file today this is where it now stops..... All I did to bbackupd.conf was add KeepAliveTime = 60 and comment out the backup state file - in case either of these were causing the issue ... On the server side here is what I see ... May 27 17:28:57 pegasus Box Backup (bbstored)[11289]: WARNING: Message from child process 11300: Incoming connection from 79.123.17.250 port 57982 May 27 17:28:57 pegasus Box Backup (bbstored)[11300]: Client certificate CN: BACKUP-7 May 27 17:28:57 pegasus Box Backup (bbstored)[11300]: Send block allocation size is 1024 May 27 17:28:57 pegasus Box Backup (bbstored)[11300]: NOTICE: Login from Client ID 0x00000007 Read/Write May 27 17:32:39 pegasus Box Backup (bbstored)[11258]: WARNING: Exception thrown: ConnectionException(Conn_Protocol_Timeout) at Protocol.cpp(229) May 27 17:32:41 pegasus Box Backup (bbstored)[11258]: ERROR: Error in child process, terminating connection: Connection Protocol_Timeout (Probably a network issue between client and server.) (7/41) :((( Matt From boxbackup at fluffy.co.uk Tue May 27 17:45:49 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Tue, 27 May 2008 16:45:49 +0000 (WET) Subject: [Box Backup] Odd Box Issue In-Reply-To: <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> Message-ID: Hi Matt, On Tue, 27 May 2008, Matt Brown wrote: > 17:31:04 Uploading complete file: /etc/box/bbackupd.conf > 17:31:04 Send > StoreFile(0x1e565,0x44e38cea68fc0,0x8dbf71bc59981ca,0,bbackupd.conf) > 17:31:04 Read 4096 bytes at 4096, 1515 remain, eta 0s > 17:31:04 Read 1515 bytes at 5611, 0 remain, eta 0s > > This is where everything grinds to a halt, however it has over time got past > this file and moved on - but as I only changed this file today this is where > it now stops..... Could you strace the bbackupd client while it's "ground to a halt" and see what it's doing? The server seems to think that it should be expecting a command from the client at this point, but the client obviously has other ideas. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue May 27 18:17:40 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Tue, 27 May 2008 18:17:40 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> Message-ID: <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> Hi Chris, >> This is where everything grinds to a halt, however it has over time >> got past this file and moved on - but as I only changed this file >> today this is where it now stops..... > > Could you strace the bbackupd client while it's "ground to a halt" > and see what it's doing? The server seems to think that it should be > expecting a command from the client at this point, but the client > obviously has other ideas. Not being familiar with strace *blush* .. I hope this is what you need ........ 8960 read(5, 0x8108738, 5) = -1 EAGAIN (Resource temporarily unavailable) 8960 gettimeofday({1211907986, 342308}, NULL) = 0 8960 gettimeofday({1211907986, 342337}, NULL) = 0 8960 poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 900000) = 1 8960 read(5, "\27\3\1\0 ", 5) = 5 8960 read(5, "\305\275\315\216\350(\230\236 \262\333\316\273\355o \351"..., 32) = 32 8960 read(5, "\27\3\1\0 ", 5) = 5 8960 read(5, "5n\375\316\342\373i\251\0169\220\340\33h \351\316\214\367"..., 32) = 32 8960 read(5, 0x8108738, 5) = -1 EAGAIN (Resource temporarily unavailable) 8960 gettimeofday({1211907986, 394440}, NULL) = 0 8960 gettimeofday({1211907986, 394467}, NULL) = 0 8960 poll([{fd=5, events=POLLIN, revents=POLLIN}], 1, 900000) = 1 8960 read(5, "\27\3\1\0 ", 5) = 5 8960 read(5, "wF\212GH\377\305\n\210\241\177\2155- \354\225\276\371\30"..., 32) = 32 8960 read(5, "\27\3\1\0 ", 5) = 5 8960 read(5, "<\332I\271&\220\367d\213;\200s6~~\360y \211\26\274\331z"..., 32) = 32 8960 time([1211907986]) = 1211907986 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 send(4, "<182>May 27 18:06:26 Box Backup "..., 76, MSG_NOSIGNAL) = 76 8960 read(5, "\27\3\1\0 ", 5) = 5 8960 read(5, "\317\3203)\211lL\343i\352D\267(\35\331\216t\253%\211+ \314"..., 32) = 32 8960 read(5, "\27\3\1\0\240", 5) = 5 8960 read(5, "\216V\271a]\356\321Icb\3720b\311K\225\313_\t\323 U \262"..., 160) = 160 8960 lstat64("/etc/box", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 8960 llistxattr("/etc/box", 0x81bce08, 1000) = 0 8960 lstat64("/etc/box/bbackupd.conf", {st_mode=S_IFREG|0644, st_size=5611, ...}) = 0 8960 llistxattr("/etc/box/bbackupd.conf", 0x81bce08, 1000) = 0 8960 lstat64("/etc/box/bbackupd.conf", {st_mode=S_IFREG|0644, st_size=5611, ...}) = 0 8960 llistxattr("/etc/box/bbackupd.conf", 0x81bdf80, 1000) = 0 8960 open("/etc/box/bbackupd.conf", O_RDONLY|O_LARGEFILE) = 6 8960 fstat64(6, {st_mode=S_IFREG|0644, st_size=5611, ...}) = 0 8960 _llseek(6, 0, [0], SEEK_CUR) = 0 8960 gettimeofday({1211907986, 482938}, NULL) = 0 8960 write(5, "\27\3\1\0 \377;\220\211m\245`\333\256 at Y|\303c \252\36\216"..., 122) = 122 8960 time([1211907986]) = 1211907986 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 send(4, "<182>May 27 18:06:26 Box Backup "..., 80, MSG_NOSIGNAL) = 80 8960 write(5, "\27\3\1\0 \16>\360`|5\251\33i\244\"\254W\224\r \235\263"..., 74) = 74 8960 brk(0x81f0000) = 0x81f0000 8960 read(6, "\nStoreHostname = pegasus.3aitsup"..., 4096) = 4096 8960 gettimeofday({1211907986, 483542}, NULL) = 0 8960 brk(0x8220000) = 0x8220000 8960 brk(0x8210000) = 0x8210000 8960 brk(0x8200000) = 0x8200000 8960 brk(0x81f0000) = 0x81f0000 8960 read(6, "MP3 file in particular.\n# \n# In "..., 1515) = 1515 8960 gettimeofday({1211907986, 484214}, NULL) = 0 8960 brk(0x8220000) = 0x8220000 8960 brk(0x8210000) = 0x8210000 8960 brk(0x8200000) = 0x8200000 8960 brk(0x81f0000) = 0x81f0000 8960 write(5, "\27\3\1\0 \232\372,m at ER\270=H:\2263\263\37\252&{\v \353"..., 2762) = 2762 8960 write(5, "\27\3\1\0 \334\254\233\371^\177J\306\356\304\255R \323\371"..., 154) = 154 8960 write(5, "\27\3\1\0 \25\312\30\273W\276I\376[.\222\24\37\303t\r \375"..., 74) = 74 8960 write(5, "\27\3\1\0 +\246\21w\235WZ[K\204cY\37\26y \313\30\354\30"..., 74) = 74 8960 brk(0x81e0000) = 0x81e0000 8960 read(5, 0x8108738, 5) = -1 EAGAIN (Resource temporarily unavailable) 8960 gettimeofday({1211907986, 485176}, NULL) = 0 8960 gettimeofday({1211907986, 485204}, NULL) = 0 8960 poll( If not, please let me know what I need to capture and if you need the whole file.. Ta Matt From boxbackup at fluffy.co.uk Tue May 27 18:40:20 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Tue, 27 May 2008 18:40:20 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> Message-ID: Hi Chris, > > If not, please let me know what I need to capture and if you need > the whole file.. After leaving the strace on and waiting for the connection error message a few more lines appeared in the log ... 8960 write(5, "\27\3\1\0 \232\372,m at ER\270=H:\2263\263\37\252&{\v \353"..., 2762) = 2762 8960 write(5, "\27\3\1\0 \334\254\233\371^\177J\306\356\304\255R \323\371"..., 154) = 154 8960 write(5, "\27\3\1\0 \25\312\30\273W\276I\376[.\222\24\37\303t\r \375"..., 74) = 74 8960 write(5, "\27\3\1\0 +\246\21w\235WZ[K\204cY\37\26y \313\30\354\30"..., 74) = 74 8960 brk(0x81e0000) = 0x81e0000 8960 read(5, 0x8108738, 5) = -1 EAGAIN (Resource temporarily unavailable) 8960 gettimeofday({1211907986, 485176}, NULL) = 0 8960 gettimeofday({1211907986, 485204}, NULL) = 0 8960 poll([{fd=5, events=POLLIN}], 1, 900000) = -1 EINTR (Interrupted system call) 8960 --- SIGALRM (Alarm clock) @ 0 (0) --- 8960 sigreturn() = ? (mask now []) 8960 gettimeofday({1211908023, 47980}, NULL) = 0 8960 poll([{fd=5, events=POLLIN}], 1, 863438) = 0 8960 time([1211908886]) = 1211908886 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1323, ...}) = 0 8960 send(4, "<180>May 27 18:21:26 Box Backup "..., 140, MSG_NOSIGNAL) = 140 8960 close(6) = 0 and continues its retry to backup again .... Matt From boxbackup at fluffy.co.uk Tue May 27 19:16:14 2008 From: boxbackup at fluffy.co.uk (Brent Saner) Date: Tue, 27 May 2008 14:16:14 -0400 Subject: [Box Backup] box client for embedded devices Message-ID: <483C4FEE.8070705@acetechgroup.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 I was wondering if there's a sort of lean Box Backup client for embedded application OS's; i.e. a type of NASBox/FreeNAS plugin... - -- I wish ZORK had co-op mode. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFIPE/u8u2Zh4MtlQoRAzOhAJ9USwiRgjrpPbMxI//6MYK2KCIU5gCgqZ6z R/JjSPRM3be44/AUhloAMCE= =mQYE -----END PGP SIGNATURE----- From boxbackup at fluffy.co.uk Wed May 28 09:53:42 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 28 May 2008 08:53:42 +0000 (WET) Subject: [Box Backup] box client for embedded devices In-Reply-To: <483C4FEE.8070705@acetechgroup.com> References: <483C4FEE.8070705@acetechgroup.com> Message-ID: Hi Brent, On Tue, 27 May 2008, Brent Saner wrote: > I was wondering if there's a sort of lean Box Backup client for embedded > application OS's; > > i.e. a type of NASBox/FreeNAS plugin... Doesn't bbackupd work? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 28 10:01:49 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 28 May 2008 09:01:49 +0000 (WET) Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> Message-ID: Hi Matt, On Tue, 27 May 2008, Matt Brown wrote: > 8960 send(4, "<180>May 27 18:21:26 Box Backup "..., 140, MSG_NOSIGNAL) = 140 What appeared in your Box logs at 18:21:26? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 28 10:15:23 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 28 May 2008 09:15:23 +0000 (WET) Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> Message-ID: Hi Matt, It looks like a protocol issue where both sides are waiting for a message from the other (deadlock). In order to debug this, please could you enable ExtendedLogging (or ExtendedLogFile) on both sides, and send me the last 50 or so lines of protocol messages on both sides, and the last few log messages, up to the point where the connection times out and the server reports Connection Protocol_Timeout. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 28 12:54:49 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Wed, 28 May 2008 13:54:49 +0200 Subject: [Box Backup] notifyadmin in 0.11rc2 to chatty Message-ID: <483D4809.9090909@trexler.at> Hi, I just installed 0.11rc2 and was wondering why the event backup-start and backup-finish where not handled separately in notifyadmin Per default notifyadmin doesn't know these two states and sends an mail "The backup daemon on ... reported an unknown error. / FILES MAY NOT BE BAACKED UP" on every start and end of backup. That's not what one would expect on a normal backup start and stop... I recommend to add these two lines in notifyadmin in line 5 to shut up the notifier ;-) if [ "$1" = "backup-start" -o "$1" = "backup-finish" ]; then exit 0 fi best regards Wolfgang From boxbackup at fluffy.co.uk Wed May 28 13:01:25 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 28 May 2008 12:01:25 +0000 (WET) Subject: [Box Backup] notifyadmin in 0.11rc2 to chatty In-Reply-To: <483D4809.9090909@trexler.at> References: <483D4809.9090909@trexler.at> Message-ID: Hi Wolfgang, On Wed, 28 May 2008, Wolfgang Trexler wrote: > I just installed 0.11rc2 and was wondering why the event > > backup-start and backup-finish > > where not handled separately in notifyadmin Did you upgrade or install fresh? Where did your notify script come from? The NotifySysadmin.sh created by bbackupd-conf in new versions does handle these events. If you upgraded, did you read the Upgrading to 0.11 page on the Wiki? if not, could you suggest a place to put a link to this where you would have found it, to help ensure that others do find it in future? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 28 14:52:43 2008 From: boxbackup at fluffy.co.uk (Wolfgang Trexler) Date: Wed, 28 May 2008 15:52:43 +0200 Subject: [Box Backup] notifyadmin in 0.11rc2 to chatty In-Reply-To: References: <483D4809.9090909@trexler.at> Message-ID: <483D63AB.80202@trexler.at> Chris Wilson schrieb: > Hi Wolfgang, > > On Wed, 28 May 2008, Wolfgang Trexler wrote: > >> I just installed 0.11rc2 and was wondering why the event >> >> backup-start and backup-finish >> >> where not handled separately in notifyadmin > > Did you upgrade or install fresh? Hi Chris, I upgraded the Debian packages from 0.10 to 0.11rc2 using the etch backport packages on www.backports.org BUT: As I was forced due to the Debian open-ssl disaster, I did an aptitude "remove" first, moved the old /etc/boxbackup/ directory away and did a new "aptitude install" of the boxbackup (client/server) packages. > > Where did your notify script come from? Package from www.backports.org which generated "notifyadmin" I checked my bbackupd-conf script and it would create the new NotifySysadmin.sh script! I test-installed the Debian backports package on another machine, and it showed that the "old" version of the notifyadmin script is generated during the package setup. This is what I get for "aptitude install boxbackup-client" Entpacke boxbackup-client (aus .../boxbackup-client_0.11~rc2-3~bpo40+1_i386.deb) ... Richte boxbackup-client ein (0.11~rc2-3~bpo40+1) ... Creating config file /etc/boxbackup/bbackupd.conf with new version Creating config file /etc/boxbackup/bbackupd/notifyadmin with new version The output (and notifyadmin) is generated from the Debian postinst script (in control.tar.gz) that runs after the installation I'll forward this to the Debian package maintainer Reinhard T. in case he's not checking the list. br Wolfgang From boxbackup at fluffy.co.uk Wed May 28 15:15:01 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 28 May 2008 15:15:01 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> Message-ID: <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> Hi Chris, > It looks like a protocol issue where both sides are waiting for a > message from the other (deadlock). > > In order to debug this, please could you enable ExtendedLogging (or > ExtendedLogFile) on both sides, and send me the last 50 or so lines > of protocol messages on both sides, and the last few log messages, > up to the point where the connection times out and the server > reports Connection Protocol_Timeout. It would appear after multiple attempts overnight last night (looking at the logs) finally today it has backed up ! - as this is a school they are away for half term so no new files :) It has managed to get over the file that was causing the deadlock and completed. However I think another account is showing similar issues as I dont appear to have a report from that server .... I will check it and get back to you. Matt From boxbackup at fluffy.co.uk Wed May 28 15:35:53 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 28 May 2008 15:35:53 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> Message-ID: <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> Hi Chris, > > It would appear after multiple attempts overnight last night > (looking at the logs) finally today it has backed up ! - as this is > a school they are away for half term so no new files :) It has > managed to get over the file that was causing the deadlock and > completed. > > However I think another account is showing similar issues as I dont > appear to have a report from that server .... I will check it and > get back to you. Something I have noticed - I am unable to stop a backup once in progress in snapshot mode with the following... /usr/local/bin/bbackupctl terminate All I get back is ... bbackupctl terminate NOTICE: Using configuration file /etc/box/bbackupd.conf Is this expected behaviour ? Matt From boxbackup at fluffy.co.uk Wed May 28 15:38:55 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 28 May 2008 14:38:55 +0000 (WET) Subject: [Box Backup] Odd Box Issue In-Reply-To: <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> Message-ID: Hi Matt, On Wed, 28 May 2008, Matt Brown wrote: >> However I think another account is showing similar issues as I dont >> appear to have a report from that server .... I will check it and get >> back to you. > > Something I have noticed - I am unable to stop a backup once in progress > in snapshot mode with the following... > > /usr/local/bin/bbackupctl terminate > > All I get back is ... > > bbackupctl terminate > NOTICE: Using configuration file /etc/box/bbackupd.conf > > Is this expected behaviour ? Unfortunately so. There was some consensus when I proposed implementing it that it was a bad idea to poll the command socket all the time. However now that we have efficient timers it shouldn't be too hard to implement regular polling. I might need you to build the latest Box from source on client and server to help debug this problem, would that be possible for you? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed May 28 15:49:26 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 28 May 2008 15:49:26 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> Message-ID: Hi Chris, > I might need you to build the latest Box from source on client and > server to help debug this problem, would that be possible for you? Sure, happy to. At the moment, should the backup go over its time window(s) - cron issues a rather nasty kill to halt any overrunning processes, which I am sure is going to bite me soon ! Regards Matt From boxbackup at fluffy.co.uk Wed May 28 17:00:30 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Wed, 28 May 2008 17:00:30 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> Message-ID: <94526E79-5ADA-432A-A24A-6CF3AFAF2AB8@mbrown.co.uk> Hi Chris, >> I might need you to build the latest Box from source on client and >> server to help debug this problem, would that be possible for you? > > Sure, happy to. At the moment, should the backup go over its time > window(s) - cron issues a rather nasty kill to halt any overrunning > processes, which I am sure is going to bite me soon ! I have checked out r2184 and building :) Matt From boxbackup at fluffy.co.uk Thu May 29 08:51:49 2008 From: boxbackup at fluffy.co.uk (Johann Glaser) Date: Thu, 29 May 2008 09:51:49 +0200 Subject: [Box Backup] notifyadmin in 0.11rc2 to chatty In-Reply-To: <483D63AB.80202@trexler.at> References: <483D4809.9090909@trexler.at> <483D63AB.80202@trexler.at> Message-ID: <1212047509.17219.5.camel@glaser> Hi! Am Mittwoch, den 28.05.2008, 15:52 +0200 schrieb Wolfgang Trexler: > Chris Wilson schrieb: > > Hi Wolfgang, > > > > On Wed, 28 May 2008, Wolfgang Trexler wrote: > > > >> I just installed 0.11rc2 and was wondering why the event > >> > >> backup-start and backup-finish > >> > >> where not handled separately in notifyadmin > > > > Did you upgrade or install fresh? We experience the same problem where we upgraded from 0.10 to 0.11. The usual Debian way is to supply the config files with the .deb package, but marked as config files. Therefore the installer will only overwrite old config files, if the user has not changed them. If there are changes the use is prompted whether he wants the new file or keep his. Probably we should report this issue to the Debian package maintainer. Bye Hansi From boxbackup at fluffy.co.uk Thu May 29 13:12:42 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 29 May 2008 22:12:42 +1000 Subject: [Box Backup] Windows regex help Message-ID: Hello guys, I've read the wiki and the mailling lists, but I can't seem to find the regex I need. I'm using the windows client. All I want to do is backup a bunch of BU08xxxx folders. So I have a hierarchy like this: D:\ D:\somefolder D:\somefiles.zip D:\somefiles2.zip D:\BU080521 D:\BU080522 D:\BU080523 etc. At the moment I have the following regex: Path = D:\ # So we don't copy gb's and gb's of old data ExcludeFilesRegex = D:\\.*$ ExcludeDirsRegex = D:\\.*$ AlwaysIncludeDirsRegex = D:\BU08\.* But this doesn't backup anything!! In the logs, box connects, then disconnects straight away, so I know it's a problem with my regex. Can someone please help? Thanks, Timothy. From boxbackup at fluffy.co.uk Thu May 29 13:45:53 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Thu, 29 May 2008 08:45:53 -0400 Subject: [Box Backup] Windows regex help In-Reply-To: References: Message-ID: <74d01c7a0805290545t6dfdf500kdc2ba7a3ee59df0b@mail.gmail.com> Hi Timothy, I'm not sure, but I think this line: > AlwaysIncludeDirsRegex = D:\BU08\.* should be more like: > AlwaysIncludeDirsRegex = D:\\BU08.* I hope that helps. Good luck. Pete On Thu, May 29, 2008 at 8:12 AM, Timothy Wilson wrote: > Hello guys, > > I've read the wiki and the mailling lists, but I can't seem to find > the regex I need. > > I'm using the windows client. All I want to do is backup a bunch of > BU08xxxx folders. So I have a hierarchy like this: > > D:\ > D:\somefolder > D:\somefiles.zip > D:\somefiles2.zip > D:\BU080521 > D:\BU080522 > D:\BU080523 > > etc. > > At the moment I have the following regex: > > Path = D:\ > # So we don't copy gb's and gb's of old data > ExcludeFilesRegex = D:\\.*$ > ExcludeDirsRegex = D:\\.*$ > AlwaysIncludeDirsRegex = D:\BU08\.* > > > But this doesn't backup anything!! In the logs, box connects, then > disconnects straight away, so I know it's a problem with my regex. Can > someone please help? > > Thanks, > Timothy. From boxbackup at fluffy.co.uk Thu May 29 13:52:27 2008 From: boxbackup at fluffy.co.uk (Timothy Wilson) Date: Thu, 29 May 2008 22:52:27 +1000 Subject: [Box Backup] Windows regex help In-Reply-To: <74d01c7a0805290545t6dfdf500kdc2ba7a3ee59df0b@mail.gmail.com> References: <74d01c7a0805290545t6dfdf500kdc2ba7a3ee59df0b@mail.gmail.com> Message-ID: Hello Peter, Thanks for the prompt response. It's now connecting and transferring, I'll check on it in the morning to see if it's worked. Thanks very much!! Timothy. 2008/5/29 Peter Jalajas, GigaLock Backup Services : > Hi Timothy, > > I'm not sure, but I think this line: >> AlwaysIncludeDirsRegex = D:\BU08\.* > should be more like: >> AlwaysIncludeDirsRegex = D:\\BU08.* > > I hope that helps. > > Good luck. > > Pete > > On Thu, May 29, 2008 at 8:12 AM, Timothy Wilson > wrote: >> Hello guys, >> >> I've read the wiki and the mailling lists, but I can't seem to find >> the regex I need. >> >> I'm using the windows client. All I want to do is backup a bunch of >> BU08xxxx folders. So I have a hierarchy like this: >> >> D:\ >> D:\somefolder >> D:\somefiles.zip >> D:\somefiles2.zip >> D:\BU080521 >> D:\BU080522 >> D:\BU080523 >> >> etc. >> >> At the moment I have the following regex: >> >> Path = D:\ >> # So we don't copy gb's and gb's of old data >> ExcludeFilesRegex = D:\\.*$ >> ExcludeDirsRegex = D:\\.*$ >> AlwaysIncludeDirsRegex = D:\BU08\.* >> >> >> But this doesn't backup anything!! In the logs, box connects, then >> disconnects straight away, so I know it's a problem with my regex. Can >> someone please help? >> >> Thanks, >> Timothy. > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu May 29 19:46:44 2008 From: boxbackup at fluffy.co.uk (Alexandre Mallais) Date: Thu, 29 May 2008 14:46:44 -0400 Subject: [Box Backup] Simple Question about housekeeping Message-ID: <483EFA14.2010907@avior.ca> with Box Backup how does the "housekeeping" decides what to delete when the soft limit is reached ? Is it the oldest of the deleted file first, then the oldest of the "Old files"... etc ? or is it completly random (the first deleted or old files it finds, it deletes it) ? Can I trust boxbackup for having some rotation in my files, and just delete the oldest of them all ? Sorry for the bad english. From boxbackup at fluffy.co.uk Fri May 30 11:29:35 2008 From: boxbackup at fluffy.co.uk (Roy) Date: Fri, 30 May 2008 12:29:35 +0200 Subject: [Box Backup] Simple question about usage info Message-ID: <483FD70F.3060704@hostingbrothers.nl> Hi all, I use boxbackup to backup several Windows XP, Windows 2003 and Linux machines. This all works very well. But now I have a question about the usage numbers of bbstoreaccounts info . It gives the following numbers in one case: Blocks used: 7759098 (15154.49Mb) Blocks used by old files: 37443 (73.13Mb) Blocks used by deleted files: 2557218 (4994.57Mb) Is blocks used the total number of blocks or must I add the block used by old files and deleted files to it? Thus total number of blocks used = 7759098 or is it: 7759098 + 37443 + 2557218 Thanks in advance, Roy From boxbackup at fluffy.co.uk Fri May 30 13:08:03 2008 From: boxbackup at fluffy.co.uk (Greg Bolshaw) Date: Fri, 30 May 2008 13:08:03 +0100 Subject: [Box Backup] Asymmetric vs symmetric encryption Message-ID: Dear list It seems that a limitation exists in Box Backup in that the whole system relies on the safe storage of the certificate file. If this is lost, the backups are rendered useless. Would it be possible to offer symmetric encryption as an alternative? This would work in a similar way to GPG's -c option: > -c, --symmetric [file] > Encrypt with a symmetric cipher using a passphrase. The default > symmetric cipher used is CAST5, but may be chosen with the --cipher- > algo option. This option may be combined with --sign (for a signed > and symmetrically encrypted message), --encrypt (for a message that > may be decrypted via a secret key or a passphrase), or --sign and -- > encrypt together (for a signed message that may be decrypted via a > secret key or a passphrase). A secret passphrase would be used to encrypt/decrypt the backup data rather than a certificate. This would just leave the issue of how to authenticate bbackupd against bbstored. Understandably, it would be less secure to protect data using a passphrase (which could then be subject to a brute force attack, etc.), but in the balance of security and practicality, would this be a reasonable compromise? (pun unavoidable!) Any thoughts on this? Thanks Greg From boxbackup at fluffy.co.uk Fri May 30 13:09:45 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 30 May 2008 13:09:45 +0100 (BST) Subject: [Box Backup] Simple question about usage info In-Reply-To: <483FD70F.3060704@hostingbrothers.nl> References: <483FD70F.3060704@hostingbrothers.nl> Message-ID: Hi Roy, On Fri, 30 May 2008, Roy wrote: > Blocks used: 7759098 (15154.49Mb) > Blocks used by old files: 37443 (73.13Mb) > Blocks used by deleted files: 2557218 (4994.57Mb) > > Is blocks used the total number of blocks or must I add the block used > by old files and deleted files to it? I think that old + deleted + directories + files (not shown) = total blocks used. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 30 13:12:20 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Fri, 30 May 2008 08:12:20 -0400 Subject: [Box Backup] Simple question about usage info In-Reply-To: <483FD70F.3060704@hostingbrothers.nl> References: <483FD70F.3060704@hostingbrothers.nl> Message-ID: <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> Hi Roy, Yes, "Blocks used" is the total, I believe. Note that I think the current info output is a bit unclear; files that are both old AND deleted are counted twice, once each in the "old" line and the "deleted" line (so, no, you cannot, and I guess must not, add "old" and "deleted", else you'll be double counting). I've added a Feature Request in this regard (search for "counted twice" on http://www.boxbackup.org/trac/wiki/FeatureRequests ). Pete On Fri, May 30, 2008 at 6:29 AM, Roy wrote: > Hi all, > > I use boxbackup to backup several Windows XP, Windows 2003 and Linux > machines. This all works very well. But now I have a question about the > usage numbers of bbstoreaccounts info . It gives the following > numbers in one case: > Blocks used: 7759098 (15154.49Mb) > Blocks used by old files: 37443 (73.13Mb) > Blocks used by deleted files: 2557218 (4994.57Mb) > > Is blocks used the total number of blocks or must I add the block used by > old files and deleted files to it? > Thus total number of blocks used = 7759098 or is it: 7759098 + 37443 + > 2557218 > > Thanks in advance, > > Roy > From boxbackup at fluffy.co.uk Fri May 30 13:30:11 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Fri, 30 May 2008 08:30:11 -0400 Subject: [Box Backup] Asymmetric vs symmetric encryption In-Reply-To: References: Message-ID: <74d01c7a0805300530qea7e754n53ea5d219f19dfb1@mail.gmail.com> Hi Greg, I don't have the skills to really comment on your idea (although my gut says that I can't see Box Backup changing to a password model as you propose), but you raise an important issue. What we _could_ do is provide a fairly long list of suggested methods for securely making available the FileEncKeys.raw file (I think that's the one you mean; as I phrase it to my customers, "Without that file, you have no backup; with that file, someone else owns your backup."). Suggestions might include, in no particular order yet: a. Burning it to CD and putting it into a safe. b. Strongly encrypting _copies_ of it with GPG (either symmetric or PKI) and putting those strongly-encrypted copies in several fairly secure areas, including off-site. c. Secret-splitting it among trusted colleagues, including off-site. Something like that. I personally have a pile of them encrypted into 7z files (I used 7z for convenient secure distribution to my Windows-based clients; I will like change that back to GPG for boring reasons we can discuss separately). . Thanks, Pete On Fri, May 30, 2008 at 8:08 AM, Greg Bolshaw wrote: > Dear list > > It seems that a limitation exists in Box Backup in that the whole system > relies on the safe storage of the certificate file. If this is lost, the > backups are rendered useless. > > Would it be possible to offer symmetric encryption as an alternative? This > would work in a similar way to GPG's -c option: > >> -c, --symmetric [file] >> Encrypt with a symmetric cipher using a passphrase. The default symmetric >> cipher used is CAST5, but may be chosen with the --cipher-algo option. This >> option may be combined with --sign (for a signed and symmetrically encrypted >> message), --encrypt (for a message that may be decrypted via a secret key or >> a passphrase), or --sign and --encrypt together (for a signed message that >> may be decrypted via a secret key or a passphrase). > > > A secret passphrase would be used to encrypt/decrypt the backup data rather > than a certificate. This would just leave the issue of how to authenticate > bbackupd against bbstored. > > Understandably, it would be less secure to protect data using a passphrase > (which could then be subject to a brute force attack, etc.), but in the > balance of security and practicality, would this be a reasonable compromise? > (pun unavoidable!) > > Any thoughts on this? > > Thanks > Greg From boxbackup at fluffy.co.uk Fri May 30 13:34:41 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 30 May 2008 13:34:41 +0100 (BST) Subject: [Box Backup] Asymmetric vs symmetric encryption In-Reply-To: References: Message-ID: Hi Greg, On Fri, 30 May 2008, Greg Bolshaw wrote: > It seems that a limitation exists in Box Backup in that the whole system > relies on the safe storage of the certificate file. If this is lost, the > backups are rendered useless. Not certificates but the keys file. > Would it be possible to offer symmetric encryption as an alternative? > This would work in a similar way to GPG's -c option: It is symmetric encryption. You're talking about deriving the keys from a shorter password that's entered manually. It is possible but I don't know any standard tools to do it easily. You could do something like this to generate your keys file from a password: dd if=/dev/zero bs=1k count=1 \ | openssl enc -bf -K `echo mypassword | md5` -iv 1234 \ > bbackupd.keys This generates the same keys file every time, in my quick tests. > A secret passphrase would be used to encrypt/decrypt the backup data > rather than a certificate. This would just leave the issue of how to > authenticate bbackupd against bbstored. That's a completely separate problem which involves a certificate, but one that is easily replaceable. > Understandably, it would be less secure to protect data using a > passphrase (which could then be subject to a brute force attack, etc.), > but in the balance of security and practicality, would this be a > reasonable compromise? (pun unavoidable!) It's much less secure, but if you want it you can do it. What's wrong with backing up the raw keys onto a CD, encrypted by GPG with a passphrase, and keeping that CD somewhere safe? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 30 13:43:42 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 30 May 2008 13:43:42 +0100 (BST) Subject: [Box Backup] Simple question about usage info In-Reply-To: <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> References: <483FD70F.3060704@hostingbrothers.nl> <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> Message-ID: Hi Pete, On Fri, 30 May 2008, Peter Jalajas, GigaLock Backup Services wrote: > Note that I think the current info output is a bit unclear; files that > are both old AND deleted are counted twice, once each in the "old" line > and the "deleted" line (so, no, you cannot, and I guess must not, add > "old" and "deleted", else you'll be double counting). I don't think that should ever happen (a file being old and deleted at the same time), can you show us some examples and do you have any idea how it came about? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 30 14:00:23 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Fri, 30 May 2008 14:00:23 +0100 Subject: [Box Backup] Simple question about usage info In-Reply-To: References: <483FD70F.3060704@hostingbrothers.nl> <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> Message-ID: <05A11C17-350D-457C-8A56-1D53EC4D9BAA@mbrown.co.uk> Hi Chris, > I don't think that should ever happen (a file being old and deleted > at the > same time), can you show us some examples and do you have any idea > how it > came about? Not exactly the same issue, but I did have a scare yesterday. A client called me to advise a file has become corrupted on the server, and asked me to restore from a backup. I logged in and ran bbackupquery, got to the directory I needed and issued a ls -odt so I could see all the old files - to my horror, there was none for this particular file :( Luckily as a safety precaution I had an Rsync on the same server running locally to another HDD which carries around 7 days of changes - so I was able to get the file back. Q is ... how much overhead should you allow for Old and Deleted files - as in this case the file was not deleted. What is the priority Deleted files ? Can you reserve a certain amount of space for Old Files I am reviewing all the sites today to see what Old and Deleted files usage is - just to cover myself :) The current stats (after increasing the space) are now... Used 30825.9Mb 54% ********************* Old files 41.4Mb 0% Deleted files 7212.4Mb 12% ***** Directories 36.7Mb 0% Soft limit 46080.0Mb 81% ******************************** Hard limit 56320.0Mb 100% **************************************** Before the limit increase: Used 30982.9Mb 91% ************************************ Old files 42.0Mb 0% Deleted files 7380.6Mb 21% ******** Directories 36.8Mb 0% Soft limit 30720.0Mb 90% ************************************ Hard limit 33792.0Mb 100% **************************************** Regards Matt From boxbackup at fluffy.co.uk Fri May 30 14:00:53 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Fri, 30 May 2008 09:00:53 -0400 Subject: [Box Backup] Simple question about usage info In-Reply-To: References: <483FD70F.3060704@hostingbrothers.nl> <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> Message-ID: <74d01c7a0805300600q48980ea5o2ff4cbe978e8b9c0@mail.gmail.com> Hi Chris, I don't have ready access at the moment, but see the Feature Request http://www.boxbackup.org/trac/wiki/FeatureRequests (search for "obviously counted twice") for my classic true-life case. I believe I recall seeing lots of files with both "o" and "X" flags in that client (and others, too, I think). Thanks, Pete On Fri, May 30, 2008 at 8:43 AM, Chris Wilson wrote: > Hi Pete, > > On Fri, 30 May 2008, Peter Jalajas, GigaLock Backup Services wrote: > >> Note that I think the current info output is a bit unclear; files that >> are both old AND deleted are counted twice, once each in the "old" line >> and the "deleted" line (so, no, you cannot, and I guess must not, add >> "old" and "deleted", else you'll be double counting). > > I don't think that should ever happen (a file being old and deleted at the > same time), can you show us some examples and do you have any idea how it > came about? > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Fri May 30 14:02:04 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Fri, 30 May 2008 14:02:04 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> Message-ID: <17ACC0BF-1065-4553-90F2-407FD829887F@mbrown.co.uk> Hi Chris, > I might need you to build the latest Box from source on client and > server to help debug this problem, would that be possible for you? I have r2184 built and installed at Client and Server - is this rev sufficient for the tests ? Regards Matt From boxbackup at fluffy.co.uk Fri May 30 14:19:31 2008 From: boxbackup at fluffy.co.uk (Peter Jalajas, GigaLock Backup Services) Date: Fri, 30 May 2008 09:19:31 -0400 Subject: [Box Backup] Simple question about usage info In-Reply-To: <05A11C17-350D-457C-8A56-1D53EC4D9BAA@mbrown.co.uk> References: <483FD70F.3060704@hostingbrothers.nl> <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> <05A11C17-350D-457C-8A56-1D53EC4D9BAA@mbrown.co.uk> Message-ID: <74d01c7a0805300619w4bd8a9a4uda1cf07b0cf359e9@mail.gmail.com> Hi Matt, Note that Box can't really yet do what you are requesting (see the Feature Requests page for things like snapshot and restore back to a point in time). You just need to make sure your "Soft limit" is well above your "Used". Note also that the "100%" of the chart below is a tad misleading. You were at 100% of your day-to-day quota (91%/90%=100+% of your soft limit), so you were essentially burning up old and deleted files to make room for your new files, even though the report says you were at (only) "91%" Used. (For me, I make the freeboard space between the soft limit and the hard limit a couple of times larger than the largest file I expect to upload (which is usually a 2GB Outlook.pst file), so I usually make the hard limit 5-10GB higher than the day-to-day quota soft-limit that I want to give a client.) I hope that helps. Thanks, Pete On Fri, May 30, 2008 at 9:00 AM, Matt Brown wrote: > Hi Chris, > > Q is ... how much overhead should you allow for Old and Deleted files - as > in this case the file was not deleted. What is the priority Deleted files ? > Can you reserve a certain amount of space for Old Files > > Before the limit increase: > > Used 30982.9Mb 91% ************************************ > Old files 42.0Mb 0% > Deleted files 7380.6Mb 21% ******** > Directories 36.8Mb 0% > Soft limit 30720.0Mb 90% ************************************ > Hard limit 33792.0Mb 100% **************************************** From boxbackup at fluffy.co.uk Fri May 30 15:30:03 2008 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 30 May 2008 15:30:03 +0100 (BST) Subject: [Box Backup] Odd Box Issue In-Reply-To: <17ACC0BF-1065-4553-90F2-407FD829887F@mbrown.co.uk> References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> <17ACC0BF-1065-4553-90F2-407FD829887F@mbrown.co.uk> Message-ID: Hi Matt, On Fri, 30 May 2008, Matt Brown wrote: > >I might need you to build the latest Box from source on client and > >server to help debug this problem, would that be possible for you? > > I have r2184 built and installed at Client and Server - is this rev > sufficient for the tests ? Sorry, I found a bug with the new backtrace logic, it might crash on an exception. Please could you update to the latest SVN and test with that? Also please make sure that you see TRACE level messages in the logs on both sides (run with -V and configure syslog appropriately). Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \ _/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at fluffy.co.uk Fri May 30 18:27:09 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Fri, 30 May 2008 18:27:09 +0100 Subject: [Box Backup] Simple question about usage info In-Reply-To: <74d01c7a0805300619w4bd8a9a4uda1cf07b0cf359e9@mail.gmail.com> References: <483FD70F.3060704@hostingbrothers.nl> <74d01c7a0805300512ie69fe42u48f01045e7715b52@mail.gmail.com> <05A11C17-350D-457C-8A56-1D53EC4D9BAA@mbrown.co.uk> <74d01c7a0805300619w4bd8a9a4uda1cf07b0cf359e9@mail.gmail.com> Message-ID: <963015CD-A640-4328-8524-29C7C36CFF01@mbrown.co.uk> Hi Pete, > (For me, I make the freeboard space between the soft limit and the > hard limit a couple of times larger than the largest file I expect to > upload (which is usually a 2GB Outlook.pst file), so I usually make > the hard limit 5-10GB higher than the day-to-day quota soft-limit that > I want to give a client.) > > I hope that helps. Sure does, thanks - I do tend to give most clients about 10% or so extra space, but will increase it a bit to be safe :) Thanks. Matt From boxbackup at fluffy.co.uk Fri May 30 18:28:04 2008 From: boxbackup at fluffy.co.uk (Matt Brown) Date: Fri, 30 May 2008 18:28:04 +0100 Subject: [Box Backup] Odd Box Issue In-Reply-To: References: <911A3A9B-E1C4-47BB-AB67-B36872A0C94E@mbrown.co.uk> <5726F15D-2304-42F4-A0C6-40C7D51D3A40@mbrown.co.uk> <2B81D9B8-CBDF-4FD6-B537-53BA594D248C@mbrown.co.uk> <79AFA73F-7463-4287-ACB5-89E176072C98@mbrown.co.uk> <9E240568-5194-4D9F-AA99-2D8C9A103198@mbrown.co.uk> <17ACC0BF-1065-4553-90F2-407FD829887F@mbrown.co.uk> Message-ID: <67BA1099-A5B9-4709-9CBD-B47FB3D12A3E@mbrown.co.uk> Hi Chris, > Sorry, I found a bug with the new backtrace logic, it might crash on > an > exception. Please could you update to the latest SVN and test with > that? > Also please make sure that you see TRACE level messages in the logs on > both sides (run with -V and configure syslog appropriately). No probs, thats why I thought I should check. Checked out trunk r2190 Matt From boxbackup at boxbackup.org Thu May 8 23:06:23 2008 From: boxbackup at boxbackup.org (Greg Bolshaw) Date: Thu, 8 May 2008 23:06:23 +0100 Subject: [Box Backup] Deleting from backup In-Reply-To: <4822FA78.2010106@homann.se> References: <4822FA78.2010106@homann.se> Message-ID: --Apple-Mail-9-48949770 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit On 8 May 2008, at 14:04, Magnus Homann wrote: > That's what I practiced during my attempts to recover a corrupt > storage. Good luck! Thank you, that worked perfectly. > A 'delete ' would be good in bbackupquery. I agree, it would be great to have such a function. Ideally this would also allow for the removal of directories including subdirectories and files. Would this be difficult to implement? Is it possible to automatically delete files from the backup after a number of days, rather than keeping them until the storage limit is reached? I would like to create a backup scheme where the last 30 days of files are always available, rather than the last n GB. Thanks Greg --Apple-Mail-9-48949770 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable
On 8 May 2008, at = 14:04, Magnus Homann wrote:
That's what I practiced during my = attempts to recover a corrupt storage. Good = luck!

Thank you, that worked = perfectly.

A 'delete <object-id>' would be = good in bbackupquery.

I agree, it = would be great to have such a function. Ideally this would also allow = for the removal of directories including subdirectories and files. Would = this be difficult to implement?

Is it possible = to automatically delete files from the backup after a number of days, = rather than keeping them until the storage limit is reached? I would = like to create a backup scheme where the last 30 days of files are = always available, rather than the last n = GB.

Thanks
Greg
= --Apple-Mail-9-48949770-- From boxbackup at boxbackup.org Thu May 15 13:41:21 2008 From: boxbackup at boxbackup.org (Peter Jalajas, GigaLock Backup Services) Date: Thu, 15 May 2008 08:41:21 -0400 Subject: [Box Backup] New openssl packages fix predictable random number generator In-Reply-To: <1210851587.9122.70.camel@localhost.localdomain> References: <482AF50C.8020400@trexler.at> <1210789333.7416.0.camel@shodan> <1210799773.9122.6.camel@localhost.localdomain> <1210800584.7416.7.camel@shodan> <1210807357.9122.40.camel@localhost.localdomain> <1210840064.18401.76.camel@helios> <1210851587.9122.70.camel@localhost.localdomain> Message-ID: <74d01c7a0805150541g64e9b3a2ocd156ed407f4f73a@mail.gmail.com> ------=_Part_6685_26636333.1210855281671 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Thu, May 15, 2008 at 7:39 AM, Bjarne Carlsen wrote: > > The way I see it, we are defending against the usual suspects here: The > malicious server operator, anyone from the outside able to gain > privileges on the server and finally outright theft of the whole server. > The problem is, though, that a, if not _the_, fundamental design goal of BoxBackup is the ability to survive such attacks on the server, I think. >From very near the top of http://www.boxbackup.org/trac/ : "The server is trusted only to make files available when they are required - all data is encrypted and can be decoded only by the original client. This makes it ideal for backing up over an untrusted network (such as the Internet), or where the server is in an uncontrolled environment." ------=_Part_6685_26636333.1210855281671 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Thu, May 15, 2008 at 7:39 AM, Bjarne Carlsen <bjarne at mail2net.dk> wrote:
<snip>
The way I see it, we are defending against the usual suspects here: The
malicious server operator, anyone from the outside able to gain
privileges on the server and finally outright theft of the whole server.
<snip>

The problem is, though, that a, if not _the_, fundamental design goal of BoxBackup is the ability to survive such attacks on the server, I think.

From very near the top of http://www.boxbackup.org/trac/ :   "The server is trusted only to make files available when they are required - all data is encrypted and can be decoded only by the original client. This makes it ideal for backing up over an untrusted network (such as the Internet), or where the server is in an uncontrolled environment."
------=_Part_6685_26636333.1210855281671--