From boxbackup at fluffy.co.uk Wed Nov 1 09:01:25 2006 From: boxbackup at fluffy.co.uk (=?ISO-8859-1?Q?Simon_Lundstr=F6m?=) Date: Wed, 1 Nov 2006 10:01:25 +0100 Subject: [Box Backup] Backup the backup In-Reply-To: References: <20061031213809.15669.qmail@web60622.mail.yahoo.com> Message-ID: --Apple-Mail-15--619107505 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed On Oct 31, 2006, at 10:43 PM, Chris Wilson wrote: Thanks for all the suggestions and discussion, I haven't been able to =20= answer the mail until today. > I'm talking about a way to recover from a fatal and prolonged crash =20= > of the local backup store (bbstored) by switching to an offsite =20 > mirror of the store, which is readable and writable to the client, =20 > until the local store can be repaired (at which point I assume that =20= > you would change back to using the local one). Not really what I want, but kind of. I want an way to have two (or more) backup server that are in sync =20 (sort of). The client can use the local backup store located on the =20 company LAN when they are on the company LAN. Also I want the ability =20= to choose another secondary backup store that is available from the =20 Internet when they are travelling or otherwise not on the company LAN. See http://waza.se/crap/natdiagram.jpg for a "better" description, =20 I'm sorry that the names are in Swedish. "Fj=E4rr" =3D Remote or =20 secondary. "Lokal" =3D Local, "Klient" =3D Client. And yes, Internets is a pun at GWB. see http://en.wikipedia.org/wiki/=20 Internets_%28colloquialism%29 Thanks, - Simon --Apple-Mail-15--619107505 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGMTCCAuow ggJToAMCAQICEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDgyNTE2NDc1OVoXDTA3MDgyNTE2NDc1 OVowSjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEnMCUGCSqGSIb3DQEJARYYc2lt b25sdW5kc3Ryb21AZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArrb+ /12bXTb5oixk4K2QhcJtsN6o3bu3p/fJK9xTP4gtF7NpnOzTrnRbmGGQ2HJEgV3bVuGOIaR1qqyk toR5RzxM3QXkepq0KUxdpCJgkDu/GlG/z7It1Wh5y+J5suPhVyWjr3n2RYJ5J+nssP/pw1EiLV3g OH7imu9uUPCziK+J2glMXwFNE/pA+xNIPAflNFpdaexIYAbKnm/N119usWLZlzCoIuj9yTSSV0wG /SpPN5UJtA0O0zxD6GMLFfm0zFc9e0sAvynaVIfVDmSEvY3FzrfGg/fZiCWFHRcc/Q23Olw/OxDw ioSo1DfQU2e2XzBiX5bX9ApUbQ+RFZRGHQIDAQABozUwMzAjBgNVHREEHDAagRhzaW1vbmx1bmRz dHJvbUBnbWFpbC5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOBgQAmV6ABcDdunTJ8 5SkJl6oEa0lIh8+dfXseipg4bMDCRiJahR5RvpGWFnNbDN66NxYFvIasAy9ss4VOsZ2Eq3FQkQNA 4UL4rqjJDF/XKGfRYEZpF55ZztDm54vUMLpXtkjZUCCzSQp+1X04d3Vjz/1r2PtX3MxnBnZdfQuk LvkJFjCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJBgNVBAYTAlpBMRUwEwYD VQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgGA1UEChMRVGhhd3RlIENv bnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNV BAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwt ZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJ BgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQD EyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDX AmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l 0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMG A1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVt YWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/TCG4 +DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amcOY6MIE9l X5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggMQMIIDDAIBATB2MGIx CzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYD VQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQQStBdrLuqCK7MX8wXDle 5TAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0wNjExMDEwOTAxMjZaMCMGCSqGSIb3DQEJBDEWBBRXNcEK/amn5TffTtwsydbaYQV6rjCBhQYJ KwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC EEErQXay7qgiuzF/MFw5XuUwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMG A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEBBQAE ggEAhzfIjKfVDTGLCZr2j5M0q2PJj9B1aZnhwMFsIvE367EFx3n5SWVfA8cFUJaLf/hgtHU4Iygz wsZSvL4qEbhgJjkWFyrcKEMtAfztcv3LzXA7FcFEulLX3XqAE58VCCeeKdFEHIZLaYaf/NFpgoUy iFfhUmdhX3G+QnPLK1q06x+PsZ39qcuJcxZcqVeyiGinBsqolgaM8FJwl00qRu+LNbz23pC9H5Mh OPVkK0gXXQJjoMkvXUc2UK6DOGwxoAWXqrVDTSR7kiCZbQZSbGokeB++vahxmqHUkjHhxMaxit0l O6VVRSE/2LkDEE5jySwosDGjww/3A+p82iEtt796tgAAAAAAAA== --Apple-Mail-15--619107505-- From boxbackup at fluffy.co.uk Wed Nov 1 09:13:19 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 1 Nov 2006 09:13:19 +0000 (GMT) Subject: [Box Backup] Backup the backup In-Reply-To: References: <20061031213809.15669.qmail@web60622.mail.yahoo.com> Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---128931150-442338922-1162372399=:27391 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Hi Simon, On Wed, 1 Nov 2006, [ISO-8859-1] Simon Lundstr=F6m wrote: >> I'm talking about a way to recover from a fatal and prolonged crash of = the=20 >> local backup store (bbstored) by switching to an offsite mirror of the= =20 >> store, which is readable and writable to the client, until the local st= ore=20 >> can be repaired (at which point I assume that you would change back to= =20 >> using the local one). > > Not really what I want, but kind of. > > I want an way to have two (or more) backup server that are in sync (sort = of).=20 > The client can use the local backup store located on the company LAN when= =20 > they are on the company LAN. Also I want the ability to choose another=20 > secondary backup store that is available from the Internet when they are= =20 > travelling or otherwise not on the company LAN. This is very difficult. Whenever the client is using the remote server,=20 the local server is more up-to-date than the local one. rsync will not=20 realise this and will happily overwrite the new data with old. If you try= =20 to do an incomplete sync, for example telling rsync not to overwrite newer= =20 files on the secondary server, you will end up with a mess on the remote=20 server. I can't think of a way to make this work reliably with existing tools. The= =20 only possibility is some very complicated two-way sync mechanism or=20 replicating filesystem between the local and remote servers. I would=20 certainly hesitate to do something so experimental as a primary backup=20 solution, and I would test it very heavily. Why not have the client back up to the local server even when it's on the= =20 road? You should have plenty of download bandwidth on those sites (more=20 than the client has upload bandwidth) and it avoids this problem=20 completely. Cheers, Chris. --=20 _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | ---128931150-442338922-1162372399=:27391-- From boxbackup at fluffy.co.uk Wed Nov 1 09:31:02 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 1 Nov 2006 09:31:02 +0000 Subject: [Box Backup] Backup Ninja In-Reply-To: <45473CD5.8060801@versado.net> References: <45473CD5.8060801@versado.net> Message-ID: <874D543F-52A5-4A05-9776-C0F0C1D1B37E@fluffy.co.uk> On 31 Oct 2006, at 12:08, Jamie Neil wrote: > In case there are any other Debian (or derived distro) Box Backup > users > out there, you might want to take a look at a program called Backup > Ninja. > > It's a backup scheduler that lets you configure a number of different > backup jobs in one place. Support for rdiff-backup and duplicity is > built in, but because it allows you to execute arbitary scripts > integrating boxbackup is trivial. > > We have it set up to: > > 1) Dump the debian package list to a text file. > > 2) Dump all MySQL databases to compressed text files. Don't compress them. It'll stop the block based differencing working. Box Backup compresses anyway. Ben From boxbackup at fluffy.co.uk Wed Nov 1 11:51:12 2006 From: boxbackup at fluffy.co.uk (=?ISO-8859-1?Q?Simon_Lundstr=F6m?=) Date: Wed, 1 Nov 2006 12:51:12 +0100 Subject: [Box Backup] Backup the backup In-Reply-To: References: <20061031213809.15669.qmail@web60622.mail.yahoo.com> Message-ID: --Apple-Mail-16--608920649 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed On Nov 1, 2006, at 10:13 AM, Chris Wilson wrote: > This is very difficult. Whenever the client is using the remote > server, the local server is more up-to-date than the local one. > rsync will not realise this and will happily overwrite the new data > with old. If you try to do an incomplete sync, for example telling > rsync not to overwrite newer files on the secondary server, you > will end up with a mess on the remote server. From man rsync: -u, --update update only (don't overwrite newer files) I thought that was going to work, but maybe not? > I can't think of a way to make this work reliably with existing > tools. The only possibility is some very complicated two-way sync > mechanism or replicating filesystem between the local and remote > servers. I would certainly hesitate to do something so experimental > as a primary backup solution, and I would test it very heavily. Maybe unison http://en.wikipedia.org/wiki/Unison_%28file_synchronizer% 29 is the way to go? > Why not have the client back up to the local server even when it's > on the road? You should have plenty of download bandwidth on those > sites (more than the client has upload bandwidth) and it avoids > this problem completely. Because many of these companies are small and don't have a lot of bandwidth since they only surf and check their mail they can't only use remote backup but needs local backup. (5-10 ppl transmitting 2-10GB over 2Mbit can be come quite slow. I will only use lazy mode, i.e. backups will not be automatic. Besides not all clients can have servers or ports opened in their firewall. - Simon --Apple-Mail-16--608920649 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGMTCCAuow ggJToAMCAQICEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDgyNTE2NDc1OVoXDTA3MDgyNTE2NDc1 OVowSjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEnMCUGCSqGSIb3DQEJARYYc2lt b25sdW5kc3Ryb21AZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArrb+ /12bXTb5oixk4K2QhcJtsN6o3bu3p/fJK9xTP4gtF7NpnOzTrnRbmGGQ2HJEgV3bVuGOIaR1qqyk toR5RzxM3QXkepq0KUxdpCJgkDu/GlG/z7It1Wh5y+J5suPhVyWjr3n2RYJ5J+nssP/pw1EiLV3g OH7imu9uUPCziK+J2glMXwFNE/pA+xNIPAflNFpdaexIYAbKnm/N119usWLZlzCoIuj9yTSSV0wG /SpPN5UJtA0O0zxD6GMLFfm0zFc9e0sAvynaVIfVDmSEvY3FzrfGg/fZiCWFHRcc/Q23Olw/OxDw ioSo1DfQU2e2XzBiX5bX9ApUbQ+RFZRGHQIDAQABozUwMzAjBgNVHREEHDAagRhzaW1vbmx1bmRz dHJvbUBnbWFpbC5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOBgQAmV6ABcDdunTJ8 5SkJl6oEa0lIh8+dfXseipg4bMDCRiJahR5RvpGWFnNbDN66NxYFvIasAy9ss4VOsZ2Eq3FQkQNA 4UL4rqjJDF/XKGfRYEZpF55ZztDm54vUMLpXtkjZUCCzSQp+1X04d3Vjz/1r2PtX3MxnBnZdfQuk LvkJFjCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJBgNVBAYTAlpBMRUwEwYD VQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgGA1UEChMRVGhhd3RlIENv bnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNV BAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwt ZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJ BgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQD EyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDX AmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l 0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMG A1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVt YWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/TCG4 +DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amcOY6MIE9l X5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggMQMIIDDAIBATB2MGIx CzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYD VQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQQStBdrLuqCK7MX8wXDle 5TAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0wNjExMDExMTUxMTNaMCMGCSqGSIb3DQEJBDEWBBQBHVclUUTguQmA5CF6c7SxosU87jCBhQYJ KwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC EEErQXay7qgiuzF/MFw5XuUwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMG A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEBBQAE ggEAnk4ENMVPLCafYDIvI7Gc3Mg3BKtRLToozg/9HptzuUmo3+Rxxt6A+E3CSl7hHj346lWU2xY9 S+3zZ7l4FLg8LlXWzACuqgoT42OGDkWCBzVBAlz6zEfEHdysJqD3LKKl11XQhIAl1Se3jiXECCfz 8X/nPJ3QvmrjoDdnxAy4mmexQZ3+N8tQeSTX1oEE5rhusXTsYDS9SFC/evrVVoB8yaWpF3BgUl24 +YF60CyQ08LtDr3KeVy//7iUx2BnkuQqgl1diIQSHuQtbWdH+FoS2RB+maL/XvAaPZy0TnBKmXWH tEM8Lr8rv/vlR1LVGv8QzEJpV3DOhixQ9ubWzimieQAAAAAAAA== --Apple-Mail-16--608920649-- From boxbackup at fluffy.co.uk Wed Nov 1 14:51:04 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 1 Nov 2006 14:51:04 +0000 (GMT) Subject: [Box Backup] Backup the backup In-Reply-To: References: <20061031213809.15669.qmail@web60622.mail.yahoo.com> Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323328-1540948780-1162392664=:25509 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Hi Simon, On Wed, 1 Nov 2006, Simon Lundstr=F6m wrote: > From man rsync: > -u, --update update only (don't overwrite newer files) > > I thought that was going to work, but maybe not? I suspect that you will get a mixture of newer and older files on the=20 server, as soon as the client switches in the middle of an rsync process,= =20 which is guaranteed to happen eventually. I also suspect that you may have problems with synchronisation of=20 directories between the two servers, but I can't put my finger on an exact= =20 failure mode right now. Go ahead and try it, but be extremely careful,=20 that would be my advice. Also, run bbstoreaccounts check on each account=20 regularly on both sides and watch out for errors. > Maybe unison http://en.wikipedia.org/wiki/Unison_%28file_synchronizer%29= =20 > is the way to go? Maybe, but I found it's unsuitable for automatic synchronisation due to=20 its tendency to develop conflicts in practice. For example, if you updated= =20 a file and a directory on one side, and the directory (but not the file)=20 on the other, you would get a conflict on the directory but not on the=20 file. But you can't just upload the file without overriding one version of= =20 the directory, otherwise the result is inconsistent (doesn't agree with=20 either side) and may be invalid. > Because many of these companies are small and don't have a lot of=20 > bandwidth since they only surf and check their mail they can't only use= =20 > remote backup but needs local backup. (5-10 ppl transmitting 2-10GB over= =20 > 2Mbit can be come quite slow. But they're only syncing changes and only when they're on the road, so I=20 don't think it would be so bad. Besides you would probably use exactly the= =20 same amount of bandwidth in the other direction (upload) to update the=20 secondary server, and upload bandwidth is more scarce than download=20 bandwidth. I suppose it depends on whether the clients are more usually on= =20 the road or in the office. > I will only use lazy mode, i.e. backups will not be automatic. Besides=20 > not all clients can have servers or ports opened in their firewall. You can get around the latter restriction by tunneling to your server=20 through the firewall. Cheers, Chris. --=20 _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | --8323328-1540948780-1162392664=:25509-- From boxbackup at fluffy.co.uk Wed Nov 1 15:00:10 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 1 Nov 2006 15:00:10 +0000 Subject: [Box Backup] Backup the backup In-Reply-To: References: <20061031213809.15669.qmail@web60622.mail.yahoo.com> Message-ID: <7CC83F3B-423F-4412-B54D-8760E55BB689@fluffy.co.uk> On 1 Nov 2006, at 11:51, Simon Lundstr=F6m wrote: > > On Nov 1, 2006, at 10:13 AM, Chris Wilson wrote: > >> This is very difficult. Whenever the client is using the remote =20 >> server, the local server is more up-to-date than the local one. =20 >> rsync will not realise this and will happily overwrite the new =20 >> data with old. If you try to do an incomplete sync, for example =20 >> telling rsync not to overwrite newer files on the secondary =20 >> server, you will end up with a mess on the remote server. > > =46rom man rsync: > -u, --update update only (don't overwrite newer files) > > I thought that was going to work, but maybe not? No. That'll mean the rsyncing computer won't send it's directories =20 properly after an update. > >> I can't think of a way to make this work reliably with existing =20 >> tools. The only possibility is some very complicated two-way sync =20 >> mechanism or replicating filesystem between the local and remote =20 >> servers. I would certainly hesitate to do something so =20 >> experimental as a primary backup solution, and I would test it =20 >> very heavily. > > Maybe unison http://en.wikipedia.org/wiki/Unison_%=20 > 28file_synchronizer%29 is the way to go? No. Two way sync will corrupt the store. > >> Why not have the client back up to the local server even when it's =20= >> on the road? You should have plenty of download bandwidth on those =20= >> sites (more than the client has upload bandwidth) and it avoids =20 >> this problem completely. > > Because many of these companies are small and don't have a lot of =20 > bandwidth since they only surf and check their mail they can't only =20= > use remote backup but needs local backup. (5-10 ppl transmitting =20 > 2-10GB over 2Mbit can be come quite slow. I will only use lazy =20 > mode, i.e. backups will not be automatic. Besides not all clients =20 > can have servers or ports opened in their firewall. I'm not convinced you're going to get the current system to do what =20 you want. I think you need servers to co-operate with each other =20 before this setup will be happy. Ben From boxbackup at fluffy.co.uk Wed Nov 1 19:26:35 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Wed, 1 Nov 2006 11:26:35 -0800 (PST) Subject: [Box Backup] Backup the backup In-Reply-To: Message-ID: <20061101192635.92916.qmail@web60623.mail.yahoo.com> Hi Chris, Yeah, I'm just thinking that this silly trick might be a reasonably easy way for the client to have geographic and, a new concept, "corporate" redundancy, where "Corporate redundancy" means having your data backed up to two separate companies, just in case the first one becomes essentially permanently unavailable to the client for whatever reason. For example, someone in the States, if it were important enough to the them, could sign up for box backup with me on the east coast and Per (I owe you an email, Per) on the left coast, do this bbackupd.conf switching, and have yet another layer of protection. I'm not sure, but maybe it could be used to only backup problematic BackupLocations during certain hours of the day. There are probably other uses for this bbackupd.conf-switching trick as well. For the casual reader of this list, don't try this at home--these tricks would take a lot more in-depth serious thought and testing. Pete --- Chris Wilson wrote: > Hi Pete, > > > This is an interesting idea, changing bbackupd.conf periodically to > > make bbackupd do different things at, say, different times of the > day. > > Is that what you guys are talking about? > > I think it would work, provided that you appropriately signal the > bbackupd > process to reread its configuration file afterwards (with bbackupctl > reload or kill -HUP). However, it's not what I'm talking about, and > if it > is what Ben's talking about then we're talking at cross purposes :-) > From boxbackup at fluffy.co.uk Thu Nov 2 04:12:02 2006 From: boxbackup at fluffy.co.uk (W. Chris Shank) Date: Wed, 1 Nov 2006 23:12:02 -0500 (EST) Subject: [Box Backup] Off topic: Request for server advice In-Reply-To: Message-ID: <23842520.201162440722970.JavaMail.root@banshee> I've had great success using standard whitebox systems with Intel boards & chipsets using Ubuntu Dapper Server. My standard config is 2 ethernets (I install a second NIC), 2 SATA drives with SW RAID. If you find a maker who won't charge the Windows tax these boxes will only cost about $450 each. Good luck. ----- Original Message ----- From: Chris Wilson To: Box Backup List Sent: Wednesday, October 25, 2006 11:52:04 AM GMT-0500 Subject: [Box Backup] Off topic: Request for server advice Hi all, I hope you don't mind me asking this here. It's for a good cause :-) My company, Aidworld (www.aidworld.org) is planning to donate four servers to Kenyan universities for use as bandwidth management boxes (traffic shaping, firewall, proxy servers). We need to buy these very quickly as we're leaving for Kenya on the 4th November, and taking them with us. I know that many of you have a lot of experience with servers running open source software, so I'd really appreciate your advice. We don't need a very fast processor, but I think we need about 1 GB of RAM, 2 x 80 GB SATA disks (RAID 1), dual Ethernet. We're planning to run either Linux or FreeBSD on these boxes. I'm currently looking at a few options for the servers: * Dell PowerEdge SC440 (tower case), GBP 481 * Dell PowerEdge SC1425 SATA (rack mount), GBP 626 * HP ProLiant MI310G3 (tower case), GBP 700 (upgraded to spec) * HP ProLiant ML110G3 (tower case), GBP 400 (upgraded to spec) * HP DX5150 Desktop Athlon64 (desktop case), GBP 350 (upgraded to spec) * Any unbranded desktop box, about GBP 400 at spec Does anyone have any thoughts about relative merits of rack mount versus desktop and tower cases, given that these places may not have racks? Is it worth paying GBP 140 for potential rack-mountability? How about hardware support for these boxes in Linux and FreeBSD? Booting with no keyboard connected? Booting from a second hard disk if the first one fails? Working in hot climates with dodgy electricity? Thanks very much for your help and advice. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | _______________________________________________ boxbackup mailing list boxbackup at fluffy.co.uk http://lists.warhead.org.uk/mailman/listinfo/boxbackup -- W. Chris Shank ACE Technology Group, LLC www.myremoteITdept.com (610) 640-4223 -------------------------------- Security Note: To protect against computer viruses, e-mail programs may prevent sending or receiving certain types of file attachments. Check your e-mail security settings to determine how attachments are handled. From boxbackup at fluffy.co.uk Thu Nov 2 10:51:37 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Thu, 02 Nov 2006 11:51:37 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? Message-ID: <4549CDB9.60809@kontrapunkt.com> Hello. I'm running box 0.10 client on a OS X machine and the 0.10 store on a debian machine. I'm, backing up more than 1TB of data. bbackupd will complete if run for the first time since a reload of the .conf file. If the .conf is not reloaded prior to every backup run, it wil fail with the error below. If I use the StoreObjectInfoFile feature and reload the .conf prior to every run, the backup still fails with the error below. The error: Exception caught (Connection Protocol_Timeout (Probably a network issue between client and server.) 7/41), reset state and waiting to retry... The conjobs that avoids the errors: 20 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/bbackupd.conf reload 25 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/bbackupd.conf sync Any ideas? Thanks, Tobias From boxbackup at fluffy.co.uk Thu Nov 2 10:54:45 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 2 Nov 2006 10:54:45 +0000 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <4549CDB9.60809@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> Message-ID: <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> Try using KeepAliveTime in bbackupd.conf, if you're not already doing so. Ben On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: > Hello. > > I'm running box 0.10 client on a OS X machine and the 0.10 store on > a debian machine. > I'm, backing up more than 1TB of data. bbackupd will complete if > run for the first time since a reload of the .conf file. If > the .conf is not reloaded prior to every backup run, it wil fail > with the error below. If I use the StoreObjectInfoFile feature and > reload the .conf prior to every run, the backup still fails with > the error below. > > The error: > Exception caught (Connection Protocol_Timeout (Probably a network > issue > between client and server.) 7/41), reset state and waiting to retry... > > > The conjobs that avoids the errors: > 20 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ > bbackupd.conf reload > 25 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ > bbackupd.conf sync > > > Any ideas? > > Thanks, > Tobias > > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu Nov 2 11:06:00 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Thu, 02 Nov 2006 12:06:00 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> Message-ID: <4549D118.3080704@kontrapunkt.com> Hello. I currently use: KeepAliveTime = 10 What value would you suggest? Tobias Ben Summers wrote: > > Try using KeepAliveTime in bbackupd.conf, if you're not already doing so. > > Ben > > > On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: > >> Hello. >> >> I'm running box 0.10 client on a OS X machine and the 0.10 store on a >> debian machine. >> I'm, backing up more than 1TB of data. bbackupd will complete if run >> for the first time since a reload of the .conf file. If the .conf is >> not reloaded prior to every backup run, it wil fail with the error >> below. If I use the StoreObjectInfoFile feature and reload the .conf >> prior to every run, the backup still fails with the error below. >> >> The error: >> Exception caught (Connection Protocol_Timeout (Probably a network issue >> between client and server.) 7/41), reset state and waiting to retry... >> >> >> The conjobs that avoids the errors: >> 20 10 * * * /usr/local/bin/bbackupctl -c >> /etc/box_minimoni/bbackupd.conf reload >> 25 10 * * * /usr/local/bin/bbackupctl -c >> /etc/box_minimoni/bbackupd.conf sync >> >> >> Any ideas? >> >> Thanks, >> Tobias >> >> >> >> >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 2 11:09:15 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Thu, 02 Nov 2006 12:09:15 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> Message-ID: <4549D1DB.90109@kontrapunkt.com> Hello again. Would changing the KeepAliveTime parameter ,as a fix for my problem, make sense considering the backup run ALWAYS completes on the first run since a reload of the bbackupd.conf file ? The point being that the first run completes with the KeepAliveTime = 10. Tobias Ben Summers wrote: > > Try using KeepAliveTime in bbackupd.conf, if you're not already doing so. > > Ben > > > On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: > >> Hello. >> >> I'm running box 0.10 client on a OS X machine and the 0.10 store on a >> debian machine. >> I'm, backing up more than 1TB of data. bbackupd will complete if run >> for the first time since a reload of the .conf file. If the .conf is >> not reloaded prior to every backup run, it wil fail with the error >> below. If I use the StoreObjectInfoFile feature and reload the .conf >> prior to every run, the backup still fails with the error below. >> >> The error: >> Exception caught (Connection Protocol_Timeout (Probably a network issue >> between client and server.) 7/41), reset state and waiting to retry... >> >> >> The conjobs that avoids the errors: >> 20 10 * * * /usr/local/bin/bbackupctl -c >> /etc/box_minimoni/bbackupd.conf reload >> 25 10 * * * /usr/local/bin/bbackupctl -c >> /etc/box_minimoni/bbackupd.conf sync >> >> >> Any ideas? >> >> Thanks, >> Tobias >> >> >> >> >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 2 11:17:18 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 2 Nov 2006 11:17:18 +0000 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <4549D1DB.90109@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> Message-ID: <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> Please turn on ExtendedLogging = yes on client and server, and we'll see what it's doing at the time of the exception. Ben On 2 Nov 2006, at 11:09, Tobias Balle-Petersen wrote: > Hello again. > > Would changing the KeepAliveTime parameter ,as a fix for my > problem, make sense considering the backup run ALWAYS completes on > the first run since a reload of the bbackupd.conf file ? The point > being that the first run completes with the KeepAliveTime = 10. > > Tobias > > > Ben Summers wrote: >> Try using KeepAliveTime in bbackupd.conf, if you're not already >> doing so. >> Ben >> On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: >>> Hello. >>> >>> I'm running box 0.10 client on a OS X machine and the 0.10 store >>> on a debian machine. >>> I'm, backing up more than 1TB of data. bbackupd will complete if >>> run for the first time since a reload of the .conf file. If >>> the .conf is not reloaded prior to every backup run, it wil fail >>> with the error below. If I use the StoreObjectInfoFile feature >>> and reload the .conf prior to every run, the backup still fails >>> with the error below. >>> >>> The error: >>> Exception caught (Connection Protocol_Timeout (Probably a network >>> issue >>> between client and server.) 7/41), reset state and waiting to >>> retry... >>> >>> >>> The conjobs that avoids the errors: >>> 20 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ >>> bbackupd.conf reload >>> 25 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ >>> bbackupd.conf sync >>> >>> >>> Any ideas? >>> >>> Thanks, >>> Tobias >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu Nov 2 11:24:28 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Thu, 02 Nov 2006 12:24:28 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> Message-ID: <4549D56C.6060602@kontrapunkt.com> Hello. Just to let you know, I have tried to delete the contents of /var/bbackupd I have had this problem for a while. Here is an older log from the client showing the error: Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating EncodingBuffer from 2601 to 4777\n Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating EncodingBuffer from 1068 to 2606\n ................... Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating EncodingBuffer from 1068 to 3244\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating EncodingBuffer from 3244 to 5413\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/recruitment_ms\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/recruitment_tos\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/recruitment2_ms\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/re.kmo\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/recruitment transport fredag\n Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/re.transp.mandag\n Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/Medicon\n Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/clients/MOUNT\n Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/guests/Bergen Bybane\n Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record for /Volumes/raidA/files/guests/Library1\n Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection TLSReadFailed 7/34), reset state and waiting to retry... Aug 24 11:42:09 yoiko bbackupd[11514]: File statistics: total file size uploaded 13410118455, bytes already on server 341406646, encoded size 9996343505 Aug 24 11:42:09 yoiko bbackupd[11514]: TRACE: Wait on command socket, delay = 1024000000\n Aug 24 11:59:13 yoiko bbackupd[11514]: TRACE: Wait on command socket, delay = 1024000000\n Tobias Ben Summers wrote: > > Please turn on ExtendedLogging = yes on client and server, and we'll see > what it's doing at the time of the exception. > > Ben > > > > > On 2 Nov 2006, at 11:09, Tobias Balle-Petersen wrote: > >> Hello again. >> >> Would changing the KeepAliveTime parameter ,as a fix for my problem, >> make sense considering the backup run ALWAYS completes on the first >> run since a reload of the bbackupd.conf file ? The point being that >> the first run completes with the KeepAliveTime = 10. >> >> Tobias >> >> >> Ben Summers wrote: >>> Try using KeepAliveTime in bbackupd.conf, if you're not already doing >>> so. >>> Ben >>> On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: >>>> Hello. >>>> >>>> I'm running box 0.10 client on a OS X machine and the 0.10 store on >>>> a debian machine. >>>> I'm, backing up more than 1TB of data. bbackupd will complete if run >>>> for the first time since a reload of the .conf file. If the .conf is >>>> not reloaded prior to every backup run, it wil fail with the error >>>> below. If I use the StoreObjectInfoFile feature and reload the .conf >>>> prior to every run, the backup still fails with the error below. >>>> >>>> The error: >>>> Exception caught (Connection Protocol_Timeout (Probably a network issue >>>> between client and server.) 7/41), reset state and waiting to retry... >>>> >>>> >>>> The conjobs that avoids the errors: >>>> 20 10 * * * /usr/local/bin/bbackupctl -c >>>> /etc/box_minimoni/bbackupd.conf reload >>>> 25 10 * * * /usr/local/bin/bbackupctl -c >>>> /etc/box_minimoni/bbackupd.conf sync >>>> >>>> >>>> Any ideas? >>>> >>>> Thanks, >>>> Tobias >>>> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 2 13:15:18 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 2 Nov 2006 13:15:18 +0000 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <4549D56C.6060602@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> Message-ID: <16755366-735A-4C21-BD6D-A8CB112B5853@fluffy.co.uk> That's logging from a debug build, not with ExtendedLogging set in the .conf file. I'm afraid that doesn't really tell me much. Ben On 2 Nov 2006, at 11:24, Tobias Balle-Petersen wrote: > Hello. > > Just to let you know, I have tried to delete the contents of /var/ > bbackupd > > > > I have had this problem for a while. > Here is an older log from the client showing the error: > > Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating > EncodingBuffer from 2601 to 4777\n > Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating > EncodingBuffer from 1068 to 2606\n > ................... > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating > EncodingBuffer from 1068 to 3244\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating > EncodingBuffer from 3244 to 5413\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > recruitment_ms\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > recruitment_tos\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > recruitment2_ms\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > re.kmo\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > recruitment transport fredag\n > Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/ > re.transp.mandag\n > Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/Medicon\n > Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/clients/MOUNT\n > Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/guests/Bergen Bybane\n > Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory > record for /Volumes/raidA/files/guests/Library1\n > Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: > ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n > Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: > ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n > Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection > TLSReadFailed 7/34), reset state and waiting to retry... > Aug 24 11:42:09 yoiko bbackupd[11514]: File statistics: total file > size uploaded 13410118455, bytes already on server 341406646, > encoded size 9996343505 > Aug 24 11:42:09 yoiko bbackupd[11514]: TRACE: Wait on command > socket, delay = 1024000000\n > Aug 24 11:59:13 yoiko bbackupd[11514]: TRACE: Wait on command > socket, delay = 1024000000\n > > Tobias > > > Ben Summers wrote: >> Please turn on ExtendedLogging = yes on client and server, and >> we'll see what it's doing at the time of the exception. >> Ben >> On 2 Nov 2006, at 11:09, Tobias Balle-Petersen wrote: >>> Hello again. >>> >>> Would changing the KeepAliveTime parameter ,as a fix for my >>> problem, make sense considering the backup run ALWAYS completes >>> on the first run since a reload of the bbackupd.conf file ? The >>> point being that the first run completes with the KeepAliveTime = >>> 10. >>> >>> Tobias >>> >>> >>> Ben Summers wrote: >>>> Try using KeepAliveTime in bbackupd.conf, if you're not already >>>> doing so. >>>> Ben >>>> On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: >>>>> Hello. >>>>> >>>>> I'm running box 0.10 client on a OS X machine and the 0.10 >>>>> store on a debian machine. >>>>> I'm, backing up more than 1TB of data. bbackupd will complete >>>>> if run for the first time since a reload of the .conf file. If >>>>> the .conf is not reloaded prior to every backup run, it wil >>>>> fail with the error below. If I use the StoreObjectInfoFile >>>>> feature and reload the .conf prior to every run, the backup >>>>> still fails with the error below. >>>>> >>>>> The error: >>>>> Exception caught (Connection Protocol_Timeout (Probably a >>>>> network issue >>>>> between client and server.) 7/41), reset state and waiting to >>>>> retry... >>>>> >>>>> >>>>> The conjobs that avoids the errors: >>>>> 20 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ >>>>> bbackupd.conf reload >>>>> 25 10 * * * /usr/local/bin/bbackupctl -c /etc/box_minimoni/ >>>>> bbackupd.conf sync >>>>> >>>>> >>>>> Any ideas? >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> boxbackup mailing list >>>>> boxbackup at fluffy.co.uk >>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu Nov 2 13:27:58 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Thu, 02 Nov 2006 14:27:58 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <16755366-735A-4C21-BD6D-A8CB112B5853@fluffy.co.uk> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <16755366-735A-4C21-BD6D-A8CB112B5853@fluffy.co.uk> Message-ID: <4549F25E.80903@kontrapunkt.com> Hello. OK. I'll enable ExtendedLogging and report back. Thank you for helping out, Tobias Ben Summers wrote: > > That's logging from a debug build, not with ExtendedLogging set in the > .conf file. I'm afraid that doesn't really tell me much. > > Ben > > > On 2 Nov 2006, at 11:24, Tobias Balle-Petersen wrote: > >> Hello. >> >> Just to let you know, I have tried to delete the contents of >> /var/bbackupd >> >> >> >> I have had this problem for a while. >> Here is an older log from the client showing the error: >> >> Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating >> EncodingBuffer from 2601 to 4777\n >> Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating >> EncodingBuffer from 1068 to 2606\n >> ................... >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating >> EncodingBuffer from 1068 to 3244\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating >> EncodingBuffer from 3244 to 5413\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo >> Nordisk/Recruitment/recruitment_ms\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo >> Nordisk/Recruitment/recruitment_tos\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo >> Nordisk/Recruitment/recruitment2_ms\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/re.kmo\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo Nordisk/Recruitment/recruitment >> transport fredag\n >> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Novo >> Nordisk/Recruitment/re.transp.mandag\n >> Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/Medicon\n >> Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/clients/MOUNT\n >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Bergen Bybane\n >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Library1\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n >> Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection >> TLSReadFailed 7/34), reset state and waiting to retry... >> Aug 24 11:42:09 yoiko bbackupd[11514]: File statistics: total file >> size uploaded 13410118455, bytes already on server 341406646, encoded >> size 9996343505 >> Aug 24 11:42:09 yoiko bbackupd[11514]: TRACE: Wait on command socket, >> delay = 1024000000\n >> Aug 24 11:59:13 yoiko bbackupd[11514]: TRACE: Wait on command socket, >> delay = 1024000000\n >> >> Tobias >> >> >> Ben Summers wrote: >>> Please turn on ExtendedLogging = yes on client and server, and we'll >>> see what it's doing at the time of the exception. >>> Ben >>> On 2 Nov 2006, at 11:09, Tobias Balle-Petersen wrote: >>>> Hello again. >>>> >>>> Would changing the KeepAliveTime parameter ,as a fix for my problem, >>>> make sense considering the backup run ALWAYS completes on the first >>>> run since a reload of the bbackupd.conf file ? The point being that >>>> the first run completes with the KeepAliveTime = 10. >>>> >>>> Tobias >>>> >>>> >>>> Ben Summers wrote: >>>>> Try using KeepAliveTime in bbackupd.conf, if you're not already >>>>> doing so. >>>>> Ben >>>>> On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: >>>>>> Hello. >>>>>> >>>>>> I'm running box 0.10 client on a OS X machine and the 0.10 store >>>>>> on a debian machine. >>>>>> I'm, backing up more than 1TB of data. bbackupd will complete if >>>>>> run for the first time since a reload of the .conf file. If the >>>>>> .conf is not reloaded prior to every backup run, it wil fail with >>>>>> the error below. If I use the StoreObjectInfoFile feature and >>>>>> reload the .conf prior to every run, the backup still fails with >>>>>> the error below. >>>>>> >>>>>> The error: >>>>>> Exception caught (Connection Protocol_Timeout (Probably a network >>>>>> issue >>>>>> between client and server.) 7/41), reset state and waiting to >>>>>> retry... >>>>>> >>>>>> >>>>>> The conjobs that avoids the errors: >>>>>> 20 10 * * * /usr/local/bin/bbackupctl -c >>>>>> /etc/box_minimoni/bbackupd.conf reload >>>>>> 25 10 * * * /usr/local/bin/bbackupctl -c >>>>>> /etc/box_minimoni/bbackupd.conf sync >>>>>> >>>>>> >>>>>> Any ideas? >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> boxbackup mailing list >>>>>> boxbackup at fluffy.co.uk >>>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>>> _______________________________________________ >>>>> boxbackup mailing list >>>>> boxbackup at fluffy.co.uk >>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>> >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 2 20:37:09 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 2 Nov 2006 20:37:09 +0000 Subject: [Box Backup] Restore Symlinks In-Reply-To: <45478D4A.2010504@jetnet.ch> References: <4546147D.7040405@jetnet.ch> <22F47F23-283C-47BE-A94A-AEA061F1673B@fluffy.co.uk> <45478D4A.2010504@jetnet.ch> Message-ID: Do the tests for 0.10 pass on that box? Something's not quite right there. Ben On 31 Oct 2006, at 17:52, Andreas Schrafl wrote: > The problem occurred when restoring to a OpenBSD 4.0 box. > The server was is a OpenBSD 3.9 machine. As I reported when running > the restore on a windows it works. > I also tried to restore to the server so no OS differences there > (the machine that was backuped was 3.7 but that should not be of > any interest for a backup application) > > I've run ./runtest.pl ALL on the server (where I compiled the > files) for the 0.09 and the 0.10 version: > > 0.09: > PASSED > Block 7c6723c0 freed, but not known. Error? Or allocated in startup > static allocation? > -------- > common: PASSED > crypto: PASSED > compress: FAILED: Exception caught: St9bad_alloc > basicserver: PASSED > raidfile: PASSED > backupstore: Block 8bd9b200 freed, but not known. Error? Or > allocated in startup static allocation? > backupstorefix: PASSED > backupstorepatch: Block 7fa751c0 freed, but not known. Error? Or > allocated in startup static allocation? > backupdiff: Block 7e6f5000 freed, but not known. Error? Or > allocated in startup static allocation? > bbackupd: Block 7c6723c0 freed, but not known. Error? Or allocated > in startup static allocation? > > > 0.10: > PASSED > -------- > common: PASSED > crypto: PASSED > compress: FAILED: Exception caught: St9bad_alloc > basicserver: PASSED > raidfile: PASSED > backupstore: PASSED > backupstorefix: PASSED > backupstorepatch: PASSED > backupdiff: PASSED > bbackupd: PASSED > > So there seem to be some issues. I'm not sure what they mean (not a > programmer) but perhaps you all can tell me. > > Ben Summers wrote: >> Andy >> What platform are you using? >> Could you run the tests on your client, and see if you get any >> failures? Symlink support is explicitly tested with a backup and >> restore cycle. >> Ben >> On 30 Oct 2006, at 15:04, Andreas Schrafl wrote: >>> I'm very glad I used BoxBackup. Saved all my data after a raid >>> crashed and the restore shreddered the data that was left. >>> >>> Now I encountered some problem while restoring concerning symlinks. >>> >>> Simply it didn't restore them. I had several symlinks like form >>> one directory to an other (same mountpoint). These symlink just >>> would not be restored. >>> >>> One symlink even crashed the bbackupquery session repeatedly and >>> so I had to restore everything surrounding this directory >>> separately. >>> I tried to fix it on the server (checking the account and fixing >>> it) but there were no errors so nothing to fix. >>> >>> I extracted this directory to a windows machinne and there it >>> just skipped these files. >>> >>> Perhaps this got fixed lately (still using 0.09 but I could not >>> find anything on this topic). >>> >>> Thanks for the nice backuptool it really is simple an useful. >>> Andy >>> >>> PS: are there any special things to look out for when updating a >>> server from 0.09 to 0.10? >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > -- > Andreas Schrafl > Unterdorfstrasse 19 > 8124 Maur > Switzerland > > Tel.: +41 (0)44 980 3842 > Mobile: +41 (0)76 433 3842 > Fax: +41 (0)44 980 4031 > E-mail: andy at jetnet.ch > Web: personal.jetnet.ch/andy > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Thu Nov 2 21:07:31 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 2 Nov 2006 21:07:31 +0000 (GMT) Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <4549D56C.6060602@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> Message-ID: Hi Tobias, Ben and all, On Thu, 2 Nov 2006, Tobias Balle-Petersen wrote: > I have had this problem for a while. Here is an older log from the > client showing the error: [...] > Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record for > /Volumes/raidA/files/guests/Bergen Bybane\n > Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record for > /Volumes/raidA/files/guests/Library1\n > Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: > ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n > Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: > ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n > Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection > TLSReadFailed 7/34), reset state and waiting to retry... I have an idea what the problem might be. If bbackupd spends a long time scanning directories without sending anything to the server (which might well be the case if you have 1 Tb of data in mainly small files) then the connection could still time out. The KeepAliveTime relates only to sending keepalives while diffing a single file. It does nothing while bbackupd is scanning directories. Tobias, I could come up with a patched version against trunk which does implement keepalives while scanning directories if you're willing to test it. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Nov 2 21:25:58 2006 From: boxbackup at fluffy.co.uk (tbp) Date: Thu, 02 Nov 2006 22:25:58 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> Message-ID: <454A6266.40706@kontrapunkt.com> Hello Chris. I think you are right about the source of my problem. I have never had the problem with backups of "smaller" filesystems. If you could let me try out an alternative version of bbackupd I would love to test it. There is no need to make any changes on the store right? Still, is it not strange that the problem does not occur after a reloaf of the bbackupd.cong file? Thanks, Tobias Chris Wilson wrote: > Hi Tobias, Ben and all, > > On Thu, 2 Nov 2006, Tobias Balle-Petersen wrote: > >> I have had this problem for a while. Here is an older log from the >> client showing the error: > [...] >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Bergen Bybane\n >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Library1\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n >> Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection >> TLSReadFailed 7/34), reset state and waiting to retry... > > I have an idea what the problem might be. If bbackupd spends a long time > scanning directories without sending anything to the server (which might > well be the case if you have 1 Tb of data in mainly small files) then > the connection could still time out. The KeepAliveTime relates only to > sending keepalives while diffing a single file. It does nothing while > bbackupd is scanning directories. > > Tobias, I could come up with a patched version against trunk which does > implement keepalives while scanning directories if you're willing to > test it. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 2 21:36:46 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 2 Nov 2006 21:36:46 +0000 (GMT) Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <454A6266.40706@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <454A6266.40706@kontrapunkt.com> Message-ID: Hi Tobias, > I think you are right about the source of my problem. I have never had > the problem with backups of "smaller" filesystems. > > If you could let me try out an alternative version of bbackupd I would > love to test it. OK, I'm working on it now. > There is no need to make any changes on the store right? No, the store will not be affected, assuming that you are running 0.10 which already has support for keepalives on the store side (I think you are). > Still, is it not strange that the problem does not occur after a reloaf > of the bbackupd.cong file? No, not strange. Box Backup caches a lot of information on the first run or after a reload, which requires frequent round trips to the store, listing the contents of each directory in turn. Once the data is cached (and not discarded by a reload), subsequent runs may not need to contact the store at all if nothing has changed, or contact it only for those directories which have changed. If you back up a large data set where only a few files change between runs, you might see a long delay between data requests from the store, which might cause the SSL layer to time out. Your extended log appears to show this happening. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Nov 2 21:43:36 2006 From: boxbackup at fluffy.co.uk (tbp) Date: Thu, 02 Nov 2006 22:43:36 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <454A6266.40706@kontrapunkt.com> Message-ID: <454A6688.1040905@kontrapunkt.com> Hello Chris. Chris Wilson wrote: > Hi Tobias, > >> I think you are right about the source of my problem. I have never had >> the problem with backups of "smaller" filesystems. >> >> If you could let me try out an alternative version of bbackupd I would >> love to test it. > > OK, I'm working on it now. Sounds wonderfull. Thank you! >> Still, is it not strange that the problem does not occur after a >> reloaf of the bbackupd.cong file? > > No, not strange. Box Backup caches a lot of information on the first run > or after a reload, which requires frequent round trips to the store, > listing the contents of each directory in turn. Once the data is cached > (and not discarded by a reload), subsequent runs may not need to contact > the store at all if nothing has changed, or contact it only for those > directories which have changed. > > If you back up a large data set where only a few files change between > runs, you might see a long delay between data requests from the store, > which might cause the SSL layer to time out. Your extended log appears > to show this happening. OK. Thanks, Tobias From boxbackup at fluffy.co.uk Fri Nov 3 14:34:59 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Fri, 03 Nov 2006 15:34:59 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <4549F25E.80903@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <16755366-735A-4C21-BD6D-A8CB112B5853@fluffy.co.uk> <4549F25E.80903@kontrapunkt.com> Message-ID: <454B5393.7000800@kontrapunkt.com> Hello. I think Chris' suggestion as to the cause of the timeouts is correct. If an alternative version of bbackupd does not resolve the problem I will post relevant logs. Have a nice weekend, Tobias Tobias Balle-Petersen wrote: > Hello. > > OK. I'll enable ExtendedLogging and report back. > > Thank you for helping out, > Tobias > > > > > Ben Summers wrote: >> >> That's logging from a debug build, not with ExtendedLogging set in the >> .conf file. I'm afraid that doesn't really tell me much. >> >> Ben >> >> >> On 2 Nov 2006, at 11:24, Tobias Balle-Petersen wrote: >> >>> Hello. >>> >>> Just to let you know, I have tried to delete the contents of >>> /var/bbackupd >>> >>> >>> >>> I have had this problem for a while. >>> Here is an older log from the client showing the error: >>> >>> Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating >>> EncodingBuffer from 2601 to 4777\n >>> Aug 24 11:11:27 yoiko bbackupd[11514]: TRACE: Reallocating >>> EncodingBuffer from 1068 to 2606\n >>> ................... >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating >>> EncodingBuffer from 1068 to 3244\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Reallocating >>> EncodingBuffer from 3244 to 5413\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/recruitment_ms\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/recruitment_tos\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/recruitment2_ms\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/re.kmo\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/recruitment transport fredag\n >>> Aug 24 11:16:19 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Novo >>> Nordisk/Recruitment/re.transp.mandag\n >>> Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/Medicon\n >>> Aug 24 11:20:28 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/clients/MOUNT\n >>> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/guests/Bergen Bybane\n >>> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory >>> record for /Volumes/raidA/files/guests/Library1\n >>> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >>> ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n >>> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >>> ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n >>> Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection >>> TLSReadFailed 7/34), reset state and waiting to retry... >>> Aug 24 11:42:09 yoiko bbackupd[11514]: File statistics: total file >>> size uploaded 13410118455, bytes already on server 341406646, encoded >>> size 9996343505 >>> Aug 24 11:42:09 yoiko bbackupd[11514]: TRACE: Wait on command socket, >>> delay = 1024000000\n >>> Aug 24 11:59:13 yoiko bbackupd[11514]: TRACE: Wait on command socket, >>> delay = 1024000000\n >>> >>> Tobias >>> >>> >>> Ben Summers wrote: >>>> Please turn on ExtendedLogging = yes on client and server, and we'll >>>> see what it's doing at the time of the exception. >>>> Ben >>>> On 2 Nov 2006, at 11:09, Tobias Balle-Petersen wrote: >>>>> Hello again. >>>>> >>>>> Would changing the KeepAliveTime parameter ,as a fix for my >>>>> problem, make sense considering the backup run ALWAYS completes on >>>>> the first run since a reload of the bbackupd.conf file ? The point >>>>> being that the first run completes with the KeepAliveTime = 10. >>>>> >>>>> Tobias >>>>> >>>>> >>>>> Ben Summers wrote: >>>>>> Try using KeepAliveTime in bbackupd.conf, if you're not already >>>>>> doing so. >>>>>> Ben >>>>>> On 2 Nov 2006, at 10:51, Tobias Balle-Petersen wrote: >>>>>>> Hello. >>>>>>> >>>>>>> I'm running box 0.10 client on a OS X machine and the 0.10 store >>>>>>> on a debian machine. >>>>>>> I'm, backing up more than 1TB of data. bbackupd will complete if >>>>>>> run for the first time since a reload of the .conf file. If the >>>>>>> .conf is not reloaded prior to every backup run, it wil fail with >>>>>>> the error below. If I use the StoreObjectInfoFile feature and >>>>>>> reload the .conf prior to every run, the backup still fails with >>>>>>> the error below. >>>>>>> >>>>>>> The error: >>>>>>> Exception caught (Connection Protocol_Timeout (Probably a network >>>>>>> issue >>>>>>> between client and server.) 7/41), reset state and waiting to >>>>>>> retry... >>>>>>> >>>>>>> >>>>>>> The conjobs that avoids the errors: >>>>>>> 20 10 * * * /usr/local/bin/bbackupctl -c >>>>>>> /etc/box_minimoni/bbackupd.conf reload >>>>>>> 25 10 * * * /usr/local/bin/bbackupctl -c >>>>>>> /etc/box_minimoni/bbackupd.conf sync >>>>>>> >>>>>>> >>>>>>> Any ideas? >>>>>>> >>>>>>> Thanks, >>>>>>> Tobias >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> boxbackup mailing list >>>>>>> boxbackup at fluffy.co.uk >>>>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>>>> _______________________________________________ >>>>>> boxbackup mailing list >>>>>> boxbackup at fluffy.co.uk >>>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>>> >>>>> _______________________________________________ >>>>> boxbackup mailing list >>>>> boxbackup at fluffy.co.uk >>>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Sun Nov 5 20:16:21 2006 From: boxbackup at fluffy.co.uk (Andreas Schrafl) Date: Sun, 05 Nov 2006 21:16:21 +0100 Subject: [Box Backup] Restore Symlinks In-Reply-To: References: <4546147D.7040405@jetnet.ch> <22F47F23-283C-47BE-A94A-AEA061F1673B@fluffy.co.uk> <45478D4A.2010504@jetnet.ch> Message-ID: <454E4695.7050301@jetnet.ch> As I wrote the compress test for 0.10 failes: compress: FAILED: Exception caught: St9bad_alloc this on 2 different i386 boxes running obsd 4.0 any idea? Ben Summers wrote: > > Do the tests for 0.10 pass on that box? Something's not quite right there. > > Ben > > > On 31 Oct 2006, at 17:52, Andreas Schrafl wrote: > >> The problem occurred when restoring to a OpenBSD 4.0 box. >> The server was is a OpenBSD 3.9 machine. As I reported when running >> the restore on a windows it works. >> I also tried to restore to the server so no OS differences there (the >> machine that was backuped was 3.7 but that should not be of any >> interest for a backup application) >> >> I've run ./runtest.pl ALL on the server (where I compiled the files) >> for the 0.09 and the 0.10 version: >> >> 0.09: >> PASSED >> Block 7c6723c0 freed, but not known. Error? Or allocated in startup >> static allocation? >> -------- >> common: PASSED >> crypto: PASSED >> compress: FAILED: Exception caught: St9bad_alloc >> basicserver: PASSED >> raidfile: PASSED >> backupstore: Block 8bd9b200 freed, but not known. Error? Or allocated >> in startup static allocation? >> backupstorefix: PASSED >> backupstorepatch: Block 7fa751c0 freed, but not known. Error? Or >> allocated in startup static allocation? >> backupdiff: Block 7e6f5000 freed, but not known. Error? Or allocated >> in startup static allocation? >> bbackupd: Block 7c6723c0 freed, but not known. Error? Or allocated in >> startup static allocation? >> >> >> 0.10: >> PASSED >> -------- >> common: PASSED >> crypto: PASSED >> compress: FAILED: Exception caught: St9bad_alloc >> basicserver: PASSED >> raidfile: PASSED >> backupstore: PASSED >> backupstorefix: PASSED >> backupstorepatch: PASSED >> backupdiff: PASSED >> bbackupd: PASSED >> >> So there seem to be some issues. I'm not sure what they mean (not a >> programmer) but perhaps you all can tell me. >> >> Ben Summers wrote: >>> Andy >>> What platform are you using? >>> Could you run the tests on your client, and see if you get any >>> failures? Symlink support is explicitly tested with a backup and >>> restore cycle. >>> Ben >>> On 30 Oct 2006, at 15:04, Andreas Schrafl wrote: >>>> I'm very glad I used BoxBackup. Saved all my data after a raid >>>> crashed and the restore shreddered the data that was left. >>>> >>>> Now I encountered some problem while restoring concerning symlinks. >>>> >>>> Simply it didn't restore them. I had several symlinks like form one >>>> directory to an other (same mountpoint). These symlink just would >>>> not be restored. >>>> >>>> One symlink even crashed the bbackupquery session repeatedly and so >>>> I had to restore everything surrounding this directory separately. >>>> I tried to fix it on the server (checking the account and fixing it) >>>> but there were no errors so nothing to fix. >>>> >>>> I extracted this directory to a windows machinne and there it just >>>> skipped these files. >>>> >>>> Perhaps this got fixed lately (still using 0.09 but I could not find >>>> anything on this topic). >>>> >>>> Thanks for the nice backuptool it really is simple an useful. >>>> Andy >>>> >>>> PS: are there any special things to look out for when updating a >>>> server from 0.09 to 0.10? >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> --Andreas Schrafl >> Unterdorfstrasse 19 >> 8124 Maur >> Switzerland >> >> Tel.: +41 (0)44 980 3842 >> Mobile: +41 (0)76 433 3842 >> Fax: +41 (0)44 980 4031 >> E-mail: andy at jetnet.ch >> Web: personal.jetnet.ch/andy >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup -- Andreas Schrafl Unterdorfstrasse 19 8124 Maur Switzerland Tel.: +41 (0)44 980 3842 Mobile: +41 (0)76 433 3842 Fax: +41 (0)44 980 4031 E-mail: andy at jetnet.ch Web: personal.jetnet.ch/andy From boxbackup at fluffy.co.uk Wed Nov 8 16:52:36 2006 From: boxbackup at fluffy.co.uk (Andreas Schrafl) Date: Wed, 08 Nov 2006 17:52:36 +0100 Subject: [Box Backup] multiple Names for bbstore Message-ID: <45520B54.40104@jetnet.ch> I have a server running in two networks. I make backups over both networks and since I'd like to use the most effective way I wanted to try to contact bbstore on both names. So same machine, different networkcards, different networks, different names, same bbstore. Is this possible? Second since when I switch one to the internal network I'll get trouble when taking notebook within both networks. Is it for bbackup possible to look for a running server? thanks andy From boxbackup at fluffy.co.uk Wed Nov 8 16:55:42 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 8 Nov 2006 16:55:42 +0000 Subject: [Box Backup] multiple Names for bbstore In-Reply-To: <45520B54.40104@jetnet.ch> References: <45520B54.40104@jetnet.ch> Message-ID: On 8 Nov 2006, at 16:52, Andreas Schrafl wrote: > I have a server running in two networks. > > I make backups over both networks and since I'd like to use the > most effective way I wanted to try to contact bbstore on both > names. So same machine, different networkcards, different networks, > different names, same bbstore. > > Is this possible? Yes. Use ListenAddresses = inet:0.0.0.0 to listen on all interfaces, arrange for bbstored to connect to any of them. The name in the certificate is not checked, just that it's signed by the Server CA. > > Second since when I switch one to the internal network I'll get > trouble when taking notebook within both networks. Is it for > bbackup possible to look for a running server? Look into using SyncAllowScript in bbstored.conf. Write a script to determine whether the store is reachable. Ben From boxbackup at fluffy.co.uk Wed Nov 8 20:14:19 2006 From: boxbackup at fluffy.co.uk (tbp) Date: Wed, 08 Nov 2006 21:14:19 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> Message-ID: <45523A9B.1030003@kontrapunkt.com> Hello. Maybe I could run the touch command on some of the directories in my file hierarchy and avoid the timeouts that way? Tobias Chris Wilson wrote: > Hi Tobias, Ben and all, > > On Thu, 2 Nov 2006, Tobias Balle-Petersen wrote: > >> I have had this problem for a while. Here is an older log from the >> client showing the error: > [...] >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Bergen Bybane\n >> Aug 24 11:20:44 yoiko bbackupd[11514]: TRACE: Deleted directory record >> for /Volumes/raidA/files/guests/Library1\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSReadFailed) at SocketStreamTLS.cpp(361)\n >> Aug 24 11:41:58 yoiko bbackupd[11514]: TRACE: Exception thrown: >> ConnectionException(Conn_TLSWriteFailed) at SocketStreamTLS.cpp(426)\n >> Aug 24 11:41:59 yoiko bbackupd[11514]: Exception caught (Connection >> TLSReadFailed 7/34), reset state and waiting to retry... > > I have an idea what the problem might be. If bbackupd spends a long time > scanning directories without sending anything to the server (which might > well be the case if you have 1 Tb of data in mainly small files) then > the connection could still time out. The KeepAliveTime relates only to > sending keepalives while diffing a single file. It does nothing while > bbackupd is scanning directories. > > Tobias, I could come up with a patched version against trunk which does > implement keepalives while scanning directories if you're willing to > test it. > > Cheers, Chris. > -- > _ ___ __ _ > / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | > \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed Nov 8 20:21:02 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 8 Nov 2006 23:21:02 +0300 (EAT) Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <45523A9B.1030003@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <45523A9B.1030003@kontrapunkt.com> Message-ID: Hi Tobias, > Maybe I could run the touch command on some of the directories in my > file hierarchy and avoid the timeouts that way? Touching files might well work, I don't think touching directories would. I'm still working on the keepalive code, I'm afraid it turned into a bigger rewrite than I had hoped. I hope to have a new version ready for you to test soon. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Wed Nov 8 21:21:58 2006 From: boxbackup at fluffy.co.uk (tbp) Date: Wed, 08 Nov 2006 22:21:58 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <45523A9B.1030003@kontrapunkt.com> Message-ID: <45524A76.2070803@kontrapunkt.com> Hi Chris. I guess I could create som hidden files in my directories and touch those. Thank you for all your hard work on Box Backup! Tobias Chris Wilson wrote: > Hi Tobias, > >> Maybe I could run the touch command on some of the directories in my >> file hierarchy and avoid the timeouts that way? > > Touching files might well work, I don't think touching directories would. > > I'm still working on the keepalive code, I'm afraid it turned into a > bigger rewrite than I had hoped. I hope to have a new version ready for > you to test soon. > > Cheers, Chris. From boxbackup at fluffy.co.uk Wed Nov 8 22:18:37 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 08 Nov 2006 23:18:37 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <45524A76.2070803@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <45523A9B.1030003@kontrapunkt.com> <45524A76.2070803@kontrapunkt.com> Message-ID: <455257BD.7020800@kontrapunkt.com> To test if touching files can avoid my timeouts, I have created a cronjob to create hidden files that will be touched daily prior to the backup run. I will report back with the result. My command to create the hidden files called .no_timeout: find clients -type d -maxdepth 1 -mindepth 1 -exec touch {}/.no_timeout \; Tobias tbp wrote: > Hi Chris. > > I guess I could create som hidden files in my directories and touch those. > > Thank you for all your hard work on Box Backup! > > Tobias > > > Chris Wilson wrote: >> Hi Tobias, >> >>> Maybe I could run the touch command on some of the directories in my >>> file hierarchy and avoid the timeouts that way? >> Touching files might well work, I don't think touching directories would. >> >> I'm still working on the keepalive code, I'm afraid it turned into a >> bigger rewrite than I had hoped. I hope to have a new version ready for >> you to test soon. >> >> Cheers, Chris. > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 9 18:46:46 2006 From: boxbackup at fluffy.co.uk (Andreas Schrafl) Date: Thu, 09 Nov 2006 19:46:46 +0100 Subject: [Box Backup] multiple Names for bbstore In-Reply-To: References: <45520B54.40104@jetnet.ch> Message-ID: <45537796.6020701@jetnet.ch> Thanks. this works nice. Have changed it to several addresses, separated with comma and it works. great. Thing with the script good idea. Thanks Andy Ben Summers wrote: > > On 8 Nov 2006, at 16:52, Andreas Schrafl wrote: > >> I have a server running in two networks. >> >> I make backups over both networks and since I'd like to use the most >> effective way I wanted to try to contact bbstore on both names. So >> same machine, different networkcards, different networks, different >> names, same bbstore. >> >> Is this possible? > > Yes. Use ListenAddresses = inet:0.0.0.0 to listen on all interfaces, > arrange for bbstored to connect to any of them. The name in the > certificate is not checked, just that it's signed by the Server CA. > > >> >> Second since when I switch one to the internal network I'll get >> trouble when taking notebook within both networks. Is it for bbackup >> possible to look for a running server? > > Look into using SyncAllowScript in bbstored.conf. Write a script to > determine whether the store is reachable. > > Ben > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup -- Andreas Schrafl Unterdorfstrasse 19 8124 Maur Switzerland Tel.: +41 (0)44 980 3842 Mobile: +41 (0)76 433 3842 Fax: +41 (0)44 980 4031 E-mail: andy at jetnet.ch Web: personal.jetnet.ch/andy From boxbackup at fluffy.co.uk Mon Nov 13 15:08:56 2006 From: boxbackup at fluffy.co.uk (Andreas Schrafl) Date: Mon, 13 Nov 2006 16:08:56 +0100 Subject: [Box Backup] multiple servers from windows Message-ID: <45588A88.5030704@jetnet.ch> This is an strange request but I think perhaps somebody of you knows how to do this: I have a backup server that serves in two networks: extern.domain.com intern.domain.com Now I can change the settings every time I use my notebook in a different network, but nicer would be to use the SyncAllow script to test if the intern.domain.com is reachable, then connect to that one and if it isn't then connect to extern.domain.com I'm trying with the boxscript.bat that was posted but I'm not sure about how to change to configuration. Thanks for any idea's and if anybody else would be interested. Andy From boxbackup at fluffy.co.uk Mon Nov 13 15:13:54 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Mon, 13 Nov 2006 15:13:54 +0000 Subject: [Box Backup] multiple servers from windows In-Reply-To: <45588A88.5030704@jetnet.ch> References: <45588A88.5030704@jetnet.ch> Message-ID: <947CC6EC-0981-4A79-A043-EFD34401F0D5@fluffy.co.uk> Can you play with the DNS so the same name resolves to different IPs on different networks, or allow the networks to route between themselves? Ben On 13 Nov 2006, at 15:08, Andreas Schrafl wrote: > This is an strange request but I think perhaps somebody of you > knows how to do this: > > I have a backup server that serves in two networks: > > extern.domain.com > intern.domain.com > > Now I can change the settings every time I use my notebook in a > different network, but nicer would be to use the SyncAllow script > to test if the intern.domain.com is reachable, then connect to that > one and if it isn't then connect to extern.domain.com > > I'm trying with the boxscript.bat that was posted but I'm not sure > about how to change to configuration. > > Thanks for any idea's and if anybody else would be interested. > Andy > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Nov 14 03:27:00 2006 From: boxbackup at fluffy.co.uk (Stuart Sanders) Date: Tue, 14 Nov 2006 11:27:00 +0800 Subject: [Box Backup] multiple servers from windows Message-ID: I have something like this working using 2 DNS servers for my email system. I use an external host to maintain the majority of the DNS records for my d= omain. It points to the fixed ip address at my office using the name north= point.bitshk.com. Inside the office network I use an internal DNS server that forwards all DN= S requests except for my domain to my ISP (not the same as the domain host)= . So internally the DNS looks like northpoint.bitshk.com - 192.168.10.10. Since I am in and out of the office a lot, sometimes working from home or m= aybe at client sites for multiple days or even in and out several times dur= ing the day, this keeps the transition for IMAP email and server management= simple as I don't need to change any settings regardless of where I might = be. Stuart > -----Original Message----- > From: boxbackup-admin at fluffy.co.uk=20 > [mailto:boxbackup-admin at fluffy.co.uk] On Behalf Of Ben Summers > Sent: Monday, 13 November 2006 11:14 PM > To: boxbackup at fluffy.co.uk > Subject: Re: [Box Backup] multiple servers from windows >=20 >=20 >=20 > Can you play with the DNS so the same name resolves to different IPs =20 > on different networks, or allow the networks to route between =20 > themselves? >=20 > Ben >=20 >=20 >=20 > On 13 Nov 2006, at 15:08, Andreas Schrafl wrote: >=20 > > This is an strange request but I think perhaps somebody of you =20 > > knows how to do this: > > > > I have a backup server that serves in two networks: > > > > extern.domain.com > > intern.domain.com > > > > Now I can change the settings every time I use my notebook in a =20 > > different network, but nicer would be to use the SyncAllow script =20 > > to test if the intern.domain.com is reachable, then connect=20 > to that =20 > > one and if it isn't then connect to extern.domain.com > > > > I'm trying with the boxscript.bat that was posted but I'm not sure =20 > > about how to change to configuration. > > > > Thanks for any idea's and if anybody else would be interested. > > Andy > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup >=20 > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue Nov 14 08:34:19 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Tue, 14 Nov 2006 09:34:19 +0100 Subject: [Box Backup] Need to reload bbackupd conf to avoid timeouts? In-Reply-To: <455257BD.7020800@kontrapunkt.com> References: <4549CDB9.60809@kontrapunkt.com> <0B3F5141-1D55-429D-8BBD-95C254999CA2@fluffy.co.uk> <4549D1DB.90109@kontrapunkt.com> <972809B3-D977-465B-9CCC-057F3FE64B07@fluffy.co.uk> <4549D56C.6060602@kontrapunkt.com> <45523A9B.1030003@kontrapunkt.com> <45524A76.2070803@kontrapunkt.com> <455257BD.7020800@kontrapunkt.com> Message-ID: <45597F8B.2010709@kontrapunkt.com> Hello. Touching the hidden files prior to backup-runs seems to have solved my problem. Tobias Tobias Balle-Petersen wrote: > To test if touching files can avoid my timeouts, I have created a > cronjob to create hidden files that will be touched daily prior to the > backup run. I will report back with the result. > > My command to create the hidden files called .no_timeout: > find clients -type d -maxdepth 1 -mindepth 1 -exec touch {}/.no_timeout \; > > Tobias > > > tbp wrote: >> Hi Chris. >> >> I guess I could create som hidden files in my directories and touch those. >> >> Thank you for all your hard work on Box Backup! >> >> Tobias >> >> >> Chris Wilson wrote: >>> Hi Tobias, >>> >>>> Maybe I could run the touch command on some of the directories in my >>>> file hierarchy and avoid the timeouts that way? >>> Touching files might well work, I don't think touching directories would. >>> >>> I'm still working on the keepalive code, I'm afraid it turned into a >>> bigger rewrite than I had hoped. I hope to have a new version ready for >>> you to test soon. >>> >>> Cheers, Chris. >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Tue Nov 14 14:54:33 2006 From: boxbackup at fluffy.co.uk (Torsten Boob) Date: Tue, 14 Nov 2006 15:54:33 +0100 Subject: [Box Backup] keeping files specified time Message-ID: <20061114155433.70f385f3@matrix.tuxianer.homelinux.net> Hello, if the Softlimit is reached for a user, then old Files will be deleted by the housekeeping process. So only one, the last, version of the file is backed up. I would like to have a function to keep the files several days. In traditional Backup Systems it is possible to recover files, that are n days old. Is it possible to implement such an function into boxbackup ? I imagine to modifiy the function HousekeepStoreAccount::DeleteFiles() not to delete files younger than n days. Do i have to pay attention to other functions not to get into trouble with side effects ? Is there an other oder better way for my dreams ? :) Thanks Torsten From boxbackup at fluffy.co.uk Tue Nov 14 16:09:15 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 14 Nov 2006 16:09:15 +0000 Subject: [Box Backup] keeping files specified time In-Reply-To: <20061114155433.70f385f3@matrix.tuxianer.homelinux.net> References: <20061114155433.70f385f3@matrix.tuxianer.homelinux.net> Message-ID: <3DC171CA-E6C1-4F03-84E0-04B0B948EA7C@fluffy.co.uk> The current algorithm is generally sufficient to maintain a good range of old versions of files. It doesn't have the level of control that you want, but we're planning this for a future version. If you do want to mess with the code, find this clause // Potentially add it to the list if it's deleted, if it's an old version or deleted if((enFlags & (BackupStoreDirectory::Entry::Flags_Deleted | BackupStoreDirectory::Entry::Flags_OldVersion)) != 0) { in HousekeepStoreAccount::ScanDirectory() (around line 350 or so) and then add extra bits to stop it becoming true. This clause selects the files which might be deleted. Ben On 14 Nov 2006, at 14:54, Torsten Boob wrote: > Hello, > > if the Softlimit is reached for a user, then old Files will be > deleted by the housekeeping process. So only one, the last, version > of the file is backed up. > > I would like to have a function to keep the files several days. In > traditional Backup Systems it is possible to recover files, that > are n days old. > > Is it possible to implement such an function into boxbackup ? I > imagine > to modifiy the function HousekeepStoreAccount::DeleteFiles() not to > delete files younger than n days. Do i have to pay attention to > other functions not to get into trouble with side effects ? > > Is there an other oder better way for my dreams ? :) > > Thanks > Torsten > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Nov 14 21:27:32 2006 From: boxbackup at fluffy.co.uk (Dave Bamford) Date: Tue, 14 Nov 2006 21:27:32 +0000 Subject: [Box Backup] OpenBSD 3.4 binaries Message-ID: <455A34C4.60601@logical-progress.com> Hi Can anyone generate v 0.10 binaries for OpenBSD 3.4 I have an old cut down server but no means of compiling and wish to migrate from 0.09. I think last time Ben very kindly supplied binaries for me. Just need the client and bbackupquery please Thanks Dave Bamford From boxbackup at fluffy.co.uk Tue Nov 14 21:33:00 2006 From: boxbackup at fluffy.co.uk (=?ISO-8859-1?Q?Simon_Lundstr=F6m?=) Date: Tue, 14 Nov 2006 22:33:00 +0100 Subject: [Box Backup] OpenBSD 3.4 binaries In-Reply-To: <455A34C4.60601@logical-progress.com> References: <455A34C4.60601@logical-progress.com> Message-ID: <05FBC261-A30B-45A9-A019-3ACCDFF2176C@gmail.com> --Apple-Mail-17-549187659 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed You can create it on your of if you have another PC with qemu or the free VMWare Player or Server. 3.4 is _really_ old and not supported by OpenBSD, you should upgrade. - Simon On Nov 14, 2006, at 10:27 PM, Dave Bamford wrote: > Hi > > Can anyone generate v 0.10 binaries for OpenBSD 3.4 > I have an old cut down server but no means of compiling and wish > to migrate from 0.09. I think last time Ben very kindly supplied > binaries > for me. Just need the client and bbackupquery please > > Thanks > > Dave Bamford > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup --Apple-Mail-17-549187659 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGMTCCAuow ggJToAMCAQICEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDgyNTE2NDc1OVoXDTA3MDgyNTE2NDc1 OVowSjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEnMCUGCSqGSIb3DQEJARYYc2lt b25sdW5kc3Ryb21AZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArrb+ /12bXTb5oixk4K2QhcJtsN6o3bu3p/fJK9xTP4gtF7NpnOzTrnRbmGGQ2HJEgV3bVuGOIaR1qqyk toR5RzxM3QXkepq0KUxdpCJgkDu/GlG/z7It1Wh5y+J5suPhVyWjr3n2RYJ5J+nssP/pw1EiLV3g OH7imu9uUPCziK+J2glMXwFNE/pA+xNIPAflNFpdaexIYAbKnm/N119usWLZlzCoIuj9yTSSV0wG /SpPN5UJtA0O0zxD6GMLFfm0zFc9e0sAvynaVIfVDmSEvY3FzrfGg/fZiCWFHRcc/Q23Olw/OxDw ioSo1DfQU2e2XzBiX5bX9ApUbQ+RFZRGHQIDAQABozUwMzAjBgNVHREEHDAagRhzaW1vbmx1bmRz dHJvbUBnbWFpbC5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOBgQAmV6ABcDdunTJ8 5SkJl6oEa0lIh8+dfXseipg4bMDCRiJahR5RvpGWFnNbDN66NxYFvIasAy9ss4VOsZ2Eq3FQkQNA 4UL4rqjJDF/XKGfRYEZpF55ZztDm54vUMLpXtkjZUCCzSQp+1X04d3Vjz/1r2PtX3MxnBnZdfQuk LvkJFjCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJBgNVBAYTAlpBMRUwEwYD VQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgGA1UEChMRVGhhd3RlIENv bnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNV BAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwt ZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJ BgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQD EyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDX AmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l 0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMG A1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVt YWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/TCG4 +DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amcOY6MIE9l X5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggMQMIIDDAIBATB2MGIx CzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYD VQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQQStBdrLuqCK7MX8wXDle 5TAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0wNjExMTQyMTMzMDFaMCMGCSqGSIb3DQEJBDEWBBTsjAQ1dgv2si2m0aSCT5KJIDkPWTCBhQYJ KwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC EEErQXay7qgiuzF/MFw5XuUwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMG A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEBBQAE ggEAl1iS3k+l9Ihn6MJtRWEj7X2W/s/bvExLf9s5i7JSwTqtO3rjNBZb3e47goQFpvE9y5bZu+8x A2DhxWxX1L25/cCLTCTyAucH5MNTxIKCUdN3cqZCuw0DAvgrFkpHiAhebBaU40psMMuxFta0OLWh 5H3/a+UO884j+tMKR8odTDhyNi1QaXt5AkTPU+6x8mteb/8nMkBsUKicPvSqf+xj5kuPY5Fq9o7p y9un9jUbR6ujk4/mgFMxHh+KfhS5HPmIrCBSians/V7XWap4/ZET8c3k8n7dwTYZZCWNxR85Tvhr taRgYiVX5tgG8IR44hHeBhP4b1DSX8j28SD9uXZQWwAAAAAAAA== --Apple-Mail-17-549187659-- From boxbackup at fluffy.co.uk Tue Nov 14 22:02:06 2006 From: boxbackup at fluffy.co.uk (Dave Bamford) Date: Tue, 14 Nov 2006 22:02:06 +0000 Subject: [Box Backup] OpenBSD 3.4 binaries In-Reply-To: <05FBC261-A30B-45A9-A019-3ACCDFF2176C@gmail.com> References: <455A34C4.60601@logical-progress.com> <05FBC261-A30B-45A9-A019-3ACCDFF2176C@gmail.com> Message-ID: <455A3CDE.5080200@logical-progress.com> I know I should upgrade, but the server is in use every day and up to now has been ultra reliable. An internal power supply just failed so I intend to build a backup server. Its the only one left on my 0.09 boxbackup server. The server is rather critical as it is used to dispense Methadone to addicts. Dave Simon Lundstr?m wrote: > You can create it on your of if you have another PC with qemu or the > free VMWare Player or Server. > > 3.4 is _really_ old and not supported by OpenBSD, you should upgrade. > > - Simon > > On Nov 14, 2006, at 10:27 PM, Dave Bamford wrote: > >> Hi >> >> Can anyone generate v 0.10 binaries for OpenBSD 3.4 >> I have an old cut down server but no means of compiling and wish >> to migrate from 0.09. I think last time Ben very kindly supplied >> binaries >> for me. Just need the client and bbackupquery please >> >> Thanks >> >> Dave Bamford >> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Sat Nov 18 14:52:03 2006 From: boxbackup at fluffy.co.uk (Paul MacKenzie) Date: Sat, 18 Nov 2006 09:52:03 -0500 Subject: [Box Backup] Box Backup in server child, exception BackupStore Message-ID: <7.0.1.0.2.20061118021442.04cb4c50@elehost.com> I am getting a lot of these errors in server child, exception BackupStore CouldNotFindUnusedIDDuringAllocation (4/53) -- termi Can anyone shed light on fixing this problem? Thanks for your assistance Paul From boxbackup at fluffy.co.uk Sun Nov 19 01:43:05 2006 From: boxbackup at fluffy.co.uk (Paul MacKenzie) Date: Sat, 18 Nov 2006 20:43:05 -0500 Subject: [Box Backup] Box Backup check fix Segmentation fault Message-ID: <7.0.1.0.2.20061118204118.034077b0@elehost.com> Hello, In addition I have a very odd error I am not sure how to get around SKS# /usr/local/bin/bbstoreaccounts check 12345SB fix NOTE: Will fix errors encountered during checking. Check store account ID 12345SB Phase 1, check objects... Phase 2, check directories... Directory ID 249 references object 4b33c4 which does not exist. Segmentation fault I am using the latest bbstored port compiled on a Freebsd machine Can anyone shed light on fixing this problem or getting around it so the store can be fixed? Thanks for your assistance Paul From boxbackup at fluffy.co.uk Sun Nov 19 09:13:20 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 19 Nov 2006 09:13:20 +0000 Subject: [Box Backup] Box Backup check fix Segmentation fault In-Reply-To: <7.0.1.0.2.20061118204118.034077b0@elehost.com> References: <7.0.1.0.2.20061118204118.034077b0@elehost.com> Message-ID: It's likely that this problem would cause the previous one you reported. What's the output of bbstoreaccounts info for that account? Can you make a guess at the number of files backed up in that account? (or at least is that number less than 4.9 million your output below might suggest) How old is the account? Ben On 19 Nov 2006, at 01:43, Paul MacKenzie wrote: > Hello, > > In addition I have a very odd error I am not sure how to get around > > SKS# /usr/local/bin/bbstoreaccounts check 12345SB fix > NOTE: Will fix errors encountered during checking. > Check store account ID 12345SB > Phase 1, check objects... > Phase 2, check directories... > Directory ID 249 references object 4b33c4 which does not exist. > Segmentation fault > > I am using the latest bbstored port compiled on a Freebsd machine > > Can anyone shed light on fixing this problem or getting around it > so the store can be fixed? > > Thanks for your assistance > > Paul > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Sun Nov 19 14:56:51 2006 From: boxbackup at fluffy.co.uk (Paul MacKenzie) Date: Sun, 19 Nov 2006 09:56:51 -0500 Subject: [Box Backup] Box Backup check fix Segmentation fault In-Reply-To: References: <7.0.1.0.2.20061118204118.034077b0@elehost.com> Message-ID: <7.0.1.0.2.20061119095301.03ff3390@elehost.com> Hi Ben Thanks for your response Here is the output I think you are looking for Account ID: 12345SB Last object ID: 4968730 Blocks used: 15629037 (30525.46Mb) Blocks used by old files: 5356308 (10461.54Mb) Blocks used by deleted files: 1231149 (2404.59Mb) Blocks used by directories: 141788 (276.93Mb) Block soft limit: 15360000 (30000.00Mb) Block hard limit: 25600000 (50000.00Mb) Client store marker: 1163917214000000 This account is only about 6 months old. It has always been with the newest version of box backup on the server. I ran it again and the same result came up. I am not sure of the number of files but there are probably a lot of little files that are old / deleted. Oddly I have another server with definitely more files that seems to check fine so far. I was noticing the system was not backing up so I was taking this fix step in hopes that the fix would allow it to start backing up again. Any ideas on how this can be fixed or do I have to wipe it and recreate it? Thanks for you help, Paul >What's the output of bbstoreaccounts info for that account? Can you >make a guess at the number of files backed up in that account? (or at >least is that number less than 4.9 million your output below might >suggest) How old is the account? From boxbackup at fluffy.co.uk Sun Nov 19 23:36:09 2006 From: boxbackup at fluffy.co.uk (Paul MacKenzie) Date: Sun, 19 Nov 2006 18:36:09 -0500 Subject: [Box Backup] Box Backup Connection Protocol_ObjTooBig Message-ID: <7.0.1.0.2.20061119182955.04034120@elehost.com> Hello again, It is very odd as this system has been running flawlessly for quite come time I am stuck on another error on top of the others. Thus is for a store that checks fine on the backup server but does not seem to be getitng backed up. On the server that have the files I have a ton of these errors. Nov 19 18:21:21 server bbackupd[48546]: Error code when uploading was (4/55), BackupStore CannotDiffAnIncompleteStoreFile Nov 19 18:21:21 server bbackupd[48546]: Exception caught (Connection Protocol_ObjTooBig 7/42), reset state and waiting to retry. Nov 19 18:24:14 server bbackupd[48546]: Backup object failed, error when reading /var/db/pkg/php4-4.4.4_1/+CONTENTS Nov 19 18:24:14 server bbackupd[48546]: Error code when uploading was (4/55), BackupStore CannotDiffAnIncompleteStoreFile Nov 19 18:24:14 server bbackupd[48546]: Exception caught (Connection Protocol_ObjTooBig 7/42), reset state and waiting to retry. Nov 19 18:25:13 server bbackupd[48546]: Backup object failed, error when reading /var/db/pkg/php4-4.4.4_1/+CONTENTS Nov 19 18:25:13 server bbackupd[48546]: Error code when uploading was (4/55), BackupStore CannotDiffAnIncompleteStoreFile Nov 19 18:25:13 server bbackupd[48546]: Exception caught (Connection Protocol_ObjTooBig 7/42), reset state and waiting to retry. Nov 19 18:27:52 server bbackupd[48546]: Exception caught (Connection SocketConnectError (Probably a network issue between client What is odd is the messages like this come over and over again every few minutes. I can read the file no problem and it is a very small file (14k) 14824 Nov 14 22:01 /var/db/pkg/php4-4.4.4_1/+CONTENTS Any help with this would be greatly appreciated. This is going from one unix system to another with the same version of boxbackup (the latest) Thanks Paul From boxbackup at fluffy.co.uk Mon Nov 20 15:08:16 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Mon, 20 Nov 2006 15:08:16 +0000 Subject: [Box Backup] Box Backup check fix Segmentation fault In-Reply-To: <7.0.1.0.2.20061119095301.03ff3390@elehost.com> References: <7.0.1.0.2.20061118204118.034077b0@elehost.com> <7.0.1.0.2.20061119095301.03ff3390@elehost.com> Message-ID: On 19 Nov 2006, at 14:56, Paul MacKenzie wrote: > Hi Ben > > Thanks for your response > > Here is the output I think you are looking for > > Account ID: 12345SB > Last object ID: 4968730 > Blocks used: 15629037 (30525.46Mb) > Blocks used by old files: 5356308 (10461.54Mb) > Blocks used by deleted files: 1231149 (2404.59Mb) > Blocks used by directories: 141788 (276.93Mb) > Block soft limit: 15360000 (30000.00Mb) > Block hard limit: 25600000 (50000.00Mb) > Client store marker: 1163917214000000 > > This account is only about 6 months old. It has always been with > the newest version of box backup on the server. > > I ran it again and the same result came up. I am not sure of the > number of files but there are probably a lot of little files that > are old / deleted. Oddly I have another server with definitely more > files that seems to check fine so far. I was noticing the system > was not backing up so I was taking this fix step in hopes that the > fix would allow it to start backing up again. > > Any ideas on how this can be fixed or do I have to wipe it and > recreate it? Try stopping bbackupd (client) and deleting the contents of /var/ bbackupd, then restart it. I wonder if bbstoreaccounts check fix is running out of memory -- it's going to need about 16 bytes for every one of those 4968730 object IDs. But it shouldn't just go bang. Ben From boxbackup at fluffy.co.uk Mon Nov 20 21:01:07 2006 From: boxbackup at fluffy.co.uk (Remco Poelstra) Date: Mon, 20 Nov 2006 22:01:07 +0100 Subject: [Box Backup] Start on boot on Mac OSX Message-ID: <5122DDD4-064B-4D0D-B23C-C35BC93A434B@beryllium.net> Hi, I've manged to get bbackupd running on my Mac. I was wondering: what is the best way to start bbackupd on boot? Thanks in advance, Remco Poelstra From boxbackup at fluffy.co.uk Tue Nov 21 08:44:09 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 21 Nov 2006 08:44:09 +0000 Subject: [Box Backup] Start on boot on Mac OSX In-Reply-To: <5122DDD4-064B-4D0D-B23C-C35BC93A434B@beryllium.net> References: <5122DDD4-064B-4D0D-B23C-C35BC93A434B@beryllium.net> Message-ID: On 20 Nov 2006, at 21:01, Remco Poelstra wrote: > Hi, > > I've manged to get bbackupd running on my Mac. I was wondering: > what is the best way to start bbackupd on boot? Use launchd -- works well. An example plist file below. I have it in ~/Library/LaunchAgents to start it when I log in. Modify accordingly to work as a boot launch item. Note the SINGLEPROCESS argument. This allows launchd to manage it properly, since such processes should not fork and daemonise. Ben Label uk.co.fluffy.BoxBackup OnDemand ProgramArguments /Users/ben/boxbackup/bin/bbackupd /Users/ben/boxbackup/config/bbackupd.conf SINGLEPROCESS From boxbackup at fluffy.co.uk Tue Nov 21 11:08:13 2006 From: boxbackup at fluffy.co.uk (Remco Poelstra) Date: Tue, 21 Nov 2006 12:08:13 +0100 Subject: [Box Backup] Start on boot on Mac OSX In-Reply-To: References: <5122DDD4-064B-4D0D-B23C-C35BC93A434B@beryllium.net> Message-ID: Thanks! I'll look into this. Remco Poelstra Op 21-nov-2006, om 9:44 heeft Ben Summers het volgende geschreven: > > On 20 Nov 2006, at 21:01, Remco Poelstra wrote: > >> Hi, >> >> I've manged to get bbackupd running on my Mac. I was wondering: >> what is the best way to start bbackupd on boot? > > Use launchd -- works well. An example plist file below. I have it > in ~/Library/LaunchAgents to start it when I log in. Modify > accordingly to work as a boot launch item. > > Note the SINGLEPROCESS argument. This allows launchd to manage it > properly, since such processes should not fork and daemonise. > > Ben > > > > > "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> > > > Label > uk.co.fluffy.BoxBackup > OnDemand > > ProgramArguments > > /Users/ben/boxbackup/bin/bbackupd > /Users/ben/boxbackup/config/bbackupd.conf > SINGLEPROCESS > > > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Nov 21 17:58:59 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 21 Nov 2006 17:58:59 +0000 Subject: [Box Backup] Wiki spambots Message-ID: Spambots have trashed the wiki, and it needs a really careful clean. Anyone know of any good defenses, or will we just have to manually approve accounts for editing, thus making the wiki thing kind of pointless? Ben From boxbackup at fluffy.co.uk Tue Nov 21 18:19:51 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Tue, 21 Nov 2006 18:19:51 -0000 Subject: [Box Backup] Wiki spambots In-Reply-To: Message-ID: <000a01c70d99$a30b69a0$5758ea9e@groupinfra.com> I don't, I'm afraid, but I wonder whether this is the right time to migrate the contents to the Trac wiki? Trac 0.10 is supposed to have some anti-spam support built-in that uses the same techniques as WordPress (which seems to be quite effective in my experience). I'm not sure about mediawiki, but with Trac you can restrict updates to registered users whilst allowing anonymous users to create their own accounts. That, coupled with its Akismet spam detection, might turn the tables for a while at least. I wouldn't mind helping with the migration if that was the consensus as I've quite a bit of experience with Trac myself. I never really understood the rationale behind keeping both (although it's probably on a wiki page I've just not read!). Stuart -----Original Message----- From: boxbackup-admin at fluffy.co.uk [mailto:boxbackup-admin at fluffy.co.uk] On Behalf Of Ben Summers Sent: 21 November 2006 17:59 To: boxbackup at fluffy.co.uk Subject: [Box Backup] Wiki spambots Spambots have trashed the wiki, and it needs a really careful clean. Anyone know of any good defenses, or will we just have to manually approve accounts for editing, thus making the wiki thing kind of pointless? Ben _______________________________________________ boxbackup mailing list boxbackup at fluffy.co.uk http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Nov 21 18:09:19 2006 From: boxbackup at fluffy.co.uk (=?ISO-8859-1?Q?Simon_Lundstr=F6m?=) Date: Tue, 21 Nov 2006 19:09:19 +0100 Subject: [Box Backup] Wiki spambots In-Reply-To: References: Message-ID: <4C9F7D7C-AAA5-4F15-8714-A59838FC439F@gmail.com> --Apple-Mail-34--1005716944 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed CAPTCHA, http://en.wikipedia.org/wiki/CAPTCHA , seems to be the solution to spambots. This should work for mediawiki http:// wiki.seds.org/index.php/MediaWiki:Captcha - Simon On Nov 21, 2006, at 6:58 PM, Ben Summers wrote: > > Spambots have trashed the wiki, and it needs a really careful > clean. Anyone know of any good defenses, or will we just have to > manually approve accounts for editing, thus making the wiki thing > kind of pointless? > > Ben > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup --Apple-Mail-34--1005716944 Content-Transfer-Encoding: base64 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGMTCCAuow ggJToAMCAQICEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDgyNTE2NDc1OVoXDTA3MDgyNTE2NDc1 OVowSjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEnMCUGCSqGSIb3DQEJARYYc2lt b25sdW5kc3Ryb21AZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArrb+ /12bXTb5oixk4K2QhcJtsN6o3bu3p/fJK9xTP4gtF7NpnOzTrnRbmGGQ2HJEgV3bVuGOIaR1qqyk toR5RzxM3QXkepq0KUxdpCJgkDu/GlG/z7It1Wh5y+J5suPhVyWjr3n2RYJ5J+nssP/pw1EiLV3g OH7imu9uUPCziK+J2glMXwFNE/pA+xNIPAflNFpdaexIYAbKnm/N119usWLZlzCoIuj9yTSSV0wG /SpPN5UJtA0O0zxD6GMLFfm0zFc9e0sAvynaVIfVDmSEvY3FzrfGg/fZiCWFHRcc/Q23Olw/OxDw ioSo1DfQU2e2XzBiX5bX9ApUbQ+RFZRGHQIDAQABozUwMzAjBgNVHREEHDAagRhzaW1vbmx1bmRz dHJvbUBnbWFpbC5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOBgQAmV6ABcDdunTJ8 5SkJl6oEa0lIh8+dfXseipg4bMDCRiJahR5RvpGWFnNbDN66NxYFvIasAy9ss4VOsZ2Eq3FQkQNA 4UL4rqjJDF/XKGfRYEZpF55ZztDm54vUMLpXtkjZUCCzSQp+1X04d3Vjz/1r2PtX3MxnBnZdfQuk LvkJFjCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAwgdExCzAJBgNVBAYTAlpBMRUwEwYD VQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEaMBgGA1UEChMRVGhhd3RlIENv bnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRpb24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNV BAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwt ZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0wMzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJ BgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQD EyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEAxKY8VXNV+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDX AmNaLIkVcI7dyfArhVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l 0cUqVIUPSAR/p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMG A1UdHwQ8MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVt YWlsQ0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/TCG4 +DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amcOY6MIE9l X5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggMQMIIDDAIBATB2MGIx CzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYD VQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQQStBdrLuqCK7MX8wXDle 5TAJBgUrDgMCGgUAoIIBbzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0wNjExMjExODA5MjBaMCMGCSqGSIb3DQEJBDEWBBTJOPv8ws/+Rs2h00qmgCim+y4XqTCBhQYJ KwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC EEErQXay7qgiuzF/MFw5XuUwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMG A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEErQXay7qgiuzF/MFw5XuUwDQYJKoZIhvcNAQEBBQAE ggEAd/ofWeVsg6L9L6FheaYVbpMC1KvkCyz+sCx5VSNQ8Zw/Yul526r2+BLmupNALyUx7hz0IBNh RUYuRKObZk3Hw41xgwm9PsxmEZCOin0nnG7IAoivrufMM6u+UGmPbonyy5/PRgTcToCl6pCQCifn ylJdgClDINYgTc0N+pc+vIUYStc3Se5svJHPKlG89BrjJxTOTZiiCLO60MNDpwjS2OHtiiWWUE6q BB6pcUAxF692xVe0Ql98b1OwvpMvIur14KxphkBVCReWNEpFRJN7pxa8PBQ1CYH5QfQvkIXM6Sep RYxvVrrhAyJtZlXJHLmmxmyfVZxpJH/ZYbsKcPUKTwAAAAAAAA== --Apple-Mail-34--1005716944-- From boxbackup at fluffy.co.uk Tue Nov 21 18:10:48 2006 From: boxbackup at fluffy.co.uk (Filip Hlavenka) Date: Tue, 21 Nov 2006 19:10:48 +0100 Subject: [Box Backup] Wiki spambots In-Reply-To: References: Message-ID: <200611211910.48702.filip.hlavenka@nasieti.sk> You have to employ some kind of a little advanced captcha or approve accounts manualy. No other way to get rid of spammers On Tuesday 21 November 2006 18:58, Ben Summers wrote: > Spambots have trashed the wiki, and it needs a really careful clean. > Anyone know of any good defenses, or will we just have to manually > approve accounts for editing, thus making the wiki thing kind of > pointless? > > Ben > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Tue Nov 21 22:36:33 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Tue, 21 Nov 2006 22:36:33 +0000 Subject: [Box Backup] Wiki spambots In-Reply-To: <000a01c70d99$a30b69a0$5758ea9e@groupinfra.com> References: <000a01c70d99$a30b69a0$5758ea9e@groupinfra.com> Message-ID: <1164148593.3503.8.camel@avenin.ebourne.me.uk> On Tue, 2006-11-21 at 18:19 +0000, Stuart Hickinbottom wrote: > I don't, I'm afraid, but I wonder whether this is the right time to migrate > the contents to the Trac wiki? Trac 0.10 is supposed to have some anti-spam > support built-in that uses the same techniques as WordPress (which seems to > be quite effective in my experience). > > I'm not sure about mediawiki, but with Trac you can restrict updates to > registered users whilst allowing anonymous users to create their own > accounts. That, coupled with its Akismet spam detection, might turn the > tables for a while at least. > > I wouldn't mind helping with the migration if that was the consensus as I've > quite a bit of experience with Trac myself. > > I never really understood the rationale behind keeping both (although it's > probably on a wiki page I've just not read!). No, simply that the media wiki site was first, and trac only came after. A combination of "if it's not broke don't fix it", and the effort cost was quite sufficient to keep the status quo. I certainly think it would be a good idea to move the content from the old wiki to the trac wiki, don't see any advantage of having separate ones. Maybe mediawiki is slightly prettier, but the trac one's perfectly acceptable, if it helps protect against spam will be well worth it, and at the very least will allow much easier linking to the trac tickets/svn checkins etc. And one less account login as well of course. If you're volunteering to get started that seems perfect! Cheers, Martin. PS. As to the so-called captchas, I seem only to have about an 80% success rate with those. Sometimes they seem completely indecipherable. Maybe I'd fail the turing test too. ;) I personally don't really like most of those systems because they discriminate against people with a whole range of disabilities. From boxbackup at fluffy.co.uk Wed Nov 22 08:14:44 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Wed, 22 Nov 2006 08:14:44 -0000 Subject: [Box Backup] Wiki spambots In-Reply-To: <1164148593.3503.8.camel@avenin.ebourne.me.uk> Message-ID: <002101c70e0e$44bd82f0$5758ea9e@groupinfra.com> I'm quite happy to get cracking (I've some time this weekend when I can probably get a good deal, if not all, of it done), but before I do I'll wait a couple of days to make sure that is what everyone wants. In particular, I'll wait for Ben to confirm that's OK (or not) as I think the right thing to do would be to also include more of the content of Ben's original pages as I don't think all of that made it into the wiki. I can clean up the spam as I go. I can't remember who was administering the Trac system but it would be good for them to let me know their email address (to 'stuart-at-hickinbottom-dot-demon-dot-co-dot-uk) as there'll probably be a couple of things I'll want to ask about along the way (eg making sure the spam protection is turned on, and there might be a couple of Trac plugins to add). Finally, for it to be worth it we'll need to get the existing wiki cleared down once the Trac one is up and running - I can edit the mediawiki to direct users to the new one, but it'll take the administrator of that to close it down fully. Stuart -----Original Message----- From: boxbackup-admin at fluffy.co.uk [mailto:boxbackup-admin at fluffy.co.uk] On Behalf Of Martin Ebourne Sent: 21 November 2006 22:37 To: boxbackup at fluffy.co.uk Subject: RE: [Box Backup] Wiki spambots On Tue, 2006-11-21 at 18:19 +0000, Stuart Hickinbottom wrote: > I don't, I'm afraid, but I wonder whether this is the right time to migrate > the contents to the Trac wiki? Trac 0.10 is supposed to have some anti-spam > support built-in that uses the same techniques as WordPress (which seems to > be quite effective in my experience). > > I'm not sure about mediawiki, but with Trac you can restrict updates to > registered users whilst allowing anonymous users to create their own > accounts. That, coupled with its Akismet spam detection, might turn the > tables for a while at least. > > I wouldn't mind helping with the migration if that was the consensus as I've > quite a bit of experience with Trac myself. > > I never really understood the rationale behind keeping both (although it's > probably on a wiki page I've just not read!). No, simply that the media wiki site was first, and trac only came after. A combination of "if it's not broke don't fix it", and the effort cost was quite sufficient to keep the status quo. I certainly think it would be a good idea to move the content from the old wiki to the trac wiki, don't see any advantage of having separate ones. Maybe mediawiki is slightly prettier, but the trac one's perfectly acceptable, if it helps protect against spam will be well worth it, and at the very least will allow much easier linking to the trac tickets/svn checkins etc. And one less account login as well of course. If you're volunteering to get started that seems perfect! Cheers, Martin. PS. As to the so-called captchas, I seem only to have about an 80% success rate with those. Sometimes they seem completely indecipherable. Maybe I'd fail the turing test too. ;) I personally don't really like most of those systems because they discriminate against people with a whole range of disabilities. _______________________________________________ boxbackup mailing list boxbackup at fluffy.co.uk http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Nov 22 08:14:56 2006 From: boxbackup at fluffy.co.uk (Imran Niazi) Date: Wed, 22 Nov 2006 00:14:56 -0800 Subject: [Box Backup] FYI: If you lose your CA dir Message-ID: <20061122072108.M53899@naweb.com> Hi guys, I was just installing the client on two new servers, and noticed that I "lost" the 'ca' directory. This assumes you still have the server key file and the csr file etc. in your configuration dirs, as well as the csr/key files for each of the clients. If you lose the server key file, you should be able to recreate it and follow the below instructions. I don't think that should affect your old backups. But if you lose the client key file, you will lose all the backups for that client. Please correct me if I'm wrong. Anyway, this what you do to generate new ca dir etc.: a) create a new ca dir: 'bbstored-certs ca init' b) Sign the server key: bbstored-certs ca sign-server SERVERHOSTNAME-csr.pem c) copy the cert etc. as instructed d) Copy the CLIENTID-csr.pem from each of the clients to the server e) Sign each of the csr and generate new CLIENTID-cert.pem: bbstored-certs ca sign CLIENTID-csr.pem f) Copy the CLIENTID-cert.pem, and serverCA.pem, back to the client (the util tells you what to copy back) Imran Niazi www.niazi.net From boxbackup at fluffy.co.uk Wed Nov 22 08:59:31 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 22 Nov 2006 08:59:31 +0000 Subject: [Box Backup] FYI: If you lose your CA dir In-Reply-To: <20061122072108.M53899@naweb.com> References: <20061122072108.M53899@naweb.com> Message-ID: <6BEE3E3C-55A3-41CA-8311-19F0EF2257CF@fluffy.co.uk> To confirm: You can replace everything except the keys.raw file on the client, and still have access to your account. The certificates are only used to authenticate the client to the server, and the server to the client. So they can be replaced throughout a system without affecting the data. The keys.raw file for the client is used to encrypt the data. Obviously you cannot recreate this if you lose it. Guard it carefully. Ben On 22 Nov 2006, at 08:14, Imran Niazi wrote: > Hi guys, > > I was just installing the client on two new servers, and noticed > that I "lost" > the 'ca' directory. This assumes you still have the server key > file and the > csr file etc. in your configuration dirs, as well as the csr/key > files for > each of the clients. If you lose the server key file, you should > be able to > recreate it and follow the below instructions. I don't think that > should > affect your old backups. But if you lose the client key file, you > will lose > all the backups for that client. Please correct me if I'm wrong. > > Anyway, this what you do to generate new ca dir etc.: > > a) create a new ca dir: 'bbstored-certs ca init' > b) Sign the server key: > bbstored-certs ca sign-server SERVERHOSTNAME-csr.pem > c) copy the cert etc. as instructed > d) Copy the CLIENTID-csr.pem from each of the clients to the server > e) Sign each of the csr and generate new CLIENTID-cert.pem: > bbstored-certs ca sign CLIENTID-csr.pem > f) Copy the CLIENTID-cert.pem, and serverCA.pem, back to the client > (the util > tells you what to copy back) > > > Imran Niazi > www.niazi.net > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Nov 22 09:07:06 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 22 Nov 2006 09:07:06 +0000 Subject: [Box Backup] Wiki spambots In-Reply-To: <002101c70e0e$44bd82f0$5758ea9e@groupinfra.com> References: <002101c70e0e$44bd82f0$5758ea9e@groupinfra.com> Message-ID: <54733D8D-3327-43D1-A03D-7BB1FA61D640@fluffy.co.uk> I think moving the wiki content to Trac sounds like a good plan. We'll get some useful extra features, and have one less thing to defend. Thanks for volunteering to do the hard work. The content on the wiki is a bit trashed. We did turn off the ability for anonymous users to edit pages, so the bots just register accounts. Annoyingly, they remove content as well as appending stuff, so you have to go quite a way back in the history to reconstruct a good copy. My thanks to Stephen Mosher and Hostworks for the current wiki. Ben PS: There's no need to get my approval for everything. We have a strong community of sensible people, and I can't imagine why I would disagree with any consensus. On 22 Nov 2006, at 08:14, Stuart Hickinbottom wrote: > I'm quite happy to get cracking (I've some time this weekend when I > can > probably get a good deal, if not all, of it done), but before I do > I'll wait > a couple of days to make sure that is what everyone wants. In > particular, > I'll wait for Ben to confirm that's OK (or not) as I think the > right thing > to do would be to also include more of the content of Ben's > original pages > as I don't think all of that made it into the wiki. > > I can clean up the spam as I go. > > I can't remember who was administering the Trac system but it would > be good > for them to let me know their email address (to > 'stuart-at-hickinbottom-dot-demon-dot-co-dot-uk) as there'll > probably be a > couple of things I'll want to ask about along the way (eg making > sure the > spam protection is turned on, and there might be a couple of Trac > plugins to > add). > > Finally, for it to be worth it we'll need to get the existing wiki > cleared > down once the Trac one is up and running - I can edit the mediawiki to > direct users to the new one, but it'll take the administrator of > that to > close it down fully. > > Stuart > > -----Original Message----- > From: boxbackup-admin at fluffy.co.uk [mailto:boxbackup- > admin at fluffy.co.uk] On > Behalf Of Martin Ebourne > Sent: 21 November 2006 22:37 > To: boxbackup at fluffy.co.uk > Subject: RE: [Box Backup] Wiki spambots > > On Tue, 2006-11-21 at 18:19 +0000, Stuart Hickinbottom wrote: >> I don't, I'm afraid, but I wonder whether this is the right time to > migrate >> the contents to the Trac wiki? Trac 0.10 is supposed to have some > anti-spam >> support built-in that uses the same techniques as WordPress (which >> seems > to >> be quite effective in my experience). >> >> I'm not sure about mediawiki, but with Trac you can restrict >> updates to >> registered users whilst allowing anonymous users to create their own >> accounts. That, coupled with its Akismet spam detection, might >> turn the >> tables for a while at least. >> >> I wouldn't mind helping with the migration if that was the >> consensus as > I've >> quite a bit of experience with Trac myself. >> >> I never really understood the rationale behind keeping both >> (although it's >> probably on a wiki page I've just not read!). > > No, simply that the media wiki site was first, and trac only came > after. > A combination of "if it's not broke don't fix it", and the effort cost > was quite sufficient to keep the status quo. > > I certainly think it would be a good idea to move the content from the > old wiki to the trac wiki, don't see any advantage of having separate > ones. Maybe mediawiki is slightly prettier, but the trac one's > perfectly > acceptable, if it helps protect against spam will be well worth it, > and > at the very least will allow much easier linking to the trac > tickets/svn > checkins etc. And one less account login as well of course. > > If you're volunteering to get started that seems perfect! > > Cheers, > > Martin. > > PS. As to the so-called captchas, I seem only to have about an 80% > success rate with those. Sometimes they seem completely > indecipherable. > Maybe I'd fail the turing test too. ;) I personally don't really like > most of those systems because they discriminate against people with a > whole range of disabilities. > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Nov 22 09:24:00 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Wed, 22 Nov 2006 09:24:00 -0000 Subject: [Box Backup] Wiki spambots In-Reply-To: <54733D8D-3327-43D1-A03D-7BB1FA61D640@fluffy.co.uk> Message-ID: <002501c70e17$f1d42f80$5758ea9e@groupinfra.com> > PS: There's no need to get my approval for everything. We have a > strong community of sensible people, and I can't imagine why I would > disagree with any consensus. I'm going to save that to quote in the future! OK - I'll make a start and see how I get on. Stuart -----Original Message----- From: boxbackup-admin at fluffy.co.uk [mailto:boxbackup-admin at fluffy.co.uk] On Behalf Of Ben Summers Sent: 22 November 2006 09:07 To: boxbackup at fluffy.co.uk Subject: Re: [Box Backup] Wiki spambots I think moving the wiki content to Trac sounds like a good plan. We'll get some useful extra features, and have one less thing to defend. Thanks for volunteering to do the hard work. The content on the wiki is a bit trashed. We did turn off the ability for anonymous users to edit pages, so the bots just register accounts. Annoyingly, they remove content as well as appending stuff, so you have to go quite a way back in the history to reconstruct a good copy. My thanks to Stephen Mosher and Hostworks for the current wiki. Ben PS: There's no need to get my approval for everything. We have a strong community of sensible people, and I can't imagine why I would disagree with any consensus. On 22 Nov 2006, at 08:14, Stuart Hickinbottom wrote: > I'm quite happy to get cracking (I've some time this weekend when I > can > probably get a good deal, if not all, of it done), but before I do > I'll wait > a couple of days to make sure that is what everyone wants. In > particular, > I'll wait for Ben to confirm that's OK (or not) as I think the > right thing > to do would be to also include more of the content of Ben's > original pages > as I don't think all of that made it into the wiki. > > I can clean up the spam as I go. > > I can't remember who was administering the Trac system but it would > be good > for them to let me know their email address (to > 'stuart-at-hickinbottom-dot-demon-dot-co-dot-uk) as there'll > probably be a > couple of things I'll want to ask about along the way (eg making > sure the > spam protection is turned on, and there might be a couple of Trac > plugins to > add). > > Finally, for it to be worth it we'll need to get the existing wiki > cleared > down once the Trac one is up and running - I can edit the mediawiki to > direct users to the new one, but it'll take the administrator of > that to > close it down fully. > > Stuart > > -----Original Message----- > From: boxbackup-admin at fluffy.co.uk [mailto:boxbackup- > admin at fluffy.co.uk] On > Behalf Of Martin Ebourne > Sent: 21 November 2006 22:37 > To: boxbackup at fluffy.co.uk > Subject: RE: [Box Backup] Wiki spambots > > On Tue, 2006-11-21 at 18:19 +0000, Stuart Hickinbottom wrote: >> I don't, I'm afraid, but I wonder whether this is the right time to > migrate >> the contents to the Trac wiki? Trac 0.10 is supposed to have some > anti-spam >> support built-in that uses the same techniques as WordPress (which >> seems > to >> be quite effective in my experience). >> >> I'm not sure about mediawiki, but with Trac you can restrict >> updates to >> registered users whilst allowing anonymous users to create their own >> accounts. That, coupled with its Akismet spam detection, might >> turn the >> tables for a while at least. >> >> I wouldn't mind helping with the migration if that was the >> consensus as > I've >> quite a bit of experience with Trac myself. >> >> I never really understood the rationale behind keeping both >> (although it's >> probably on a wiki page I've just not read!). > > No, simply that the media wiki site was first, and trac only came > after. > A combination of "if it's not broke don't fix it", and the effort cost > was quite sufficient to keep the status quo. > > I certainly think it would be a good idea to move the content from the > old wiki to the trac wiki, don't see any advantage of having separate > ones. Maybe mediawiki is slightly prettier, but the trac one's > perfectly > acceptable, if it helps protect against spam will be well worth it, > and > at the very least will allow much easier linking to the trac > tickets/svn > checkins etc. And one less account login as well of course. > > If you're volunteering to get started that seems perfect! > > Cheers, > > Martin. > > PS. As to the so-called captchas, I seem only to have about an 80% > success rate with those. Sometimes they seem completely > indecipherable. > Maybe I'd fail the turing test too. ;) I personally don't really like > most of those systems because they discriminate against people with a > whole range of disabilities. > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup _______________________________________________ boxbackup mailing list boxbackup at fluffy.co.uk http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Sat Nov 25 16:57:17 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Sat, 25 Nov 2006 16:57:17 -0000 Subject: [Box Backup] Wiki migration Message-ID: <001301c710b2$c3ba5460$5258ea9e@groupinfra.com> I think I've now managed to complete that (thanks to James for his help along the way). For reference the wiki main page is here: http://bbdev.fluffy.co.uk/trac/wiki I've made the old wiki point to this new one on the main page (the old main page is still available there if anyone needs it). Along the way I've done quite a bit of spring cleaning of the pages, plus a small about of restructuring; hopefully that will make the wiki easier to navigate for users in particular. I think there is room for improvement, though. Enhancements and known problems should be tracked through tickets rather than on wiki pages, and more use should be made of milestones. This allows a problem/idea history to be properly tracked through raising, discussion, implementation and release. Creating a lot more tickets won't necessarily create more tickets than can be easily understood since they're only tabled for a release when a developer picks one of them up and is assigned it. I didn't want to move down that road unilaterally since a) it hasn't been discussed and b) I'm not one of the developers so I'm the wrong person to decide that how the development should be done! So, just an idea for discussion, really, but that's how I've used Trac in the past. Anyway, shout if you think anything is missing or wrong in the wiki if you don't want to look at it yourself and I'll try to take a look. Stuart From boxbackup at fluffy.co.uk Sat Nov 25 23:32:26 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Sat, 25 Nov 2006 23:32:26 +0000 Subject: [Box Backup] Wiki migration In-Reply-To: <001301c710b2$c3ba5460$5258ea9e@groupinfra.com> References: <001301c710b2$c3ba5460$5258ea9e@groupinfra.com> Message-ID: <1164497547.12061.14.camel@avenin.ebourne.me.uk> On Sat, 2006-11-25 at 16:57 +0000, Stuart Hickinbottom wrote: > I think I've now managed to complete that (thanks to James for his help > along the way). For reference the wiki main page is here: > > http://bbdev.fluffy.co.uk/trac/wiki > > I've made the old wiki point to this new one on the main page (the old main > page is still available there if anyone needs it). > > Along the way I've done quite a bit of spring cleaning of the pages, plus a > small about of restructuring; hopefully that will make the wiki easier to > navigate for users in particular. Fantastic work Stuart, thank you very much! I've had a good look through and I think that is a big improvement. A minor comment is that I feel the front page should be a little simpler, though I didn't have any good ideas on how to do that and it's certainly not excessive as it is. > I think there is room for improvement, though. Enhancements and known > problems should be tracked through tickets rather than on wiki pages, and > more use should be made of milestones. This allows a problem/idea history to > be properly tracked through raising, discussion, implementation and release. > Creating a lot more tickets won't necessarily create more tickets than can > be easily understood since they're only tabled for a release when a > developer picks one of them up and is assigned it. Problems and planned enhancements are being tracked through tickets, it's just that trac has only recently been available for Box, so there's a lot of stuff that was in the wiki in the interim. Certainly any bugs mentioned in wiki pages should be moved to trac tickets (unless already resolved). Check on the dev mailing list if not sure. As for creating tickets for enhancements, the plan is only create the ticket where there is a genuine intent to add the feature (or if it has already been developed and a patch is available). This pretty much means that enhancement tickets can be created by anyone, provided they include a patch to implement it. Planned but unimplemented features should be created as task tickets and only developers should be doing this. The reasoning is that otherwise a lot of useless tickets end up getting created for random feature requests that will never get implemented, either because they don't fit in with the goals of the project, or just because no-one will ever spend the time on them. When this happens it doesn't really benefit anyone. Cheers, Martin. From boxbackup at fluffy.co.uk Wed Nov 29 16:49:34 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 29 Nov 2006 17:49:34 +0100 Subject: [Box Backup] Common OSFileError - What file ? Message-ID: <456DBA1E.4090905@kontrapunkt.com> Hello, My backup run ends as seen below from my client log. Extended logging is enabled. How do I find the file causing the error? Log from client: Nov 29 04:58:33 yoiko bbackupd[3676]: Send ListDirectory(0x6ddf4,0xffffffff,0xc,true) Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 Nov 29 04:58:48 yoiko bbackupd[3676]: Send SetClientStoreMarker(0x4235ac8b24e00) Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common OSFileError (Error accessing a file. Check permissions.) 1/9), reset state and waiting to retry... Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file size uploaded 80626331005, bytes already on server 827066486, encoded size 55033653560 Thanks, Tobias From boxbackup at fluffy.co.uk Wed Nov 29 16:53:41 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 29 Nov 2006 16:53:41 +0000 Subject: [Box Backup] Common OSFileError - What file ? In-Reply-To: <456DBA1E.4090905@kontrapunkt.com> References: <456DBA1E.4090905@kontrapunkt.com> Message-ID: <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> The easiest way is to restart the daemon. (If you're using the store info file, delete that while the daemon is stopped.) Then the logs should give enough filenames to track it down. The most likely cause is that you're running bbackupd as non-root, and it's tried to read a file but permissions denied access. Ben On 29 Nov 2006, at 16:49, Tobias Balle-Petersen wrote: > Hello, > > My backup run ends as seen below from my client log. Extended > logging is > enabled. How do I find the file causing the error? > > Log from client: > Nov 29 04:58:33 yoiko bbackupd[3676]: Send > ListDirectory(0x6ddf4,0xffffffff,0xc,true) > Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) > Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 > Nov 29 04:58:48 yoiko bbackupd[3676]: Send > SetClientStoreMarker(0x4235ac8b24e00) > Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) > Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() > Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() > Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common > OSFileError (Error accessing a file. Check permissions.) 1/9), reset > state and waiting to retry... > Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file size > uploaded 80626331005, bytes already on server 827066486, encoded size > 55033653560 > > > Thanks, > Tobias > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Nov 29 19:41:49 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 29 Nov 2006 20:41:49 +0100 Subject: [Box Backup] Common OSFileError - What file ? In-Reply-To: <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> References: <456DBA1E.4090905@kontrapunkt.com> <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> Message-ID: <456DE27D.909@kontrapunkt.com> Hello, bbackupd is running as root. bbackupd is realoded prior to every run. Tobias Ben Summers wrote: > > The easiest way is to restart the daemon. (If you're using the store > info file, delete that while the daemon is stopped.) Then the logs > should give enough filenames to track it down. > > The most likely cause is that you're running bbackupd as non-root, and > it's tried to read a file but permissions denied access. > > Ben > > > On 29 Nov 2006, at 16:49, Tobias Balle-Petersen wrote: > >> Hello, >> >> My backup run ends as seen below from my client log. Extended logging is >> enabled. How do I find the file causing the error? >> >> Log from client: >> Nov 29 04:58:33 yoiko bbackupd[3676]: Send >> ListDirectory(0x6ddf4,0xffffffff,0xc,true) >> Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) >> Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 >> Nov 29 04:58:48 yoiko bbackupd[3676]: Send >> SetClientStoreMarker(0x4235ac8b24e00) >> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) >> Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() >> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() >> Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common >> OSFileError (Error accessing a file. Check permissions.) 1/9), reset >> state and waiting to retry... >> Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file size >> uploaded 80626331005, bytes already on server 827066486, encoded size >> 55033653560 >> >> >> Thanks, >> Tobias >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed Nov 29 19:52:33 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 29 Nov 2006 19:52:33 +0000 Subject: [Box Backup] Common OSFileError - What file ? In-Reply-To: <456DE27D.909@kontrapunkt.com> References: <456DBA1E.4090905@kontrapunkt.com> <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> <456DE27D.909@kontrapunkt.com> Message-ID: What's the last file or directory name listed in the logs then? It must mention something. Ben On 29 Nov 2006, at 19:41, Tobias Balle-Petersen wrote: > Hello, > > bbackupd is running as root. > bbackupd is realoded prior to every run. > > Tobias > > > Ben Summers wrote: >> >> The easiest way is to restart the daemon. (If you're using the store >> info file, delete that while the daemon is stopped.) Then the logs >> should give enough filenames to track it down. >> >> The most likely cause is that you're running bbackupd as non-root, >> and >> it's tried to read a file but permissions denied access. >> >> Ben >> >> >> On 29 Nov 2006, at 16:49, Tobias Balle-Petersen wrote: >> >>> Hello, >>> >>> My backup run ends as seen below from my client log. Extended >>> logging is >>> enabled. How do I find the file causing the error? >>> >>> Log from client: >>> Nov 29 04:58:33 yoiko bbackupd[3676]: Send >>> ListDirectory(0x6ddf4,0xffffffff,0xc,true) >>> Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) >>> Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 >>> Nov 29 04:58:48 yoiko bbackupd[3676]: Send >>> SetClientStoreMarker(0x4235ac8b24e00) >>> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success >>> (0x4235ac8b24e00) >>> Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() >>> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() >>> Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common >>> OSFileError (Error accessing a file. Check permissions.) 1/9), reset >>> state and waiting to retry... >>> Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file >>> size >>> uploaded 80626331005, bytes already on server 827066486, encoded >>> size >>> 55033653560 >>> >>> >>> Thanks, >>> Tobias >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >> > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Nov 29 22:17:30 2006 From: boxbackup at fluffy.co.uk (Tobias Balle-Petersen) Date: Wed, 29 Nov 2006 23:17:30 +0100 Subject: [Box Backup] Common OSFileError - What file ? In-Reply-To: References: <456DBA1E.4090905@kontrapunkt.com> <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> <456DE27D.909@kontrapunkt.com> Message-ID: <456E06FA.7090709@kontrapunkt.com> Hello, More from my log. ...... Nov 28 21:37:18 yoiko bbackupd[3676]: Sending stream, size 49 Nov 28 21:37:19 yoiko bbackupd[3676]: Receive Success(0xc631d) Nov 28 21:37:19 yoiko bbackupd[3676]: Send SetReplacementFileAttributes(0xc631d,0xd22a072450bcb931,"messe_annonce") Nov 28 21:37:19 yoiko bbackupd[3676]: Sending stream, size 113 Nov 28 21:37:19 yoiko bbackupd[3676]: Receive Success(0xc631e) Nov 28 21:37:19 yoiko bbackupd[3676]: Send SetReplacementFileAttributes(0xc631d,0xe9cb39477b29ab1e,"Sicef_messe_pr?stekst.doc") Nov 28 21:37:19 yoiko bbackupd[3676]: Sending stream, size 369 Nov 28 21:37:19 yoiko bbackupd[3676]: Receive Success(0xc631f) ........ Nov 29 02:14:32 yoiko bbackupd[3676]: Send ListDirectory(0x28fb4,0xffffffff,0xc,true) Nov 29 02:14:32 yoiko bbackupd[3676]: Receive Success(0x28fb4) Nov 29 02:14:32 yoiko bbackupd[3676]: Receiving stream, size 28634 Nov 29 02:14:32 yoiko bbackupd[3676]: Send StoreFile(0x28fb4,0x423541944b1c0,0xa4bd4258801a620e,0x0,".no_timeout") Nov 29 02:14:32 yoiko bbackupd[3676]: Sending stream, size uncertain Nov 29 02:14:32 yoiko bbackupd[3676]: Receive Success(0x26387d) ................ Nov 29 04:58:32 yoiko bbackupd[3676]: Receiving stream, size 2253 Nov 29 04:58:33 yoiko bbackupd[3676]: Send ListDirectory(0x6dde8,0xffffffff,0xc,true) Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6dde8) Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 1708 Nov 29 04:58:33 yoiko bbackupd[3676]: Send ListDirectory(0x6ddec,0xffffffff,0xc,true) Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddec) Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 13656 Nov 29 04:58:33 yoiko bbackupd[3676]: Send ListDirectory(0x6ddf4,0xffffffff,0xc,true) Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 Nov 29 04:58:48 yoiko bbackupd[3676]: Send SetClientStoreMarker(0x4235ac8b24e00) Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common OSFileError (Error accessing a file. Check permissions.) 1/9), reset state and waiting to retry... Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file size uploaded 80626331005, bytes already on server 827066486, encoded size 55033653560 Ben Summers wrote: > > What's the last file or directory name listed in the logs then? It must > mention something. > > Ben > > > On 29 Nov 2006, at 19:41, Tobias Balle-Petersen wrote: > >> Hello, >> >> bbackupd is running as root. >> bbackupd is realoded prior to every run. >> >> Tobias >> >> >> Ben Summers wrote: >>> >>> The easiest way is to restart the daemon. (If you're using the store >>> info file, delete that while the daemon is stopped.) Then the logs >>> should give enough filenames to track it down. >>> >>> The most likely cause is that you're running bbackupd as non-root, and >>> it's tried to read a file but permissions denied access. >>> >>> Ben >>> >>> >>> On 29 Nov 2006, at 16:49, Tobias Balle-Petersen wrote: >>> >>>> Hello, >>>> >>>> My backup run ends as seen below from my client log. Extended >>>> logging is >>>> enabled. How do I find the file causing the error? >>>> >>>> Log from client: >>>> Nov 29 04:58:33 yoiko bbackupd[3676]: Send >>>> ListDirectory(0x6ddf4,0xffffffff,0xc,true) >>>> Nov 29 04:58:33 yoiko bbackupd[3676]: Receive Success(0x6ddf4) >>>> Nov 29 04:58:33 yoiko bbackupd[3676]: Receiving stream, size 6597170 >>>> Nov 29 04:58:48 yoiko bbackupd[3676]: Send >>>> SetClientStoreMarker(0x4235ac8b24e00) >>>> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) >>>> Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() >>>> Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() >>>> Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common >>>> OSFileError (Error accessing a file. Check permissions.) 1/9), reset >>>> state and waiting to retry... >>>> Nov 29 04:58:58 yoiko bbackupd[3676]: File statistics: total file size >>>> uploaded 80626331005, bytes already on server 827066486, encoded size >>>> 55033653560 >>>> >>>> >>>> Thanks, >>>> Tobias >>>> _______________________________________________ >>>> boxbackup mailing list >>>> boxbackup at fluffy.co.uk >>>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> >>> _______________________________________________ >>> boxbackup mailing list >>> boxbackup at fluffy.co.uk >>> http://lists.warhead.org.uk/mailman/listinfo/boxbackup >>> >> >> _______________________________________________ >> boxbackup mailing list >> boxbackup at fluffy.co.uk >> http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Nov 30 19:07:29 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 30 Nov 2006 22:07:29 +0300 (EAT) Subject: [Box Backup] Common OSFileError - What file ? In-Reply-To: <456E06FA.7090709@kontrapunkt.com> References: <456DBA1E.4090905@kontrapunkt.com> <64DE468E-5598-4E6C-A4FF-E341E51F371C@fluffy.co.uk> <456DE27D.909@kontrapunkt.com> <456E06FA.7090709@kontrapunkt.com> Message-ID: Hi Tobias, On Wed, 29 Nov 2006, Tobias Balle-Petersen wrote: > More from my log. [...] > Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Success(0x4235ac8b24e00) > Nov 29 04:58:48 yoiko bbackupd[3676]: Send Finished() > Nov 29 04:58:48 yoiko bbackupd[3676]: Receive Finished() > Nov 29 04:58:48 yoiko bbackupd[3676]: Exception caught (Common > OSFileError (Error accessing a file. Check permissions.) 1/9), reset > state and waiting to retry... This is right at the end of the backup run. Perhaps bbackupd had problems writing to a file in the DataDirectory, such as the last_sync_finish file or the ID maps? What filesystem is your DataDirectory on, and is it full, or is that file somehow immutable? Could you change it to /tmp? I'll add better logging of this to my tree, so it should make it into trunk eventually. I don't think it could be a NotifyScript or a StoreObjectInfoFile, since both would give a more informative log entry if they had problems, but could you try disabling them if you use them? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |