From markus at the-grue.de Fri Jul 26 15:16:13 2013 From: markus at the-grue.de (markus at the-grue.de) Date: Fri, 26 Jul 2013 16:16:13 +0200 Subject: [Box Backup] ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist (and others) Message-ID: <20130726161613.1be0a94a@haktar.galaxy.home> Hello, since many weeks, I have these messages (see below) in my logfile and I have no Idea what to do about it. I ran these commands, but it didn't get any better: bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf info 1001 bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf check 1001 bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf check 1001 fix Can you help me please? Its a bad feeling if the backup spits out errors and you know nothing to do :( Jul 26 15:14:28 benji client=0x00001001[1609]: WARNING: Exception thrown: ConnectionException(Conn_Protocol_Timeout) at Protocol.cpp(190) Jul 26 15:14:28 benji bbstored[1609]: ERROR: Error in child process, terminating connection: Connection Protocol_Timeout (Probably a network issue between client and server.) (7/41) Jul 26 15:19:33 benji bbstored/hk[22079]: ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist Jul 26 15:19:33 benji bbstored/hk[22079]: WARNING: Exception thrown: RaidFileException(RaidFileDoesntExist) at RaidFileRead.cpp(1026) Jul 26 15:19:33 benji bbstored/hk[22079]: ERROR: Housekeeping on account 0x00001001 threw exception, aborting run for this account: RaidFile RaidFileDoesntExist (Error when accessing a file on the store. Check the store with bbstoreaccounts check.) (2/11) Jul 26 15:34:33 benji bbstored/hk[22079]: ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist Jul 26 15:34:33 benji bbstored/hk[22079]: WARNING: Exception thrown: RaidFileException(RaidFileDoesntExist) at RaidFileRead.cpp(1026) Jul 26 15:34:33 benji bbstored/hk[22079]: ERROR: Housekeeping on account 0x00001001 threw exception, aborting run for this account: RaidFile RaidFileDoesntExist (Error when accessing a file on the store. Check the store with bbstoreaccounts check.) (2/11) Jul 26 15:50:38 benji client=0x00001001[2711]: ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist Jul 26 15:50:38 benji client=0x00001001[2711]: WARNING: Exception thrown: RaidFileException(RaidFileDoesntExist) at RaidFileRead.cpp(1026) -- Markus Grunwald https://www.the-grue.de Fragen zur Mail? https://www.the-grue.de/mail_und_co https://www.the-grue.de/~markus/markus_grunwald.gpg From chris at qwirx.com Fri Jul 26 19:12:26 2013 From: chris at qwirx.com (Chris Wilson) Date: Fri, 26 Jul 2013 19:12:26 +0100 (BST) Subject: [Box Backup] ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist (and others) In-Reply-To: <20130726161613.1be0a94a@haktar.galaxy.home> References: <20130726161613.1be0a94a@haktar.galaxy.home> Message-ID: Hi Markus, On Fri, 26 Jul 2013, markus at the-grue.de wrote: > since many weeks, I have these messages (see below) in my logfile and I > have no Idea what to do about it. > > I ran these commands, but it didn't get any better: > > bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf info 1001 > bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf check 1001 > bbstoreaccounts -c /etc/boxbackup/server/bbstored.conf check 1001 fix > > Can you help me please? Its a bad feeling if the backup spits out errors and > you know nothing to do :( I agree, it definitely is. We have had a few sporadic reports of this problem over the years, and never got to the bottom of it. In some cases it may have been caused by doing funny things to the store filesystem, such as running it on NFS or synchronising with another machine. Do you do anything like that? What filesystem is the store on? It could also be caused by the server crashing and losing files from the filesystem due to inconsistent caches. We should really be able to fix it ourselves, though. Can you see exactly when this started happening, and whether anything else strange happened to the server at around the same time? Also are you willing to help us to debug it and fix it, by checking out the latest Box Backup source code from Subversion and compiling and testing it? And what version of Box Backup are you running? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From markus at the-grue.de Tue Jul 30 20:17:50 2013 From: markus at the-grue.de (Markus Grunwald) Date: Tue, 30 Jul 2013 21:17:50 +0200 Subject: [Box Backup] ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist (and others) In-Reply-To: References: <20130726161613.1be0a94a@haktar.galaxy.home> Message-ID: <20130730211750.0ccddece@haktar.galaxy.home> Hello Chris, > > since many weeks, I have these messages (see below) in my logfile and I > > have no Idea what to do about it. > > In some cases > it may have been caused by doing funny things to the store filesystem, > such as running it on NFS or synchronising with another machine. Do you > do anything like that? What filesystem is the store on? The store is in a USB drive with ext4. No funny business. > It could also be caused by the server crashing and losing files from the > filesystem due to inconsistent caches. That's quite possible. There was a time when the server crashed multiple times and I think the problems with the backup store started back then. > Can you see exactly when this started happening, Unfortunately not. It was a long time ago. I had a lot of patience ;) > and whether anything else > strange happened to the server at around the same time? Nothing besides the crashes. > Also are you willing to help us to debug it and fix it, by checking out > the latest Box Backup source code from Subversion and compiling and > testing it? Yes, no problem. Do I need the latest software only for the server or for all my clients, too? > And what version of Box Backup are you running? On the server: markus at benji % dpkg -l \*boxbackup\* | egrep '^ii' ii boxbackup-server 0.11~rc8~r2714-1~bpo60+1 On client 1001: markus at ix:~$ dpkg -l \*boxbackup\* | egrep '^ii' ii boxbackup-client 0.11.1~r2837-1 Hmm, I see that they differ. But that can be only since a short time, when I upgraded ix to wheezy. benji still has squeeze. -- Markus Grunwald https://www.the-grue.de Fragen zur Mail? https://www.the-grue.de/mail_und_co https://www.the-grue.de/~markus/markus_grunwald.gpg From chris at qwirx.com Tue Jul 30 20:29:39 2013 From: chris at qwirx.com (Chris Wilson) Date: Tue, 30 Jul 2013 20:29:39 +0100 (BST) Subject: [Box Backup] ERROR: Expected raidfile backup/00001001/0b/01/oc7 does not exist (and others) In-Reply-To: <20130730211750.0ccddece@haktar.galaxy.home> References: <20130726161613.1be0a94a@haktar.galaxy.home> <20130730211750.0ccddece@haktar.galaxy.home> Message-ID: Hi Markus, On Tue, 30 Jul 2013, Markus Grunwald wrote: >> It could also be caused by the server crashing and losing files from the >> filesystem due to inconsistent caches. > > That's quite possible. There was a time when the server crashed multiple times > and I think the problems with the backup store started back then. OK, it's good that we've established a possible cause. >> Also are you willing to help us to debug it and fix it, by checking out >> the latest Box Backup source code from Subversion and compiling and >> testing it? > > Yes, no problem. Do I need the latest software only for the server or > for all my clients, too? Great, thanks! The problem is just on the server, so you just need to run the latest software there. And also you don't have to replace your installed Box Backup bbstored. You can build a release version (NOT debug, because that uses a different store format that isn't compatible) and use the built copy of bbstoreaccounts to check the store, without installing it. >> And what version of Box Backup are you running? > > On the server: > markus at benji % dpkg -l \*boxbackup\* | egrep '^ii' > ii boxbackup-server 0.11~rc8~r2714-1~bpo60+1 OK, that one is a bit old, it's possible that the problem has already been fixed. Running the latest code will tell us whether it has been fixed, or help us to debug the problem if not. To build the release version, just check out the latest code, make and then run bbstoreaccounts: cd trunk ./bootstrap ./configure make release/bin/bbstoreaccounts/bbstoreaccounts check Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software |