From boxbackup at boxbackup.org Tue Feb 3 18:16:27 2009 From: boxbackup at boxbackup.org (Brendon Baumgartner) Date: Tue, 3 Feb 2009 10:16:27 -0800 Subject: [Box Backup] Comparing C:\ Message-ID: <5DB1D2D8020D4449AF3111C784529E625D397B@jeep.netcal.l> I setup the config for Path =3D C:\ and Path =3D C: and I wasn't able to = do a compare -a -q. Looking at the logs, it seems things get hung up on permission/access errors. Is it possible to have compare skip the files that error so that it does work? Errors are below. Currently in my configs, I have path=3DC:\ and then setup exclusions. query > compare -q -a WARNING: Quick compare used -- file attributes are not checked. WARNING: Location 'Cdrive' path ends with '\', compare may fail! WARNING: Exception thrown: CommonException(OSFileError) at BackupQueries.cpp(164 2) WARNING: Location 'Ddrive' path ends with '\', compare may fail! WARNING: Exception thrown: CommonException(OSFileError) at BackupQueries.cpp(164 2) query > compare -q -a WARNING: Quick compare used -- file attributes are not checked. WARNING: Failed to access local directory 'C:': Permission denied (13) WARNING: Failed to access local directory 'D:': Permission denied (13) query > From boxbackup at boxbackup.org Tue Feb 3 18:41:04 2009 From: boxbackup at boxbackup.org (Torsten) Date: Tue, 3 Feb 2009 19:41:04 +0100 Subject: [Box Backup] Old includes Deleted files? Message-ID: <200902031941.05969.ddmails@web.de> Hi, i always though that: Actual Data = Used - Old files - Deleted files - Directories But i found this: Account ID: 0x00000012 Last object ID: 0x19a Used: 20898100 blocks, 39.86 GB, 88% |************** | Old files: 17125651 blocks, 32.66 GB, 72% |*********** | Deleted files: 4625013 blocks, 8.82 GB, 19% |*** | Directories: 28 blocks, 56.00 kB, 0% | | Soft limit: 20971520 blocks, 40.00 GB, 88% |************** | Hard limit: 23592960 blocks, 45.00 GB, 100% |****************| Client store marker: 410 So there would be Old and Deleted files more than Used files? Old + Deleted = 41,48 GB Used = 39,86 GB Can anyone help me to understand this? I use svn2409. Thanks Torsten From boxbackup at boxbackup.org Tue Feb 3 18:57:13 2009 From: boxbackup at boxbackup.org (Peter Jalajas, GigaLock Backup Services) Date: Tue, 3 Feb 2009 13:57:13 -0500 Subject: [Box Backup] Old includes Deleted files? In-Reply-To: <200902031941.05969.ddmails@web.de> References: <200902031941.05969.ddmails@web.de> Message-ID: <74d01c7a0902031057v221e026xc4ca73de6ebdfd6b@mail.gmail.com> Hi Torsten, Yeah, I've noticed that, too. See my Feature Request under "Server-side Statistics" at http://www.boxbackup.org/trac/wiki/FeatureRequests#Server-sideStatistics Pete On Tue, Feb 3, 2009 at 1:41 PM, Torsten wrote: > Hi, > > i always though that: > > Actual Data = Used - Old files - Deleted files - Directories > > But i found this: > > Account ID: 0x00000012 > Last object ID: 0x19a > Used: 20898100 blocks, 39.86 GB, 88% |************** | > Old files: 17125651 blocks, 32.66 GB, 72% |*********** | > Deleted files: 4625013 blocks, 8.82 GB, 19% |*** | > Directories: 28 blocks, 56.00 kB, 0% | | > Soft limit: 20971520 blocks, 40.00 GB, 88% |************** | > Hard limit: 23592960 blocks, 45.00 GB, 100% |****************| > Client store marker: 410 > > So there would be Old and Deleted files more than Used files? > > Old + Deleted = 41,48 GB > Used = 39,86 GB > > Can anyone help me to understand this? > > I use svn2409. > > Thanks > Torsten > _______________________________________________ From boxbackup at boxbackup.org Thu Feb 5 13:26:34 2009 From: boxbackup at boxbackup.org (Torsten) Date: Thu, 5 Feb 2009 14:26:34 +0100 Subject: [Box Backup] FeatureRequest: Mass Deletion Protection In-Reply-To: References: <20080805162324.482c7322@matrix.tuxianer.homelinux.net> <20080808220018.0ce5d34f@apfeltasche> Message-ID: <200902051426.38373.ddmails@web.de> Am Saturday 09 August 2008 15:59:59 schrieb Chris Wilson: > Hi Torsten, > > On Fri, 8 Aug 2008, Torsten wrote: > > Changing delete time to "never delete" is not what i want. This would > > prevent bb to delete the directory on the store if it is not activated > > in bbackupd.conf anymore. > > > > What i want is that: If the location is activated in bbackupd.conf and > > it is not accessible in local directory - then bb should not stop > > syncing and printing errors. It should print a warning to the logs and > > then start syncing the next location. It should not mark the directory > > as deleted. Do you think anybody else would need this? I hope you can > > understand my english ... > > I think we are actually talking about the same thing. > DeleteRedundantLocationsAfter already exists in trunk, and only applies to > locations, and only if they are not found on the client's local disk. That > sounds like the same case that you describe. > > Changing the value of DeleteRedundantLocationsAfter should change the time > until the location is deleted on the server, which I think is what you're > trying to avoid happening. Setting it to some enormous value would > probably already achieve what you want to do, without patching. > > I'd hate to add another way to do exactly the same thing in a different > place in the code. Could you try this setting in bbackupd without the > patch, and see if it does what you want to do with this patch? if so, then > the patch might not be necessary. > > Cheers, Chris. Hi Chris, i tested this DeleteRedundantLocationsAfter and i think it does not exactly what i want. I have set DeleteRedundantLocationsAfter = 99999999 Locations not found on the local disk are never deleted, no matter if they are activated in bbackupd.conf or not. I want this: * Locations activated in bbackupd.conf and not found on local disk -> never delete from backup store. * Locations not activated in bbackupd.conf and not found on local disk -> delete on backup store after some time (for example the known value of 48 hours). Can you help me with this? Perhaps i forgot to activate some value in the config? thanks Torsten From boxbackup at boxbackup.org Thu Feb 5 17:08:05 2009 From: boxbackup at boxbackup.org (Torsten) Date: Thu, 5 Feb 2009 18:08:05 +0100 Subject: [Box Backup] Unknown errors In-Reply-To: References: <200812181919.25786.ddmails@web.de> <200812191256.04432.ddmails@web.de> Message-ID: <200902051808.10376.ddmails@web.de> Hi Chris, after a long test period here are my test results. Am Friday 19 December 2008 17:57:07 schrieb Chris Wilson: > Hi Torsten, > > On Fri, 19 Dec 2008, Torsten wrote: > > Dec 19 11:55:31 matrix bbackupd[6806]: NOTICE: Beginning scan of local > > files Dec 19 12:10:18 matrix bbackupd[6806]: ERROR: Failed to read from > > file: /boxmount/boxmount-SERVERHH/testfile.dat: Permission denied (13) > > Dec 19 12:10:18 exobackup bbackupd[6806]: WARNING: Exception thrown: > > CommonException(OSFileReadError) at FileStream.cpp(200) > > Dec 19 12:10:18 matrix bbackupd[6806]: ERROR: Failed to upload > > file: /boxmount/boxmount-SERVERHH/testfile.dat: caught exception: Common > > OSFileReadError (1/14) > > Dec 19 12:10:18 matrix bbackupd[6806]: WARNING: ListDirectory command > > failed: received error FileDoesNotVerify = 6 > > Dec 19 12:10:18 matrix bbackupd[6806]: WARNING: Exception thrown: > > ConnectionException(Conn_Protocol_UnexpectedReply) at > > autogen_BackupProtocolClient.cpp(1545) > > Here is the problem. For some reason, bbackupd was able to open > /boxmount/boxmount-SERVERHH/testfile.dat, but not able to read from it. > This was not an expected situation; normally the open would fail if there > was a permissions problem. Can you copy that file to a local file on > "matrix" without errors, when running as the bbackupd user? What kind of > filesystem is it, and if it's a network mount, what is the remote OS? > > After this error, it appears that we did not finish the StoreFile > command, and we sent a ListDirectory command. The server interpreted this > command as the file data that it was expecting us to upload, and as it is > not valid file data, it reported the error FileDoesNotVerify. > > The client is however expecting a response to the ListDirectory command, > and FileDoesNotVerify is not a valid response, so it throws the > Conn_Protocol_UnexpectedReply exception, which ends the backup run. The error occurs only on cifs mounted exports. But it occurs only sometimes and i could not reliable reproduce it. Some lines before the FileDoesNotVerify message from box backup i found the following in the syslog: Feb 4 12:03:26 boxbackup kernel: Status code returned 0xc0000054 NT_STATUS_FILE_LOCK_CONFLICT Feb 4 12:03:26 boxbackup kernel: CIFS VFS: Send error in read = -13 After some searching with google i found this patch for backuppc. Backuppc also mounts windows exports and backes them up. http://osdir.com/ml/sysutils.backup.backuppc.general/2004-09/msg00051.html I do not really unterstand this problem. But i think NT_STATUS_FILE_LOCK_CONFLICT should just not be rated as an error?!? > > Dec 19 11:19:36 test bbackupd[2679]: ERROR: Failed to read from > > file: /exomount/exomount-server-DiskD/WEB/PC_info/Outlook/Outlook.pst: > > Permission denied (13) > > Dec 19 11:19:36 test bbackupd[2679]: ERROR: Internal error during backup > > run: basic_string::substr > > Dec 19 11:19:36 test bbackupd[2679]: NOTICE: About to notify > > administrator about event backup-error, running > > script '/root/exobackup/boxbackup/NotifySysadmin.sh backup-error' > > I'm pretty confused about the basic_string::substr error. According to the > code this should be impossible. Please could you build a clean copy of the > latest snapshot or trunk on this machine and try running it instead of > r2237? > > Given that an error was reported about Outlook.pst just before that, I > suspect that a similar condition may be happening here as the one above. > Perhaps this one is caused by the file being locked by the operating > system on "exomount server". Can you copy this file from > /exomount/exomount-server-DiskD/WEB/PC_info/Outlook to a local disk? If > you get an error, does it copy part of the file? Could you exclude that > file from the backup and see if it runs successfully then? After an update to svn2409 i saw this error only once and i could not repoduce it. But i will keep an eye on this. thank you torsten From boxbackup at boxbackup.org Wed Feb 4 15:03:34 2009 From: boxbackup at boxbackup.org (Robin Harteveld) Date: Wed, 4 Feb 2009 16:03:34 +0100 Subject: [Box Backup] Can total hardlimitsize exceed partitionsize? Message-ID: <8ae2ccf80902040703r43532497x8b081aaca69fe984@mail.gmail.com> --000e0cd247e47420f2046219178f Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hello, I have a central boxbackup server with 125 stores on it. The size of the partition with the stores is 7.5TB. Stores are from different users. They difference is size ( 10GB - 20GB - 40GB - 100GB). The hardlimit is set to "softlimit+10GB". Sometimes a users adds more then 10 Gb to his store. In some cases the hardlimit gets full. I noticed that when a users cleans-up. The filed are not marked as old files. That way the store stays full. The only way to fix this, is to increase the hardlimit. Old-files are marked. house-keeping does its job and new files come in. Anybody recognizes this behaviour? We use boxbackup 0.11rc2. on a Linux Centos 4.4 servers. (clients and server) And now the question: Is it possible to configure all stores to a hardlimit of 120 GB? Or is this a problem with the capacity of the harddisk ( 125x 120Gb = 15 TB) None of the stores will actually get that big and we will be monitoring the freedisk-space. Does boxbackup reserve the diskspace needed by the hardlimit? Thanks, Robin Harteveld The Netherlands --000e0cd247e47420f2046219178f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello,

I have a central boxbackup server with 125 stores on it. The = size of the partition with the stores is 7.5TB.

Stores are from diff= erent users. They difference is size ( 10GB - 20GB - 40GB - 100GB). The har= dlimit is set to "softlimit+10GB".

Sometimes a users adds more then 10 Gb to his store. In some cases the = hardlimit gets full.  I noticed that when a users cleans-up. The filed= are not marked as old files. That way the store stays full.
The only wa= y to fix this, is to increase the hardlimit. Old-files are marked. house-ke= eping does its job and new files come in.

Anybody recognizes this behaviour?

We use boxbackup 0.11rc2. on = a Linux Centos 4.4 servers. (clients and server)

And now the questio= n:

Is it possible to configure all stores to a hardlimit of 120 GB?&= nbsp;  Or is this a problem with the capacity of the harddisk ( 125x 1= 20Gb =3D 15 TB)
None of the stores will actually get that big and we will be monitoring the= freedisk-space.

Does boxbackup reserve the diskspace needed by the = hardlimit?


Thanks,

Robin Harteveld
The Netherlands



--000e0cd247e47420f2046219178f-- From boxbackup at boxbackup.org Sat Feb 7 17:30:11 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Sat, 7 Feb 2009 17:30:11 +0000 (GMT) Subject: [Box Backup] Can total hardlimitsize exceed partitionsize? In-Reply-To: <8ae2ccf80902040703r43532497x8b081aaca69fe984@mail.gmail.com> References: <8ae2ccf80902040703r43532497x8b081aaca69fe984@mail.gmail.com> Message-ID: Hi Robin, On Wed, 4 Feb 2009, Robin Harteveld wrote: > I have a central boxbackup server with 125 stores on it. The size of the > partition with the stores is 7.5TB. Wow, that's a huge partition. What's it made of and what filesystem do you use on it? > Stores are from different users. They difference is size ( 10GB - 20GB - > 40GB - 100GB). The hardlimit is set to "softlimit+10GB". > > Sometimes a users adds more then 10 Gb to his store. In some cases the > hardlimit gets full. I noticed that when a users cleans-up. The filed > are not marked as old files. That way the store stays full. The only way > to fix this, is to increase the hardlimit. Old-files are marked. > house-keeping does its job and new files come in. The deleted files are supposed to be marked as deleted, not old. This is supposed to happen even if the store is full. I.e. the boxbackup client is supposed to continue going through the store and marking locally deleted files as deleted on the store, but not uploading any more, when the store is full. If that's not happening, then it's a bug in the client and I'd like to know more about it. > We use boxbackup 0.11rc2. on a Linux Centos 4.4 servers. (clients and > server) > > And now the question: > > Is it possible to configure all stores to a hardlimit of 120 GB? Or is > this a problem with the capacity of the harddisk ( 125x 120Gb = 15 TB) > None of the stores will actually get that big and we will be monitoring the > freedisk-space. > > Does boxbackup reserve the diskspace needed by the hardlimit? No, the space is not reserved. Overcommitting your disk space is possible and allowed. However recent clients basically ignore the softlimit and will keep backing up files until they reach the hardlimit. Therefore I'd recommend that you closely monitor free disk space and actual usage by the clients. If the disk does actually fill up then clients will start getting bad errors during uploads, and your and their bandwidth usage will increase a lot. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Sat Feb 7 23:45:59 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Sat, 7 Feb 2009 23:45:59 +0000 (GMT) Subject: [Box Backup] Store corruption not detected or fixed by bbstoreaccounts In-Reply-To: References: Message-ID: Hi Alex, Sorry for the delay in replying to your email. On Sat, 24 Jan 2009, Alex Harper wrote: >> Is it deep in directories? Could you copy it and its parent directories to >> a new account? Otherwise can you leave it in the account that it's in now >> for a few days? > > New discoveries (see below) make me believe I should keep the whole thing as > an exemplar. I've solved the problem by finding enough space to replicate > the account. Excellent. > On the original account with the original crypto error I reported, another > error has cropped up on a different file: > > WARNING: zlib error code = -3 > WARNING: Exception thrown: CompressException(TransformFailed) at > ../../lib/compress/Compress.h(154) > > Looking back at captured output this error was reported in the same compare > -a output as the original crypto error, but because it wasn't a fatal error > I overlooked it. However, I actually needed to restore the file yesterday > and the restore failed. Sorry to hear that. Do you run compare -a regularly? > I then ran compare -a on another user's account and discovered the exact > same error on a similar data file. In the case of the second account the > compression exception was fatal. I'm still not sure why it wasn't fatal on > my account. It might depend on the code path, as the exception might be caught on some paths and not others. If it's not caught then it will be fatal. > In both my account and the other user account the newly discovered > corrupt file is a large (1.5GB or more) mail database that Box is > backing up while open but largely quiescent. The files are from the same > program (Entourage 2008) on two different laptops of two different > architectures (PPC vs x86). In both cases this is a hot file with a > large number of revisions. > > Since the files are so structurally similar and updated similarly I'm > willing to believe that this is a problem with the diff patch. Or maybe > a problem with recipe generation in general. Is it possible for you to reproduce this problem, e.g. by setting up a dummy Entourage database and sending dummy emails to it while backing it up? > Its also possible that the problem is with the filesystem, as a part of > diagnosing this I discovered a frayed cable to the drive. That said, > fsck is fine, other non bbstored data on the drive checksums correctly, > and it seems suspicious that the corruption is so similar. If this was the case then I'd expect you to be seeing corrupted files and directories, filesystem errors, etc. So it wouldn't be the first avenue that I'd investigate. Do you back up other Entourage databases onto a different system? Do you also see problems there? > The argument against filesystem issues or cosmic rays would be that > since these are not the only hot files, why are they having such similar > errors? Agreed. Is it only Entourage databases that are suffering this problem? > If you have any diagnostics you want me to run I'm willing. Because I'm > cutting over to my secondary backup as the primary now and shutting down > bbstored there's no longer much time pressure, I can keep the account > around for a while if it helps. Please can you try to reproduce the problem in some way? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Mon Feb 9 23:32:05 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Mon, 9 Feb 2009 23:32:05 +0000 (GMT) Subject: [Box Backup] Comparing C:\ In-Reply-To: <5DB1D2D8020D4449AF3111C784529E625D397B@jeep.netcal.l> References: <5DB1D2D8020D4449AF3111C784529E625D397B@jeep.netcal.l> Message-ID: Hi Brendon, Thanks for the bug report, and sorry for not replying earlier. On Tue, 3 Feb 2009, Brendon Baumgartner wrote: > I setup the config for Path = C:\ and Path = C: and I wasn't able to do > a compare -a -q. > > Looking at the logs, it seems things get hung up on permission/access > errors. Is it possible to have compare skip the files that error so that > it does work? Errors are below. I'm not sure that it's just specific files, although I'm willing to be persuaded. I think that the error is caused by accessing the file with an invalid filename. path=c:\ should work, path=c: should not. The fact that path=c:\ does not apparently work for you is a bug, which I've added to the bug tracker and will investigate as soon as I can. Which version of Box Backup are you using on the client? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Mon Feb 9 23:37:00 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Mon, 9 Feb 2009 23:37:00 +0000 (GMT) Subject: [Box Backup] FeatureRequest: Mass Deletion Protection In-Reply-To: <200902051426.38373.ddmails@web.de> References: <20080805162324.482c7322@matrix.tuxianer.homelinux.net> <20080808220018.0ce5d34f@apfeltasche> <200902051426.38373.ddmails@web.de> Message-ID: Hi Torsten, On Thu, 5 Feb 2009, Torsten wrote: >>> What i want is that: If the location is activated in bbackupd.conf and >>> it is not accessible in local directory - then bb should not stop >>> syncing and printing errors. It should print a warning to the logs and >>> then start syncing the next location. It should not mark the directory >>> as deleted. Do you think anybody else would need this? I hope you can >>> understand my english ... > > i tested this DeleteRedundantLocationsAfter and i think it does not exactly > what i want. > > I have set DeleteRedundantLocationsAfter = 99999999 > > Locations not found on the local disk are never deleted, no matter if they are > activated in bbackupd.conf or not. > > I want this: > * Locations activated in bbackupd.conf and not found on local disk -> never > delete from backup store. > * Locations not activated in bbackupd.conf and not found on local disk -> > delete on backup store after some time (for example the known value of 48 > hours). > > Can you help me with this? Perhaps i forgot to activate some value in the > config? I agree with your assessment. I think this is a limitation of Box Backup at the moment, that it cannot tell the difference between the two situations. I will add a feature request to the bug tracker and work on fixing it when I have a chance. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Mon Feb 9 23:37:00 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Mon, 9 Feb 2009 23:37:00 +0000 (GMT) Subject: [Box Backup] FeatureRequest: Mass Deletion Protection In-Reply-To: <200902051426.38373.ddmails@web.de> References: <20080805162324.482c7322@matrix.tuxianer.homelinux.net> <20080808220018.0ce5d34f@apfeltasche> <200902051426.38373.ddmails@web.de> Message-ID: Hi Torsten, On Thu, 5 Feb 2009, Torsten wrote: >>> What i want is that: If the location is activated in bbackupd.conf and >>> it is not accessible in local directory - then bb should not stop >>> syncing and printing errors. It should print a warning to the logs and >>> then start syncing the next location. It should not mark the directory >>> as deleted. Do you think anybody else would need this? I hope you can >>> understand my english ... > > i tested this DeleteRedundantLocationsAfter and i think it does not exactly > what i want. > > I have set DeleteRedundantLocationsAfter = 99999999 > > Locations not found on the local disk are never deleted, no matter if they are > activated in bbackupd.conf or not. > > I want this: > * Locations activated in bbackupd.conf and not found on local disk -> never > delete from backup store. > * Locations not activated in bbackupd.conf and not found on local disk -> > delete on backup store after some time (for example the known value of 48 > hours). > > Can you help me with this? Perhaps i forgot to activate some value in the > config? I agree with your assessment. I think this is a limitation of Box Backup at the moment, that it cannot tell the difference between the two situations. I will add a feature request to the bug tracker and work on fixing it when I have a chance. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Mon Feb 9 23:48:32 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Mon, 9 Feb 2009 23:48:32 +0000 (GMT) Subject: [Box Backup] Unknown errors In-Reply-To: <200902051808.10376.ddmails@web.de> References: <200812181919.25786.ddmails@web.de> <200812191256.04432.ddmails@web.de> <200902051808.10376.ddmails@web.de> Message-ID: Hi Torsten, On Thu, 5 Feb 2009, Torsten wrote: > after a long test period here are my test results. >> >> Here is the problem. For some reason, bbackupd was able to open >> /boxmount/boxmount-SERVERHH/testfile.dat, but not able to read from it. >> This was not an expected situation; normally the open would fail if there >> was a permissions problem. Can you copy that file to a local file on >> "matrix" without errors, when running as the bbackupd user? What kind of >> filesystem is it, and if it's a network mount, what is the remote OS? > > The error occurs only on cifs mounted exports. But it occurs only sometimes > and i could not reliable reproduce it. > > Some lines before the FileDoesNotVerify message from box backup i found the > following in the syslog: > > Feb 4 12:03:26 boxbackup kernel: Status code returned 0xc0000054 > NT_STATUS_FILE_LOCK_CONFLICT > Feb 4 12:03:26 boxbackup kernel: CIFS VFS: Send error in read = -13 > > After some searching with google i found this patch for backuppc. Backuppc > also mounts windows exports and backes them up. > > http://osdir.com/ml/sysutils.backup.backuppc.general/2004-09/msg00051.html > > I do not really unterstand this problem. But i think > NT_STATUS_FILE_LOCK_CONFLICT should just not be rated as an error?!? I had a look at the patch to BackupPC and it appears to be related to clients running on Windows. While I agree with your assessment, unfortunately CIFS appears to return error 13, which is EACCES (Access Denied) to the client (running on a UNIX system). The only way that I can think of to distinguish this error from the usual Access Denied, when a file is locked or permissions prevent access by the client, is that in this case it appears to occur while reading the file, rather than opening it. In this case, I suspect that we should skip the file with an appropriate error message, and continue the backup. As the client will not allow us to read the file, this appears to be the only possible course of action (apart from aborting the run due to an unexpected error). The problem is that we were able to open the file and have already started sending data to the server. It's not clear to me right now what we can do to abort the upload once it's already in progress, without killing our connection to the server (perhaps that's the best thing to do). It's probably difficult to reproduce the error because it only happens when the client changes the state of the file (from unlocked to locked) during the process of backing up that file. Otherwise, we would have failed while calculating the patch, before starting the upload, and the upload would never have been started. Does this appear to make sense to you? Would it be reasonable to abort the backup run if this happens, and try to backup the file again on the next run? >> I'm pretty confused about the basic_string::substr error. According to >> the code this should be impossible. Please could you build a clean copy >> of the latest snapshot or trunk on this machine and try running it >> instead of r2237? >> >> Given that an error was reported about Outlook.pst just before that, I >> suspect that a similar condition may be happening here as the one above. You mean a protocol synchronisation error? I still don't understand how this could cause a basic_string::substr error, which should be impossible in any case. > After an update to svn2409 i saw this error only once and i could not > repoduce it. But i will keep an eye on this. Please do, thanks for your help. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Tue Feb 10 18:53:27 2009 From: boxbackup at boxbackup.org (Brendon Baumgartner) Date: Tue, 10 Feb 2009 10:53:27 -0800 Subject: [Box Backup] Comparing C:\ In-Reply-To: References: <5DB1D2D8020D4449AF3111C784529E625D397B@jeep.netcal.l> Message-ID: <5DB1D2D8020D4449AF3111C784529E625D3A1E@jeep.netcal.l> Found it... Box Backup Query Tool vtrunk_2387M, C:\Program Files\BoxBackup>bbackupquery.exe NOTICE: Box Backup Query Tool vtrunk_2387M, (c) Ben Summers and contributors 20 03-2008 Login complete. Type "help" for a list of commands. query > compare -a -q WARNING: Quick compare used -- file attributes are not checked. WARNING: Location 'Cdrive' path ends with '\', compare may fail! WARNING: Exception thrown: CommonException(OSFileError) at BackupQueries.cpp(160 3) WARNING: Location 'Ddrive' path ends with '\', compare may fail! WARNING: Exception thrown: CommonException(OSFileError) at BackupQueries.cpp(160 3) > -----Original Message----- > From: boxbackup-admin at boxbackup.org [mailto:boxbackup- > admin at boxbackup.org] On Behalf Of Chris Wilson > Sent: Monday, February 09, 2009 3:32 PM > To: boxbackup at boxbackup.org > Subject: Re: [Box Backup] Comparing C:\ >=20 > Hi Brendon, >=20 > Thanks for the bug report, and sorry for not replying earlier. >=20 > On Tue, 3 Feb 2009, Brendon Baumgartner wrote: >=20 > > I setup the config for Path =3D C:\ and Path =3D C: and I wasn't = able to > do > > a compare -a -q. > > > > Looking at the logs, it seems things get hung up on permission/access > > errors. Is it possible to have compare skip the files that error so > that > > it does work? Errors are below. >=20 > I'm not sure that it's just specific files, although I'm willing to be > persuaded. I think that the error is caused by accessing the file with > an > invalid filename. path=3Dc:\ should work, path=3Dc: should not. The = fact > that > path=3Dc:\ does not apparently work for you is a bug, which I've added to > the bug tracker and will investigate as soon as I can. >=20 > Which version of Box Backup are you using on the client? >=20 > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at boxbackup.org > http://lists.warhead.org.uk/mailman/listinfo/boxbackup >=20 From boxbackup at boxbackup.org Thu Feb 12 22:25:50 2009 From: boxbackup at boxbackup.org (Torsten) Date: Thu, 12 Feb 2009 23:25:50 +0100 Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> Message-ID: <200902122325.52548.ddmails@web.de> After an upgrade from box 0.10 to svn version i got this error for some files. Anybody an idea what the problem could be? Ist that a fatal error? nice day torsten Am Wednesday 12 November 2008 21:47:10 schrieb Brendon Baumgartner: > I can't find any previous mention of this error other than the source > code. > > > > Can someone elaborate? > > > > 11:56:53 [WARNING] Found conflicting parent ID for file ID > 18446744073709541437 (D:\apps\APPS\MP5120A\GOLD\TChinese\setup.inx): > expected 39319 (D:\apps\APPS\MP5120A\GOLD\TChinese) but found 204676 > (same directory used in two different locations?) From boxbackup at boxbackup.org Thu Feb 12 22:33:06 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Thu, 12 Feb 2009 22:33:06 +0000 (GMT) Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: <200902122325.52548.ddmails@web.de> References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902122325.52548.ddmails@web.de> Message-ID: Hi Torsten, On Thu, 12 Feb 2009, Torsten wrote: >> 11:56:53 [WARNING] Found conflicting parent ID for file ID >> 18446744073709541437 (D:\apps\APPS\MP5120A\GOLD\TChinese\setup.inx): >> expected 39319 (D:\apps\APPS\MP5120A\GOLD\TChinese) but found 204676 >> (same directory used in two different locations?) > > After an upgrade from box 0.10 to svn version i got this error for some > files. Anybody an idea what the problem could be? Ist that a fatal > error? Is this on a Windows client? It's not fatal for the backup but it may mean that both files mentioned are corrupted in the repository as they have been confused for each other. I think it's a bug that's been in the Windows client for a long time. As far as I know it shouldn't affect any other platform. Unfortunately the fix has broken my unit tests and I haven't had time to fix them yet so I haven't officially released another client, but you could try the temporary fix that I wrote for Roy: http://www.boxbackup.org/releases/temporary/boxbackup-trunk_2395M-backup-client-mingw32.zip Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Thu Feb 12 23:03:25 2009 From: boxbackup at boxbackup.org (Torsten) Date: Fri, 13 Feb 2009 00:03:25 +0100 Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902122325.52548.ddmails@web.de> Message-ID: <200902130003.28214.ddmails@web.de> Hi Chris, Am Thursday 12 February 2009 23:33:06 schrieb Chris Wilson: > Hi Torsten, > > On Thu, 12 Feb 2009, Torsten wrote: > >> 11:56:53 [WARNING] Found conflicting parent ID for file ID > >> 18446744073709541437 (D:\apps\APPS\MP5120A\GOLD\TChinese\setup.inx): > >> expected 39319 (D:\apps\APPS\MP5120A\GOLD\TChinese) but found 204676 > >> (same directory used in two different locations?) > > > > After an upgrade from box 0.10 to svn version i got this error for some > > files. Anybody an idea what the problem could be? Ist that a fatal > > error? > > Is this on a Windows client? > > It's not fatal for the backup but it may mean that both files mentioned > are corrupted in the repository as they have been confused for each other. > I think it's a bug that's been in the Windows client for a long time. As > far as I know it shouldn't affect any other platform. Unfortunately the > fix has broken my unit tests and I haven't had time to fix them yet so I > haven't officially released another client, but you could try the > temporary fix that I wrote for Roy: > > http://www.boxbackup.org/releases/temporary/boxbackup-trunk_2395M-backup-cl >ient-mingw32.zip > > Cheers, Chris. sorry for the bad news, but both platforms are linux Debian Etch systems. I ran an bbstoreaccounts check on this account and i got about 300 times this error message. WARNING: File ID 0xa6190 has different container ID, probably moved WARNING: File ID 0x6ee7e has different container ID, probably moved WARNING: File ID 0xac2f5 has different container ID, probably moved WARNING: File ID 0xaf2fd has different container ID, probably moved WARNING: File ID 0xaf757 has different container ID, probably moved WARNING: File ID 0xaf758 has different container ID, probably moved WARNING: File ID 0x96261 has different container ID, probably moved WARNING: File ID 0x91a27 has different container ID, probably moved WARNING: File ID 0xb085f has different container ID, probably moved WARNING: File ID 0xb0860 has different container ID, probably moved WARNING: File ID 0xb0861 has different container ID, probably moved WARNING: File ID 0xb5d5b has different container ID, probably moved WARNING: File ID 0xc938c has different container ID, probably moved WARNING: File ID 0xc938d has different container ID, probably moved Any ideas? thanks torsten From boxbackup at boxbackup.org Thu Feb 12 23:08:48 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Thu, 12 Feb 2009 23:08:48 +0000 (GMT) Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: <200902130003.28214.ddmails@web.de> References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902122325.52548.ddmails@web.de> <200902130003.28214.ddmails@web.de> Message-ID: Hi Torsten, On Fri, 13 Feb 2009, Torsten wrote: > sorry for the bad news, but both platforms are linux Debian Etch systems. OK, could you give me some samples of the error messages and the output of the "stat" command on the files that they report? Also what kind of filesystem are those files stored on, and the output of the "mount" command or contents of /proc/mounts? > I ran an bbstoreaccounts check on this account and i got about 300 times this > error message. > > WARNING: File ID 0xa6190 has different container ID, probably moved > WARNING: File ID 0x6ee7e has different container ID, probably moved > WARNING: File ID 0xac2f5 has different container ID, probably moved > WARNING: File ID 0xaf2fd has different container ID, probably moved > WARNING: File ID 0xaf757 has different container ID, probably moved > WARNING: File ID 0xaf758 has different container ID, probably moved > WARNING: File ID 0x96261 has different container ID, probably moved > WARNING: File ID 0x91a27 has different container ID, probably moved > WARNING: File ID 0xb085f has different container ID, probably moved > WARNING: File ID 0xb0860 has different container ID, probably moved > WARNING: File ID 0xb0861 has different container ID, probably moved > WARNING: File ID 0xb5d5b has different container ID, probably moved > WARNING: File ID 0xc938c has different container ID, probably moved > WARNING: File ID 0xc938d has different container ID, probably moved > > Any ideas? These reports show that the client has been moving files around. They may be harmless, but if those object IDs are the IDs of files reported by the client as having conflicting parent IDs, then it means that the client has wrongly moved them to new locations. What does "compare -aq" say about the client's data? Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Wed Feb 18 12:45:06 2009 From: boxbackup at boxbackup.org (=?ISO-8859-1?Q?S=F8ren_D=F8ssing?=) Date: Wed, 18 Feb 2009 21:45:06 +0900 Subject: [Box Backup] Old vs Deleted files removal Message-ID: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> It seem the house keeping process for server prefers to remove old files before deleted files when storage is full. query > usage Used 122879.9Mb 95% ************************************** Old files 1.7Mb 0% Deleted files 57668.8Mb 45% ****************** Directories 54.3Mb 0% Soft limit 122880.0Mb 96% ************************************** Hard limit 128000.0Mb 100% **************************************** query > As a result I can no longer restore any older version of any existing files. I can only restore existing files as well as deleted files. For me it's inconvenient, since the deleted files are usually deleted for a reason, and not as important to keep stored as old versions of files. I wonder if this behaviour is considered a feature or a bug. Is it possible to configure bbstored to remove deleted files before old files, or to keep a more fair balance of deleted vs. old files? Thanks, Soren From boxbackup at boxbackup.org Wed Feb 18 20:11:18 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Wed, 18 Feb 2009 20:11:18 +0000 (GMT) Subject: [Box Backup] Old vs Deleted files removal In-Reply-To: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> References: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323328-1919061994-1234987878=:4758 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT Hi Soren, On Wed, 18 Feb 2009, S?ren D?ssing wrote: > It seem the house keeping process for server prefers to remove old > files before deleted files when storage is full. ... > As a result I can no longer restore any older version of any existing > files. I can only restore existing files as well as deleted files. For > me it's inconvenient, since the deleted files are usually deleted for > a reason, and not as important to keep stored as old versions of > files. > > I wonder if this behaviour is considered a feature or a bug. Is it > possible to configure bbstored to remove deleted files before old > files, or to keep a more fair balance of deleted vs. old files? It is by design, but it bothers many people including me, and there is a ticket open to change it. Unfortunately it's not trivial as I don't understand the current selection logic and I think the best solution is to have versioned (timestamped) backups, but boxbackup was never designed for that and it will take quite a bit of work to implement it. Sorry if that's disappointing. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | --8323328-1919061994-1234987878=:4758-- From boxbackup at boxbackup.org Wed Feb 18 22:11:27 2009 From: boxbackup at boxbackup.org (Peter Jalajas, GigaLock Backup Services) Date: Wed, 18 Feb 2009 17:11:27 -0500 Subject: [Box Backup] Old vs Deleted files removal In-Reply-To: References: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> Message-ID: <74d01c7a0902181411l5097e587w53dbfd6f319544b5@mail.gmail.com> Yeah, I just squashed an account by lowering the soft limit substantially, and found the same results, Old files went to 0 GB and Deleted stayed at several GB. To me, it didn't matter for that client, but I would guess that about equal numbers of users prefer it removing olds as deleteds, depending upon their situations at any given time. I've always been intrigued by this issue. Chris, could you maybe please point me to the source code file(s) that control this? Maybe I can take a feeble stab at documenting the current logic anyway. Thanks, Pete On Wed, Feb 18, 2009 at 3:11 PM, Chris Wilson wrote: > Hi Soren, > > On Wed, 18 Feb 2009, S=F8ren D=F8ssing wrote: > >> It seem the house keeping process for server prefers to remove old >> files before deleted files when storage is full. >... > It is by design, but it bothers many people including me, and there is a > ticket open to change it. Unfortunately it's not trivial as I don't > understand the current selection logic and I think the best solution is t= o > have versioned (timestamped) backups, but boxbackup was never designed fo= r > that and it will take quite a bit of work to implement it. Sorry if that'= s > disappointing. > > Cheers, Chris. > From boxbackup at boxbackup.org Wed Feb 18 23:32:20 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Wed, 18 Feb 2009 23:32:20 +0000 (GMT) Subject: [Box Backup] Old vs Deleted files removal In-Reply-To: <74d01c7a0902181411l5097e587w53dbfd6f319544b5@mail.gmail.com> References: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> <74d01c7a0902181411l5097e587w53dbfd6f319544b5@mail.gmail.com> Message-ID: Hi Pete, On Wed, 18 Feb 2009, Peter Jalajas, GigaLock Backup Services wrote: > Yeah, I just squashed an account by lowering the soft limit > substantially, and found the same results, Old files went to 0 GB and > Deleted stayed at several GB. To me, it didn't matter for that > client, but I would guess that about equal numbers of users prefer it > removing olds as deleteds, depending upon their situations at any > given time. I suspected as much, that it woulnd't be a simple matter of reversing the priorities. > I've always been intrigued by this issue. Chris, could you maybe please > point me to the source code file(s) that control this? Maybe I can take > a feeble stab at documenting the current logic anyway. I think all the action happens in bin/bbstored/HousekeepStoreAccount.cpp. It starts in HousekeepStoreAccount::DoHousekeeping(), but that doesn't do much. The real action I suspect is in HousekeepStoreAccount::ScanDirectory() which is called on the root directory and calls itself recursively, building up a "map to count the distance from the mark" as it goes. That's where I get lost, I don't know what the "mark" is or what's done with this list yet. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Thu Feb 19 06:15:28 2009 From: boxbackup at boxbackup.org (Peter Jalajas, GigaLock Backup Services) Date: Thu, 19 Feb 2009 01:15:28 -0500 Subject: [Box Backup] Old vs Deleted files removal In-Reply-To: References: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> <74d01c7a0902181411l5097e587w53dbfd6f319544b5@mail.gmail.com> Message-ID: <74d01c7a0902182215p30c255few6823fdb568b8f074@mail.gmail.com> Hi Chris, Thanks for the lead. I don't know if any of this helps, but I'm trying... Another angle now: deleting that last copy of the Deleted file removes the last chance for the user to restore his lost file. We should keep at least the very latest Old or Deleted copy of every file for as long as possible. Then remove the last Old copy of one file (cuz it still has a Current version backed up) before removing the last Deleted copy of another file (cuz it has no Current version backed up). We can't presume that the user deleted his file intentionally. To rephrase, in order of decreasing importance (remove from bottom up): 1. Current version of file. 2. Latest Deleted version of file. (Presumes no Current version. Presumes it is newer than any Old versions?) 3. Latest Old version of file. (Presumes Current version present.) 4. Older Old versions of file, including any old Deleted versions of file that may be hanging around. (Discussion for another day: should housekeeping delete long-ago backed up Current files if needed to make room for new files trying to be backed up?) See below for highly redacted code. See "Marks" and Flags section at bottom. Sorry if this is more distracting than helpful. Thanks, Pete https://www.boxbackup.org/svn/box/snapshots/0.11_trunk_2368/bin/bbstored/HousekeepStoreAccount.cpp Not sure if this helps, but I read it as: // Name: HousekeepStoreAccount::DoHousekeeping() // Calculate how much should be deleted // Scan the directory for potential things to delete // This will also remove eligible items marked with RemoveASAP // If scan directory stopped for some reason, probably parent // instructed to terminate, stop now. // If any files were marked "delete now", then update // the size of the store // Reset the delta counts for files, as they will include // RemoveASAP flagged files deleted during the initial scan. // Go and delete items from the accounts ------ // Name: HousekeepStoreAccount::ScanDirectory(int64_t) // Get the filename // Open it. // Add the size of the directory on disc to the size being calculated // Read the directory in // Remove any files which are marked for removal as soon // as they become old or deleted. // Iterate through the directory deletedSomething = false; BackupStoreDirectory::Iterator i(dir); BackupStoreDirectory::Entry *en = 0; while((en = i.Next(BackupStoreDirectory::Entry::Flags_File)) != 0) { int16_t enFlags = en->GetFlags(); if((enFlags & BackupStoreDirectory::Entry::Flags_RemoveASAP) != 0 && (enFlags & (BackupStoreDirectory::Entry::Flags_Deleted | BackupStoreDirectory::Entry::Flags_OldVersion)) != 0) { // Delete this immediately. DeleteFile(ObjectID, en->GetObjectID(), dir, objectFilename, originalDirSizeInBlocks); // flag as having done something deletedSomething = true; // Must start the loop from the beginning again, as iterator is now // probably invalid. break; // Add files to the list of potential deletions // map to count the distance from the mark //PJ: see "Marks" below. Are we just trying to find the oldest version of a file? // map to count the distance from the mark std::map, int32_t> markVersionAges; // map of pair (filename, mark number) -> version age // NOTE: use a reverse iterator to allow the distance from mark stuff to work BackupStoreDirectory::ReverseIterator i(dir); BackupStoreDirectory::Entry *en = 0; while((en = i.Next(BackupStoreDirectory::Entry::Flags_File)) != 0) { // Update recalculated usage sizes int16_t enFlags = en->GetFlags(); int64_t enSizeInBlocks = en->GetSizeInBlocks(); mBlocksUsed += enSizeInBlocks; if(enFlags & BackupStoreDirectory::Entry::Flags_OldVersion) mBlocksInOldFiles += enSizeInBlocks; if(enFlags & BackupStoreDirectory::Entry::Flags_Deleted) mBlocksInDeletedFiles += enSizeInBlocks; // Work out ages of this version from the last mark int32_t enVersionAge = 0; std::map, int32_t>::iterator enVersionAgeI(markVersionAges.find(std::pair(en->GetName(), en->GetMarkNumber()))); if(enVersionAgeI != markVersionAges.end()) { enVersionAge = enVersionAgeI->second + 1; enVersionAgeI->second = enVersionAge; } else { markVersionAges[std::pair(en->GetName(), en->GetMarkNumber())] = enVersionAge; } // enVersionAge is now the age of this version. // Potentially add it to the list if it's deleted, if it's an old version or deleted if((enFlags & (BackupStoreDirectory::Entry::Flags_Deleted | BackupStoreDirectory::Entry::Flags_OldVersion)) != 0) { // Is deleted / old version. DelEn d; d.mObjectID = en->GetObjectID(); d.mInDirectory = ObjectID; d.mSizeInBlocks = en->GetSizeInBlocks(); d.mMarkNumber = en->GetMarkNumber(); d.mVersionAgeWithinMark = enVersionAge; d.mIsFlagDeleted = (enFlags & BackupStoreDirectory::Entry::Flags_Deleted) ? true : false; // Add it to the list mPotentialDeletions.insert(d); // Update various counts // Too much in the list of potential deletions? // (check against the deletion target + the max size in deletions, so that we never delete things // and take the total size below the deletion size target) // Make iterator for the last element, while checking that there's something there in the first place. // Nothing left in set // Make this into an iterator pointing to the last element in the set // Delete this one? // Will need to recalculate the maximum size now, because we've just deleted that element // Over the size to remove, so stop now // Because an object which was the maximum size recorded was deleted from the set // it's necessary to recalculate this maximum. // Recurse into subdirectories // Next level "Marks": So, what are these? From: // File // Name: BackupStoreDirectory.h // Purpose: Representation of a backup directory // Class // Name: BackupStoreDirectory // Purpose: In memory representation of a directory // Marks // The lowest mark number a version of a file of this name has ever had uint32_t GetMinMarkNumber() const {return mMinMarkNumber;} // The mark number on this file uint32_t GetMarkNumber() const {return mMarkNumber;} //PJ: Here are the Flags (also in BackupStoreDirectory.h). Is this why OldVersions are removed before Deleted versions? // Make sure these flags are synced with those in backupprocotol.txt // ListDirectory command enum { Flags_INCLUDE_EVERYTHING = -1, Flags_EXCLUDE_NOTHING = 0, Flags_EXCLUDE_EVERYTHING = 31, // make sure this is kept as sum of ones below! Flags_File = 1, Flags_Dir = 2, Flags_Deleted = 4, Flags_OldVersion = 8, Flags_RemoveASAP = 16 // if this flag is set, housekeeping will remove it as it is marked Deleted or OldVersion }; On Wed, Feb 18, 2009 at 6:32 PM, Chris Wilson wrote: > Hi Pete, > > On Wed, 18 Feb 2009, Peter Jalajas, GigaLock Backup Services wrote: ... > > I think all the action happens in bin/bbstored/HousekeepStoreAccount.cpp. It > starts in HousekeepStoreAccount::DoHousekeeping(), but that doesn't do much. > The real action I suspect is in HousekeepStoreAccount::ScanDirectory() which > is called on the root directory and calls itself recursively, building up a > "map to count the distance from the mark" as it goes. That's where I get > lost, I don't know what the "mark" is or what's done with this list yet. > > Cheers, Chris. From boxbackup at boxbackup.org Thu Feb 19 16:01:26 2009 From: boxbackup at boxbackup.org (Torsten) Date: Thu, 19 Feb 2009 17:01:26 +0100 Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902130003.28214.ddmails@web.de> Message-ID: <200902191701.27370.ddmails@web.de> Hi Chris, Am Friday 13 February 2009 00:08:48 schrieb Chris Wilson: > Hi Torsten, > > On Fri, 13 Feb 2009, Torsten wrote: > > sorry for the bad news, but both platforms are linux Debian Etch systems. > > OK, could you give me some samples of the error messages and the output of > the "stat" command on the files that they report? Also what kind of > filesystem are those files stored on, and the output of the "mount" > command or contents of /proc/mounts? > > > I ran an bbstoreaccounts check on this account and i got about 300 times > > this error message. > > > > WARNING: File ID 0xa6190 has different container ID, probably moved > > WARNING: File ID 0x6ee7e has different container ID, probably moved > > WARNING: File ID 0xac2f5 has different container ID, probably moved > > WARNING: File ID 0xaf2fd has different container ID, probably moved > > WARNING: File ID 0xaf757 has different container ID, probably moved > > WARNING: File ID 0xaf758 has different container ID, probably moved > > WARNING: File ID 0x96261 has different container ID, probably moved > > WARNING: File ID 0x91a27 has different container ID, probably moved > > WARNING: File ID 0xb085f has different container ID, probably moved > > WARNING: File ID 0xb0860 has different container ID, probably moved > > WARNING: File ID 0xb0861 has different container ID, probably moved > > WARNING: File ID 0xb5d5b has different container ID, probably moved > > WARNING: File ID 0xc938c has different container ID, probably moved > > WARNING: File ID 0xc938d has different container ID, probably moved > > > > Any ideas? > > These reports show that the client has been moving files around. They may > be harmless, but if those object IDs are the IDs of files reported by the > client as having conflicting parent IDs, then it means that the client has > wrongly moved them to new locations. > > What does "compare -aq" say about the client's data? > > Cheers, Chris. I think the problem is solved. After some backup runs the errors for "conflicting parent ID" are completely gone. Only hundreds of this "WARNING: File ID 0xc938d has different container ID, probably moved" still exists. But you said this is no problem. So everything is ok. Thanks. Torsten From boxbackup at boxbackup.org Thu Feb 19 22:37:22 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Thu, 19 Feb 2009 22:37:22 +0000 (GMT) Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: <200902191701.27370.ddmails@web.de> References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902130003.28214.ddmails@web.de> <200902191701.27370.ddmails@web.de> Message-ID: Hi Torsten, On Thu, 19 Feb 2009, Torsten wrote: > After some backup runs the errors for "conflicting parent ID" are completely > gone. I don't know why that would happen. The store data is supposed to be in memory, so if the new build removed the error, then it should have taken effect immediately. Are you using StoreObjectInfoFile? > Only hundreds of this > > "WARNING: File ID 0xc938d has different container ID, probably moved" > > still exists. But you said this is no problem. It can happen in normal use, but it can also mean that the previous error (conflicting parent ID) has corrupted the store. Did you run "compare -aq"? I'd strongly recommend that you do that in order to trust your backups again, and keep doing it regularly anyway. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Fri Feb 20 01:53:45 2009 From: boxbackup at boxbackup.org (=?ISO-8859-1?Q?S=F8ren_D=F8ssing?=) Date: Fri, 20 Feb 2009 10:53:45 +0900 Subject: [Box Backup] Old vs Deleted files removal In-Reply-To: References: <7137d3f80902180445v1f0ee078mf7b4b011db5bb5ff@mail.gmail.com> <74d01c7a0902181411l5097e587w53dbfd6f319544b5@mail.gmail.com> Message-ID: <7137d3f80902191753s4cc39560sb2e205d05f3ccbfa@mail.gmail.com> On Thu, Feb 19, 2009 at 8:32 AM, Chris Wilson wrote: > I think all the action happens in bin/bbstored/HousekeepStoreAccount.cpp. It > starts in HousekeepStoreAccount::DoHousekeeping(), but that doesn't do much. > The real action I suspect is in HousekeepStoreAccount::ScanDirectory() which > is called on the root directory and calls itself recursively, building up a > "map to count the distance from the mark" as it goes. That's where I get > lost, I don't know what the "mark" is or what's done with this list yet. Chris, thanks for your explanation. I looked inside the source code to see if I could find out what was going on. I think I got at least one step further towards a reasonable solution. In HousekeepStoreAccount.h I find the following explanation of Mark: int32_t mVersionAgeWithinMark; // 0 == current, 1 latest old version, etc And in HousekeepStoreAccount.cpp I find the follow calculation of Mark: // Work out ages of this version from the last mark int32_t enVersionAge = 0; std::map, int32_t>::iterator enVersionAgeI(markVersionAges.find(std::pair(en->GetName(), en->GetMarkNumber()))); if(enVersionAgeI != markVersionAges.end()) { enVersionAge = enVersionAgeI->second + 1; enVersionAgeI->second = enVersionAge; } else { markVersionAges[std::pair(en->GetName(), en->GetMarkNumber())] = enVersionAge; } // enVersionAge is now the age of this version. The way I read it is; Assume Mark is 0. If other versions of same file is found, sort them and set Mark to the sorted position. I'm speculating that the problem here is that the latest version of a deleted file is 0. This would be wrong; the latest backup version of a deleted file is not current. It should be 1 to indicate that it's one version different than what is on client disk, where the file is now gone from. A simple change is to bump up the enVersionAge by 1 for deleted files to make them comparable in revision to old files. *** HousekeepStoreAccount.cpp.orig 2008-01-29 09:58:25.000000000 +0900 --- HousekeepStoreAccount.cpp 2009-02-20 09:54:55.000000000 +0900 *************** bool HousekeepStoreAccount::ScanDirector *** 392,397 **** --- 392,398 ---- markVersionAges[std::pair(en->GetName(), en->GetMarkNumber())] = enVersionAge; } // enVersionAge is now the age of this version. + if(enFlags & BackupStoreDirectory::Entry::Flags_Deleted) ++enVersionAge; // Potentially add it to the list if it's deleted, if it's an old version or deleted if((enFlags & (BackupStoreDirectory::Entry::Flags_Deleted | BackupStoreDirectory::Entry::Flags_OldVersion)) != 0) This is just an idea for a quick fix. It's entirely untested, and probably wrong if my assumptions are incorrect. For example, I'm not sure if enVersionAge is really counting version numbers or holds age of file in seconds. I'm also concerned that enVersionAge might be used as index to an array, and incrementing it could go out of bounds for the array. That said, is anybody able to test and confirm if this approach would fix the problem? Soren From boxbackup at boxbackup.org Fri Feb 20 04:13:58 2009 From: boxbackup at boxbackup.org (Dnk) Date: Thu, 19 Feb 2009 20:13:58 -0800 Subject: [Box Backup] Current status 0.11 Message-ID: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> I was curious, as to the current status of 0.11. I have been following the list, and waiting to have a go. I am building a new backup server and was hoping to have a crack at box backup. Now I read the disclaimers about windows and such on the website and wiki. Is there any sort of eta when this version may become stable? I know that is probably an annoying and broad question. Also, is there any plans for an os x client? I wish I could code to help out with that one. Unfortunately I am no programmer. Any plans for deduplication? In my network, I am primarily backing up Linux servers, and windows workstations. I have a few mac workstations as well. Due to my clients and the curret status, would I be better off to bypass box backup at this time? I really like the feature set of this project, and the encryption for remote backups. Thanks in advance! Dnk From boxbackup at boxbackup.org Fri Feb 20 09:00:21 2009 From: boxbackup at boxbackup.org (Torsten) Date: Fri, 20 Feb 2009 10:00:21 +0100 Subject: [Box Backup] Error: Found conflicting parent ID for file ID In-Reply-To: References: <5DB1D2D8020D4449AF3111C784529E62470219@jeep.netcal.l> <200902191701.27370.ddmails@web.de> Message-ID: <200902201000.23637.ddmails@web.de> Hi Chris, Am Thursday 19 February 2009 23:37:22 schrieb Chris Wilson: > Hi Torsten, > > On Thu, 19 Feb 2009, Torsten wrote: > > After some backup runs the errors for "conflicting parent ID" are > > completely gone. > > I don't know why that would happen. The store data is supposed to be in > memory, so if the new build removed the error, then it should have taken > effect immediately. Are you using StoreObjectInfoFile? No i do not use StoreObjectInfoFile. But i think the first two backup processes where interrupted by network errors. So i think the first complete backup run fixed the errors. > > Only hundreds of this > > > > "WARNING: File ID 0xc938d has different container ID, probably moved" > > > > still exists. But you said this is no problem. > > It can happen in normal use, but it can also mean that the previous error > (conflicting parent ID) has corrupted the store. Did you run "compare > -aq"? I'd strongly recommend that you do that in order to trust your > backups again, and keep doing it regularly anyway. I ran the compare and it seems ok. Torsten From boxbackup at boxbackup.org Wed Feb 25 13:42:19 2009 From: boxbackup at boxbackup.org (berni) Date: Wed, 25 Feb 2009 14:42:19 +0100 Subject: [Box Backup] cleanup - deleted files Message-ID: <1235569339.9993.10.camel@zero> hi i have a problem with the svn version of boxbackup. There are a lot of "Deleted files" But they are using all the space on my server. I set the softlimit to a low value. I also uncommented the folder with huge files @client side but it does not help Output of bbstoreaccounts info 1 http://pastebin.ubuntu.com/122820/ i changed the time in the bbstored config and restarted bbstored. TimeBetweenHousekeeping = 10 but it is not going to delete anything !! the client config: BackupLocations { boot- { Path = /boot/ } etc- { Path = /etc/ } #home- #{ # Path = /home/ # ExcludeDir = /home/berni/Desktop/puchi # ExcludeDir = /home/berni/downloads/torrents #} when is the server doing the housekeeping ?? i had to reset the hard limit twice to a huger value because i got "Server is full" on the client. i want to get rid of this files. ps: by the way cool project. I'm part of a homeserver project (satega.org) maybe we will include this backup solution as our main backup method. yours berni From boxbackup at boxbackup.org Wed Feb 25 19:42:43 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Wed, 25 Feb 2009 19:42:43 +0000 (GMT) Subject: [Box Backup] Current status 0.11 In-Reply-To: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> References: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> Message-ID: Hi Dnk, On Thu, 19 Feb 2009, Dnk wrote: > I was curious, as to the current status of 0.11. I have been following > the list, and waiting to have a go. I am building a new backup server > and was hoping to have a crack at box backup. Now I read the disclaimers > about windows and such on the website and wiki. Is there any sort of eta > when this version may become stable? I know that is probably an annoying > and broad question. I'm afraid I can't answer that question just yet. 0.11rc2 is as close as we have got to a stable release of 0.11. Due to lack of time, I have not yet been able to analyse the changes made to trunk since then and merge them onto 0.11rc2 to produce a new release candidate 0.11rc3, which I would hope would become 0.11 after a short delay. So the best I can say is around 2 months from now, but without making any promises. > Also, is there any plans for an os x client? I wish I could code to help > out with that one. Unfortunately I am no programmer. It does already run on OS X as far as I know. Several people use it there and I spent a lot of time last year fixing the minor issues that causes problems with the unit tests until they all passed. 0.11rc2 includes all of that work. > Any plans for deduplication? No, sorry. Box Backup is intended for backing up user data, not whole filesystems, and as such deduplication should not even be necessary as far as I can see. > In my network, I am primarily backing up Linux servers, and windows > workstations. I have a few mac workstations as well. > > Due to my clients and the curret status, would I be better off to bypass > box backup at this time? We would be very happy if you would try 0.11rc2, which should work on these platforms, and let us know if you encounter any issues. Please use the temporary client release at http://www.boxbackup.org/releases/temporary/ on Windows clients, as it fixes an important bug. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Wed Feb 25 19:45:53 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Wed, 25 Feb 2009 19:45:53 +0000 (GMT) Subject: [Box Backup] cleanup - deleted files In-Reply-To: <1235569339.9993.10.camel@zero> References: <1235569339.9993.10.camel@zero> Message-ID: Hi Berni, On Wed, 25 Feb 2009, berni wrote: > i have a problem with the svn version of boxbackup. > > There are a lot of "Deleted files" But they are using all the space on > my server. > > I set the softlimit to a low value. I also uncommented the folder with > huge files @client side but it does not help That is very strange. If you set the soft limit below the amount of non-deleted and non-old data, then all old and deleted files should be removed on the next housekeeping run. If that does not happen, as it appears not to on your system, please enable debugging by running bbstored with the -V option, and enable your system logs to log debug level messages for Box Backup if necessary, and send us (or just me if you prefer) the logs from a housekeeping run. > when is the server doing the housekeeping ?? You should see "starting housekeeping" and "finished housekeeping" messages in your system logs even without enabling debugging. Please let us know if you do not, and include a copy of your /etc/syslog.conf. > ps: by the way cool project. I'm part of a homeserver project > (satega.org) maybe we will include this backup solution as our main > backup method. Thanks, I hope that you will. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Wed Feb 25 23:49:35 2009 From: boxbackup at boxbackup.org (dnk) Date: Wed, 25 Feb 2009 15:49:35 -0800 Subject: [Box Backup] Current status 0.11 In-Reply-To: References: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> Message-ID: <56D1D04F-0731-41E4-8CEE-4BBFFFACD3F5@gmail.com> On 25-Feb-09, at 11:42 AM, Chris Wilson wrote: > Hi Dnk, > > On Thu, 19 Feb 2009, Dnk wrote: > >> I was curious, as to the current status of 0.11. I have been >> following the list, and waiting to have a go. I am building a new >> backup server and was hoping to have a crack at box backup. Now I >> read the disclaimers about windows and such on the website and >> wiki. Is there any sort of eta when this version may become stable? >> I know that is probably an annoying and broad question. > > I'm afraid I can't answer that question just yet. 0.11rc2 is as > close as we have got to a stable release of 0.11. Due to lack of > time, I have not yet been able to analyse the changes made to trunk > since then and merge them onto 0.11rc2 to produce a new release > candidate 0.11rc3, which I would hope would become 0.11 after a > short delay. So the best I can say is around 2 months from now, but > without making any promises. Ok, That is doable providing the upgrade path is relatively simple. > >> Also, is there any plans for an os x client? I wish I could code to >> help out with that one. Unfortunately I am no programmer. > > It does already run on OS X as far as I know. Several people use it > there and I spent a lot of time last year fixing the minor issues > that causes problems with the unit tests until they all passed. > 0.11rc2 includes all of that work. Where can one download the other clients? I saw below that you sent a link for the win client. Are the linux/OS X clients just included with the main download? > >> Any plans for deduplication? > > No, sorry. Box Backup is intended for backing up user data, not > whole filesystems, and as such deduplication should not even be > necessary as far as I can see. Ah ok.... I was just curious, as in my case (only user data), I have a file server which has files on it, which some (probably multiple) users probably also have on their desktops... all of which would be backed up, hence multiple instances of the same files. Might still not be a bad idea.... Just a suggestion. > >> In my network, I am primarily backing up Linux servers, and windows >> workstations. I have a few mac workstations as well. >> >> Due to my clients and the curret status, would I be better off to >> bypass box backup at this time? > > We would be very happy if you would try 0.11rc2, which should work > on these platforms, and let us know if you encounter any issues. > Please use the temporary client release at http://www.boxbackup.org/releases/temporary/ > on Windows clients, as it fixes an important bug. I will do some testing. And see what comes of it. > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at boxbackup.org > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at boxbackup.org Wed Feb 25 23:59:49 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Wed, 25 Feb 2009 23:59:49 +0000 (GMT) Subject: [Box Backup] Current status 0.11 In-Reply-To: <56D1D04F-0731-41E4-8CEE-4BBFFFACD3F5@gmail.com> References: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> <56D1D04F-0731-41E4-8CEE-4BBFFFACD3F5@gmail.com> Message-ID: Hi Dnk, On Wed, 25 Feb 2009, dnk wrote: >> > I was curious, as to the current status of 0.11. I have been >> > following the list, and waiting to have a go. I am building a new >> > backup server and was hoping to have a crack at box backup. Now I >> > read the disclaimers about windows and such on the website and wiki. >> > Is there any sort of eta when this version may become stable? I know >> > that is probably an annoying and broad question. >> >> I'm afraid I can't answer that question just yet. 0.11rc2 is as close >> as we have got to a stable release of 0.11. Due to lack of time, I have >> not yet been able to analyse the changes made to trunk since then and >> merge them onto 0.11rc2 to produce a new release candidate 0.11rc3, >> which I would hope would become 0.11 after a short delay. So the best I >> can say is around 2 months from now, but without making any promises. > > Ok, That is doable providing the upgrade path is relatively simple. There should be no problems with upgrading from 0.10. bbackupd does not require any of the new commands implemented by the new bbstored, and nor does bbstored require anything new of bbackupd. The only exceptions are that bbackupquery 0.11 offers some new commands such as delete which won't work with an old 0.10 bbstored (but these are optional for the administrator to use or not), and that the bug with backing up files over 2 GB requires that both bbackupd and bbstored be upgraded to 0.11 to fix it (but does not specify the order that the upgrade must happen in). >> > Also, is there any plans for an os x client? I wish I could code to >> > help out with that one. Unfortunately I am no programmer. >> >> It does already run on OS X as far as I know. Several people use it >> there and I spent a lot of time last year fixing the minor issues that >> causes problems with the unit tests until they all passed. 0.11rc2 >> includes all of that work. > > Where can one download the other clients? I saw below that you sent a > link for the win client. Are the linux/OS X clients just included with > the main download? The source code to them all is included in the standard source download. Binary packages exist for a few different Unixes and for Windows, but not for OS X as far as I know, so you'll have to compile it yourself. >> > Any plans for deduplication? >> >> No, sorry. Box Backup is intended for backing up user data, not whole >> filesystems, and as such deduplication should not even be necessary as >> far as I can see. > > Ah ok.... I was just curious, as in my case (only user data), I have a > file server which has files on it, which some (probably multiple) users > probably also have on their desktops... all of which would be backed up, > hence multiple instances of the same files. Might still not be a bad > idea.... > > Just a suggestion. Thanks, suggestion taken. You can add it to the Feature Requests page on the wiki as well to make sure it doesn't get forgotten. (http://www.boxbackup.org/trac/wiki/FeatureRequests) >> > Due to my clients and the curret status, would I be better off to >> > bypass box backup at this time? >> >> We would be very happy if you would try 0.11rc2, which should work on >> these platforms, and let us know if you encounter any issues. Please >> use the temporary client release at >> http://www.boxbackup.org/releases/temporary/ on Windows clients, as it >> fixes an important bug. > > I will do some testing. And see what comes of it. OK, thanks. Please let us know if you have any issues. I am happy to help as much as my time permits. Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software | From boxbackup at boxbackup.org Thu Feb 26 00:16:44 2009 From: boxbackup at boxbackup.org (Dnk) Date: Wed, 25 Feb 2009 16:16:44 -0800 Subject: [Box Backup] Current status 0.11 In-Reply-To: References: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> <56D1D04F-0731-41E4-8CEE-4BBFFFACD3F5@gmail.com> Message-ID: <1D31DBEC-AA68-414B-8FE1-A85A897C67FA@gmail.com> Dustin Krysak Sent from my iPhone On 25-Feb-09, at 3:59 PM, Chris Wilson wrote: > Hi Dnk, > > On Wed, 25 Feb 2009, dnk wrote: > >>> > I was curious, as to the current status of 0.11. I have been > >>> following the list, and waiting to have a go. I am building a new >>> > backup server and was hoping to have a crack at box backup. Now >>> I > read the disclaimers about windows and such on the website and >>> wiki. > Is there any sort of eta when this version may become >>> stable? I know > that is probably an annoying and broad question. >>> I'm afraid I can't answer that question just yet. 0.11rc2 is as >>> close as we have got to a stable release of 0.11. Due to lack of >>> time, I have not yet been able to analyse the changes made to >>> trunk since then and merge them onto 0.11rc2 to produce a new >>> release candidate 0.11rc3, which I would hope would become 0.11 >>> after a short delay. So the best I can say is around 2 months from >>> now, but without making any promises. >> >> Ok, That is doable providing the upgrade path is relatively simple. > > There should be no problems with upgrading from 0.10. bbackupd does > not require any of the new commands implemented by the new bbstored, > and nor does bbstored require anything new of bbackupd. The only > exceptions are that bbackupquery 0.11 offers some new commands such > as delete which won't work with an old 0.10 bbstored (but these are > optional for the administrator to use or not), and that the bug with > backing up files over 2 GB requires that both bbackupd and bbstored > be upgraded to 0.11 to fix it (but does not specify the order that > the upgrade must happen in). Looks like I will have to go with 0.11rc2 as I have some users with 2+gb files (outlook pst). Is that link previously given for the win client compatible with 0.10 or 0.11 or both? > > >>> > Also, is there any plans for an os x client? I wish I could code >>> to > help out with that one. Unfortunately I am no programmer. >>> It does already run on OS X as far as I know. Several people use >>> it there and I spent a lot of time last year fixing the minor >>> issues that causes problems with the unit tests until they all >>> passed. 0.11rc2 includes all of that work. >> >> Where can one download the other clients? I saw below that you sent >> a link for the win client. Are the linux/OS X clients just included >> with the main download? > > The source code to them all is included in the standard source > download. Binary packages exist for a few different Unixes and for > Windows, but not for OS X as far as I know, so you'll have to > compile it yourself. Ok, will have to look into getting Xcode, etc setup. > > >>> > Any plans for deduplication? >>> No, sorry. Box Backup is intended for backing up user data, not >>> whole filesystems, and as such deduplication should not even be >>> necessary as far as I can see. >> >> Ah ok.... I was just curious, as in my case (only user data), I >> have a file server which has files on it, which some (probably >> multiple) users probably also have on their desktops... all of >> which would be backed up, hence multiple instances of the same >> files. Might still not be a bad idea.... >> >> Just a suggestion. > > Thanks, suggestion taken. You can add it to the Feature Requests > page on the wiki as well to make sure it doesn't get forgotten. (http://www.boxbackup.org/trac/wiki/FeatureRequests > ) Will do. > > >>> > Due to my clients and the curret status, would I be better off >>> to > bypass box backup at this time? >>> We would be very happy if you would try 0.11rc2, which should work >>> on these platforms, and let us know if you encounter any issues. >>> Please use the temporary client release at http://www.boxbackup.org/releases/temporary/ >>> on Windows clients, as it fixes an important bug. >> >> I will do some testing. And see what comes of it. > > OK, thanks. Please let us know if you have any issues. I am happy to > help as much as my time permits. I will do so. Thanks for the pointers. Dustin. > > > Cheers, Chris. > -- > _____ __ _ > \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | > / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | > \__/_/_/_//_/___/ | We are GNU : free your mind & your software | > _______________________________________________ > boxbackup mailing list > boxbackup at boxbackup.org > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at boxbackup.org Thu Feb 26 09:51:08 2009 From: boxbackup at boxbackup.org (Chris Wilson) Date: Thu, 26 Feb 2009 09:51:08 +0000 (GMT) Subject: [Box Backup] Current status 0.11 In-Reply-To: <1D31DBEC-AA68-414B-8FE1-A85A897C67FA@gmail.com> References: <7E6797E2-AC1B-480B-BF0D-FFA939998287@gmail.com> <56D1D04F-0731-41E4-8CEE-4BBFFFACD3F5@gmail.com> <1D31DBEC-AA68-414B-8FE1-A85A897C67FA@gmail.com> Message-ID: Hi Dnk, On Wed, 25 Feb 2009, Dnk wrote: > Looks like I will have to go with 0.11rc2 as I have some users with 2+gb > files (outlook pst). > > Is that link previously given for the win client compatible with 0.10 or > 0.11 or both? It's compatible with both, but it's a 0.11 client (more recent than 0.11rc2). Cheers, Chris. -- _____ __ _ \ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer | \__/_/_/_//_/___/ | We are GNU : free your mind & your software |