From boxbackup at fluffy.co.uk Mon Jul 3 23:25:25 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Mon, 3 Jul 2006 23:25:25 +0100 (BST) Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: <20060629211623.88498.qmail@web36707.mail.mud.yahoo.com> References: <20060629211623.88498.qmail@web36707.mail.mud.yahoo.com> Message-ID: Hi all, Please forgive me for jumping in late, and for asking what is probably a stupid question, but here goes: Would it make sense for the client to download, from the server, a list of blocks where the information supplied about each block is (1) the IV used previously by the client to encrypt that block, and (2) the current checksum on disk (on the server) of that block, freshly computed by the server? Would that be enough to allow the client to re-encrypt the block using the same IV, rather than a new IV, compute the checksum of the block, and thus determine whether its local copy was different to the one on the server? Unless of course some strange disk/memory corruption on the server or client just happened to result in the different encrypted block having the same checksum? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Tue Jul 4 11:44:11 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 4 Jul 2006 11:44:11 +0100 Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: References: <20060629211623.88498.qmail@web36707.mail.mud.yahoo.com> Message-ID: <4ADEA339-E844-47CD-93D0-DFD8FC69AB9C@fluffy.co.uk> On 3 Jul 2006, at 23:25, Chris Wilson wrote: > Hi all, > > Please forgive me for jumping in late, and for asking what is > probably a stupid question, but here goes: > > Would it make sense for the client to download, from the server, a > list of blocks where the information supplied about each block is > (1) the IV used previously by the client to encrypt that block, and > (2) the current checksum on disk (on the server) of that block, > freshly computed by the server? > > Would that be enough to allow the client to re-encrypt the block > using the same IV, rather than a new IV, compute the checksum of > the block, and thus determine whether its local copy was different > to the one on the server? Unless of course some strange disk/memory > corruption on the server or client just happened to result in the > different encrypted block having the same checksum? Yes, I think that would do the trick nicely. It doesn't require the server to be trusted any more than the minimal "store files and let us retrieve them intact" requirement. Ben From boxbackup at fluffy.co.uk Tue Jul 4 11:52:26 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Tue, 4 Jul 2006 11:52:26 +0100 Subject: [Box Backup] Storing file sizes server-side as meta data In-Reply-To: <20060629141543.a3c2wd3em84sw0gw@ebourne.me.uk> References: <02e801c69b77$10b68b50$1601060a@cs> <20060629141543.a3c2wd3em84sw0gw@ebourne.me.uk> Message-ID: <55A4B802-7A9E-4C4C-A2CF-B806985885B8@fluffy.co.uk> On 29 Jun 2006, at 14:15, Martin Ebourne wrote: > Stefan Norlin wrote: >> I was wondering whether it would be possible to store the >> original file sizes server-side in order to be able to see this >> information when browsing the store and possibly to calculate >> the amount of space required to restore files/directories? > > I think this is already stored in the directory entries. If not > then I would think it could be added. Only the client can see it, > but that's what you want anyway. You just need a feature in > bbackupquery that retrieves the directory and adds it up for you. It's probably stored as an incidental bit of data in the attributes. The attributes may not be in the directory, they're more likely to be in the stored file. So you'd need to write code to put attributes in the directory for all files, and then decrypt and display them. I have considered the exact file size a secret, although of course you can work it out to within a few percent. Ben From boxbackup at fluffy.co.uk Fri Jul 7 00:41:06 2006 From: boxbackup at fluffy.co.uk (Gary) Date: Thu, 6 Jul 2006 16:41:06 -0700 (PDT) Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: <4ADEA339-E844-47CD-93D0-DFD8FC69AB9C@fluffy.co.uk> Message-ID: <20060706234106.44578.qmail@web36709.mail.mud.yahoo.com> > > Would it make sense for the client to download, from the server, a > > list of blocks where the information supplied about each block is > > (1) the IV used previously by the client to encrypt that block, and > > (2) the current checksum on disk (on the server) of that block, > > freshly computed by the server? > Yes, I think that would do the trick nicely. The server actually stores those IVs? Interesting - what for? I presume they are stored on the serever in an encrypted state? I missed that when looking through the sources. > It doesn't require the server to be trusted any more than the minimal > "store files and let us retrieve them intact" requirement. The alternative of the client sending up to the server the ciphertext along with strong checksums for the ciphertext, to be stored and compared by the server later on, would also allow for an end-to-end bbstoredcheckaccount (without requiring client cooperation). G. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Fri Jul 7 09:18:15 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Fri, 7 Jul 2006 09:18:15 +0100 Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: <20060706234106.44578.qmail@web36709.mail.mud.yahoo.com> References: <20060706234106.44578.qmail@web36709.mail.mud.yahoo.com> Message-ID: On 7 Jul 2006, at 00:41, Gary wrote: >>> Would it make sense for the client to download, from the server, a >>> list of blocks where the information supplied about each block is >>> (1) the IV used previously by the client to encrypt that block, and >>> (2) the current checksum on disk (on the server) of that block, >>> freshly computed by the server? > >> Yes, I think that would do the trick nicely. > > The server actually stores those IVs? Interesting - what for? Just to send back to the client later. Knowing the IV is very useful when it comes to decrypting the data. > I presume they are stored on the > serever in an encrypted state? No, in the clear. There's no need to encrypt them, they don't give any information which is useful to an attacker. All you're saying is that the secret information in the first block is XORed with this value, which leaves them none the wiser about the value of the plaintext. (But of course stops attacks based on comparing cipertext blocks.) > > I missed that when looking through the sources. Check out BackupStoreFile::EncodeChunk() So, basically what you'd need to do is, for each block, send IV, encrypted length, unencrypted length, and ciphertext hash. That should allow you to encrypt each block in turn with the given IV, do the hash on the client, and compare. You will of course need to send the block index because the unencrypted length of the block is stored in the encrypted section of the block index. (see BackupStoreFileWire.h) Now there is a slight potential problems with this. Most blocks will be compressed, so you are replying on zlib emitting the same output every time it compresses the same data. Now this should work, but a change in zlib could give different output and screw up this comparison process. We could implement our own compressor. zlib may not be the best choice of compression, because it only operates on small blocks. Perhaps a more simple compressor could give just as good performance? > >> It doesn't require the server to be trusted any more than the minimal >> "store files and let us retrieve them intact" requirement. > > The alternative of the client sending up to the server the > ciphertext along with strong checksums > for the ciphertext, to be stored and compared by the server later > on, would also allow for an > end-to-end bbstoredcheckaccount (without requiring client > cooperation). I'm not sure I quite follow what you're suggesting. Ben From boxbackup at fluffy.co.uk Fri Jul 7 18:32:02 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Fri, 7 Jul 2006 18:32:02 +0100 (BST) Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: <20060706234106.44578.qmail@web36709.mail.mud.yahoo.com> References: <20060706234106.44578.qmail@web36709.mail.mud.yahoo.com> Message-ID: Hi Gary, On Thu, 6 Jul 2006, Gary wrote: > The alternative of the client sending up to the server the ciphertext > along with strong checksums for the ciphertext, to be stored and > compared by the server later on, would also allow for an end-to-end > bbstoredcheckaccount (without requiring client cooperation). Why even do that? The server can compute the checksum of the ciphertext by itself. It could write the checksum to disk alongside the encrypted data, and "bbstoreaccounts check" could verify that the encrypted block still hashes to the same checksum, and thus the encrypted data was not damaged on the server (however unlikely that might be). That proposal is pretty orthogonal to the fast client verify proposal, and it would seem to make sense to implement both. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Fri Jul 7 21:15:28 2006 From: boxbackup at fluffy.co.uk (Gary) Date: Fri, 7 Jul 2006 13:15:28 -0700 (PDT) Subject: RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf) In-Reply-To: Message-ID: <20060707201528.75016.qmail@web36701.mail.mud.yahoo.com> Ben, Chris, >> The alternative of the client sending up to the server the ciphertext >> along with strong checksums for the ciphertext, to be stored and >> compared by the server later on, would also allow for an end-to-end >> bbstoredcheckaccount (without requiring client cooperation). > I'm not sure I quite follow what you're suggesting. The way I see it, there are two ways to guarantee end-to-end data integrity to a very high statistical probability: A.) IV + re-encryption: - (compare) client downloads ciphertext IVs and ciphertext checksums from the server - (compare) client encrypts plaintext with the IV, compresses it, and compares resulting ciphertext against server ciphertext checksum - (bbstroedcheckaccount) <> B.) ciphertext-level checksum - (backup) client uploads ciphertext checksum to the server (along with ciphertext and plaintext checksum) - (compare) client issues a command to the server to compare ciphertext checksum to what's on the media Since the plaintext checksum matches local client content (-aq) and ciphertext checksum matches server content, we have end-to-end data integrity, unless both server media content and ciphertext checksum get corrupted in such a way, as to match each other (wonder how possible that is...). - (bbstroedcheckaccount) server compares all ciphertext checksums to what's on the media > Now there is a slight potential problems with this. Most blocks will > be compressed, so you are replying on zlib emitting the same output > every time it compresses the same data. Now this should work, but a > change in zlib could give different output and screw up this > comparison process. Yes, when it comes to option (A.), since there is a need to re-encrypt and re-compress before compare. Option (B.) does not have this problem, since it works with the ciphertext alone. However, option (B.) would require changing internal repository layout (to accommodate the storage of ciphertext checksums). Even if we make it backward-compatible, it is useless unless a client re-backups everything to re-submit ciphertext checksums. > Why even do that? The server can compute the checksum of the ciphertext by > itself. It could write the checksum to disk alongside the encrypted data, > and "bbstoreaccounts check" could verify that the encrypted block still > hashes to the same checksum, and thus the encrypted data was not damaged > on the server (however unlikely that might be). Yes, only I augmented the idea with the client generating ciphertext checksums and sending them along with the ciphertext in the first place (as opposed to the server generating ciphertext checksums once it receives the ciphertext from the client), since that method would also catch any kind of errors with transmission, server memory, server disks, server logic, the works. Effectively, a client says: "the plaintext checksum is what this ciphertext MEANS to me, the ciphertext checksum is the CONTENT I expect back from you in the future". I might be over-doing this, though. Gary __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From boxbackup at fluffy.co.uk Tue Jul 18 16:08:27 2006 From: boxbackup at fluffy.co.uk (John MacKenzie) Date: Tue, 18 Jul 2006 11:08:27 -0400 Subject: [Box Backup] Win32 ExcludeFile Syntax Message-ID: <7.0.1.0.2.20060718110528.0486bc98@elehost.com> Hello, Thanks for the great program. I am having trouble figuring out how to exclude a file with the win32 client (latest version) I have in the config file: ExcludeFile = C:\Program Files\Box Backup\12AB23BA-FileEncKeys.raw But it still backs it up. Can someone please tell me where this syntax is wrong? Thanks John From boxbackup at fluffy.co.uk Tue Jul 18 21:07:17 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Tue, 18 Jul 2006 13:07:17 -0700 (PDT) Subject: [Box Backup] Win32 ExcludeFile Syntax In-Reply-To: <7.0.1.0.2.20060718110528.0486bc98@elehost.com> Message-ID: <20060718200717.34502.qmail@web60625.mail.yahoo.com> Hi John, That should work. What version are you running? Maybe sending us your full BackupLocations section might reveal something? Or maybe try a Regex like I do: MyDocs { Path = C:\Documents and Settings\Mom\My Documents ExcludeFilesRegex = .*\\thumbs\.db$ } Maybe it's a, um, feature in ExcludeFile that you need to escape the slashes like in ExcludeFilesRegex like I do above? Are sure that it is "still" backing it up, or did it back it up previously and hasn't deleted it yet? Good luck, Pete --- John MacKenzie wrote: > Hello, > > Thanks for the great program. I am having trouble figuring out how to > > exclude a file with the win32 client (latest version) > > I have in the config file: > > ExcludeFile = C:\Program Files\Box Backup\12AB23BA-FileEncKeys.raw > > But it still backs it up. Can someone please tell me where this > syntax is wrong? > > Thanks > > John > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Wed Jul 19 14:59:34 2006 From: boxbackup at fluffy.co.uk (John MacKenzie) Date: Wed, 19 Jul 2006 09:59:34 -0400 Subject: [Box Backup] ExcludeFile command being ignored with Win32 client. In-Reply-To: <20060719110001.8135.56527.Mailman@love.warhead.org.uk> References: <20060719110001.8135.56527.Mailman@love.warhead.org.uk> Message-ID: <7.0.1.0.2.20060719095550.04bf4178@elehost.com> Thanks for the suggestions Pete. I actually set up a whole new box to test this theory and the command is still being ignored. I am using the latest precompiled W32 (538) build on the web site and the server has .10 (I believe the latest version as well). Here is my configuration BackupLocations { CDrive { Path = C:\ ExcludeFile = C:\Program Files\Box Backup\85AB23A-FileEncKeys.raw ExcludeFile = C:\pagefile.sys ExcludeDir = C:\windows\Temp\ ExcludeDir = C:\Temp\ ExcludeDir = C:\windows\system32\LogFiles\ ExcludeDir = C:\Documents and Settings\Default User.WINNT\Local Settings\Temporary Internet Files\ } } Do you see any problems? The ExcludeDir commands seem to work fine but both ExcludeFile commands do not (every time it stops and starts the service it tries to backup the pagefile.sys which gives an error). I also tried ExcludeFile = C:\\pagefile.sys (but that did not work as well) Thanks John From boxbackup at fluffy.co.uk Wed Jul 19 15:03:03 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Wed, 19 Jul 2006 15:03:03 +0100 Subject: [Box Backup] ExcludeFile command being ignored with Win32 client. In-Reply-To: <7.0.1.0.2.20060719095550.04bf4178@elehost.com> References: <20060719110001.8135.56527.Mailman@love.warhead.org.uk> <7.0.1.0.2.20060719095550.04bf4178@elehost.com> Message-ID: <43DCFC75-EC2B-496D-B560-6310F6E0654D@fluffy.co.uk> What about using / as the separator? Ben On 19 Jul 2006, at 14:59, John MacKenzie wrote: > Thanks for the suggestions Pete. > > I actually set up a whole new box to test this theory and the > command is still being ignored. I am using the latest precompiled > W32 (538) build on the web site and the server has .10 (I believe > the latest version as well). > > Here is my configuration > > BackupLocations > { > CDrive > { > Path = C:\ > ExcludeFile = C:\Program Files\Box Backup\85AB23A- > FileEncKeys.raw > ExcludeFile = C:\pagefile.sys > ExcludeDir = C:\windows\Temp\ > ExcludeDir = C:\Temp\ > ExcludeDir = C:\windows\system32\LogFiles\ > ExcludeDir = C:\Documents and Settings\Default > User.WINNT\Local Settings\Temporary Internet Files\ > } > > } > > Do you see any problems? The ExcludeDir commands seem to work fine > but both ExcludeFile commands do not (every time it stops and > starts the service it tries to backup the pagefile.sys which gives > an error). > > I also tried > > ExcludeFile = C:\\pagefile.sys > > (but that did not work as well) > > Thanks > > John > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Wed Jul 19 19:02:36 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Wed, 19 Jul 2006 19:02:36 +0100 (BST) Subject: [Box Backup] ExcludeFile command being ignored with Win32 client. In-Reply-To: <7.0.1.0.2.20060719095550.04bf4178@elehost.com> References: <20060719110001.8135.56527.Mailman@love.warhead.org.uk> <7.0.1.0.2.20060719095550.04bf4178@elehost.com> Message-ID: Hi John, > Do you see any problems? The ExcludeDir commands seem to work fine but both > ExcludeFile commands do not (every time it stops and starts the service it > tries to backup the pagefile.sys which gives an error). This looks like a bug. I will investigate and get back to you. Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Jul 20 06:17:38 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Wed, 19 Jul 2006 22:17:38 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? Message-ID: <20060720051738.25676.qmail@web60621.mail.yahoo.com> Hi all, I have a client who has a Windows Small Business Server network. The backup client is boxbackup-chris_general_568-backup-client-mingw32.zip on a Windows 2003 Standard machine. Box backup is working great when we stay on the local machine, but we are trying to create a BackupLocation for files that reside on another one of various Windows machines on the network, but we keep getting the 1/9 exception (and nothing gets backed up from any BackupLocation), whether we use a mapped drive letter path as a BackupLocation or a UNC path. Neither of these BackupLocations worked: BackupLocations { DocsSettings { Path = e:\Documents and Settings } CDesktop { Path = \\cdesktop\desktop$\Documents and Settings } } BackupLocations { DocsSettings { Path = e:\Documents and Settings } CDesktop { Path = T:\ } } Can this be done? If so, he'd prefer to use UNC paths, if possible, rather than clutter up his mapped-drive list with several mappings that would only be used for box backup. This is the best I could find: http://lists.warhead.org.uk/pipermail/boxbackup/2005-April/001339.html and that might be too old. (For the record, due to some OS install disk partitioning funkiness, his Box Backup app and bbackupd.conf are installed in E:\Program Files\Box Backup and it works fine. His OS is installed in E:, too, not C: as is customary.) Thanks! Pete From boxbackup at fluffy.co.uk Thu Jul 20 09:04:22 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Thu, 20 Jul 2006 09:04:22 +0100 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060720051738.25676.qmail@web60621.mail.yahoo.com> References: <20060720051738.25676.qmail@web60621.mail.yahoo.com> Message-ID: On 20 Jul 2006, at 06:17, E.W. Peter Jalajas wrote: > Hi all, > > I have a client who has a Windows Small Business Server network. The > backup client is boxbackup-chris_general_568-backup-client-mingw32.zip > on a Windows 2003 Standard machine. Box backup is working great when > we stay on the local machine, but we are trying to create a > BackupLocation for files that reside on another one of various Windows > machines on the network, but we keep getting the 1/9 exception (and > nothing gets backed up from any BackupLocation), whether we use a > mapped drive letter path as a BackupLocation or a UNC path. Box Backup is designed to back up local discs only. You should run bbackupd on every Windows machine which needs backing up. One thing which is more of a problem on network mounts than local discs is that if the network mount fails for some reasons when a backup starts, bbackupd will mark everything on the store as deleted. Ben From boxbackup at fluffy.co.uk Thu Jul 20 16:37:19 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 08:37:19 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: Message-ID: <20060720153719.65058.qmail@web60612.mail.yahoo.com> OK, thanks, Ben. I see this now on: http://boxbackup.hostworks.ca/index.php/Feature_requests "Add an option like (in the client's bbackupd.conf config file) "KeepIfNotFound = true" to a backup location; this should make boxbackup keep a directory on the server (without complaining) even if it's not found on the local drive anymore (this will help when you backup a network location or another directory that may become unavailable)" I'll add to that to make a separate test for whether the filesystem exists at all, and if not, don't mark for deletion (I think). Thanks, Ben, Pete --- Ben Summers wrote: > On 20 Jul 2006, at 06:17, E.W. Peter Jalajas wrote: > > > Hi all, > > > > I have a client who has a Windows Small Business Server network. > The > > backup client is > boxbackup-chris_general_568-backup-client-mingw32.zip > > on a Windows 2003 Standard machine. Box backup is working great > when > > we stay on the local machine, but we are trying to create a > > BackupLocation for files that reside on another one of various > Windows > > machines on the network, but we keep getting the 1/9 exception (and > > nothing gets backed up from any BackupLocation), whether we use a > > mapped drive letter path as a BackupLocation or a UNC path. > > Box Backup is designed to back up local discs only. You should run > bbackupd on every Windows machine which needs backing up. > > One thing which is more of a problem on network mounts than local > discs is that if the network mount fails for some reasons when a > backup starts, bbackupd will mark everything on the store as deleted. > > Ben > > > > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 16:47:02 2006 From: boxbackup at fluffy.co.uk (Andrei) Date: Thu, 20 Jul 2006 18:47:02 +0300 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: References: <20060720051738.25676.qmail@web60621.mail.yahoo.com> Message-ID: Actually, my windows Chris client aborts with an error if a top level directory on another machine (it's on a mounted drive eg. z:\) cannot be accessed, it does not mark them for deletion. On 7/20/06, Ben Summers wrote: > On 20 Jul 2006, at 06:17, E.W. Peter Jalajas wrote: > > > Hi all, > > > > I have a client who has a Windows Small Business Server network. The > > backup client is boxbackup-chris_general_568-backup-client-mingw32.zip > > on a Windows 2003 Standard machine. Box backup is working great when > > we stay on the local machine, but we are trying to create a > > BackupLocation for files that reside on another one of various Windows > > machines on the network, but we keep getting the 1/9 exception (and > > nothing gets backed up from any BackupLocation), whether we use a > > mapped drive letter path as a BackupLocation or a UNC path. > > Box Backup is designed to back up local discs only. You should run > bbackupd on every Windows machine which needs backing up. > > One thing which is more of a problem on network mounts than local > discs is that if the network mount fails for some reasons when a > backup starts, bbackupd will mark everything on the store as deleted. > > Ben > > > > > > > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 16:53:59 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 08:53:59 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: Message-ID: <20060720155359.3441.qmail@web60619.mail.yahoo.com> Hi Andrei, You ARE backing up from a another machine?! If so, please let me the specifics on how you did it! Thanks, Pete --- Andrei wrote: > Actually, my windows Chris client aborts with an error if a top level > directory on another machine (it's on a mounted drive eg. z:\) cannot > be accessed, it does not mark them for deletion. > From boxbackup at fluffy.co.uk Thu Jul 20 17:23:01 2006 From: boxbackup at fluffy.co.uk (Andrei) Date: Thu, 20 Jul 2006 19:23:01 +0300 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060720155359.3441.qmail@web60619.mail.yahoo.com> References: <20060720155359.3441.qmail@web60619.mail.yahoo.com> Message-ID: Well, I first mounted the other computer's network share to drive z:\ (That's in Control Panel/Network Connections/Tools/Map Network Drive - you enter a drive letter and a network path eg. "\\OTHERCOMP\myshare\") Then I entered in BoxBackup's config file the path for a backed up item (eg. z:\files\) Regards On 7/20/06, E.W. Peter Jalajas wrote: > Hi Andrei, > > You ARE backing up from a another machine?! If so, please let me the > specifics on how you did it! > > Thanks, > Pete > > --- Andrei wrote: > > > Actually, my windows Chris client aborts with an error if a top level > > directory on another machine (it's on a mounted drive eg. z:\) cannot > > be accessed, it does not mark them for deletion. > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 17:25:17 2006 From: boxbackup at fluffy.co.uk (Andrei) Date: Thu, 20 Jul 2006 19:25:17 +0300 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060720155359.3441.qmail@web60619.mail.yahoo.com> References: <20060720155359.3441.qmail@web60619.mail.yahoo.com> Message-ID: PS: Also, I noticed that you must not add a backslash after the last directory: z:\files instead of z:\files\ Regards On 7/20/06, E.W. Peter Jalajas wrote: > Hi Andrei, > > You ARE backing up from a another machine?! If so, please let me the > specifics on how you did it! > > Thanks, > Pete > > --- Andrei wrote: > > > Actually, my windows Chris client aborts with an error if a top level > > directory on another machine (it's on a mounted drive eg. z:\) cannot > > be accessed, it does not mark them for deletion. > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 17:33:10 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 09:33:10 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: Message-ID: <20060720163310.39353.qmail@web60625.mail.yahoo.com> Yeah, that's what we did last night, and we kept getting the 1/9 error (two different remote machines, a small handful of different subdirectories). Just flailing: Is there something special about how your share out the directory on the the remote machine? Are you in a workgroup or domain? Simple file sharing? Special DNS/NetBIOS/DHCP settings? Maybe we have a firewall issue on one or both of our machines. --- Andrei wrote: > Well, I first mounted the other computer's network share to drive z:\ > (That's in Control Panel/Network Connections/Tools/Map Network Drive > - > you enter a drive letter and a network path eg. > "\\OTHERCOMP\myshare\") > Then I entered in BoxBackup's config file the path for a backed up > item (eg. z:\files) From boxbackup at fluffy.co.uk Thu Jul 20 17:42:26 2006 From: boxbackup at fluffy.co.uk (Andrei) Date: Thu, 20 Jul 2006 19:42:26 +0300 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060720163310.39353.qmail@web60625.mail.yahoo.com> References: <20060720163310.39353.qmail@web60625.mail.yahoo.com> Message-ID: The computers are in a workgroup. No special settings, just a simple home network. Anyway, I don't think this has any relevance, if you can access the drive in explorer, so should BoxBackup. I remember I kept getting an error until I removed the last backslash (after the last directory in the path). Regards On 7/20/06, E.W. Peter Jalajas wrote: > Yeah, that's what we did last night, and we kept getting the 1/9 error > (two different remote machines, a small handful of different > subdirectories). > > Just flailing: Is there something special about how your share out the > directory on the the remote machine? Are you in a workgroup or domain? > Simple file sharing? Special DNS/NetBIOS/DHCP settings? Maybe we have > a firewall issue on one or both of our machines. > > --- Andrei wrote: > > > Well, I first mounted the other computer's network share to drive z:\ > > (That's in Control Panel/Network Connections/Tools/Map Network Drive > > - > > you enter a drive letter and a network path eg. > > "\\OTHERCOMP\myshare\") > > Then I entered in BoxBackup's config file the path for a backed up > > item (eg. z:\files) > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 17:52:23 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 09:52:23 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: Message-ID: <20060720165223.83845.qmail@web60621.mail.yahoo.com> Hmm. I guess we'll keep banging our heads against it. We didn't have a trailing slash. I don't think we had any spaces in at least one of our BackupLocations (not sure if it matters). We noticed that, due to some kind of permissions issue) we had read, but not write, access to the remote directory (the z:\files directory, e.g.)--that shouldn't matter, should it? All this discussion in the context of Ben's warning that missing network drives will be marked for deletion, and presumably force a complete reupload of the entire remote BackupLocation when the remote filesystem comes back on line. --- Andrei wrote: > The computers are in a workgroup. No special settings, just a simple > home network. Anyway, I don't think this has any relevance, if you > can > access the drive in explorer, so should BoxBackup. I remember I kept > getting an error until I removed the last backslash (after the last > directory in the path). > > Regards > > On 7/20/06, E.W. Peter Jalajas wrote: > > Yeah, that's what we did last night, and we kept getting the 1/9 > error > > (two different remote machines, a small handful of different > > subdirectories). > > > > Just flailing: Is there something special about how your share out > the > > directory on the the remote machine? Are you in a workgroup or > domain? > > Simple file sharing? Special DNS/NetBIOS/DHCP settings? Maybe we > have > > a firewall issue on one or both of our machines. > > > > --- Andrei wrote: > > > > > Well, I first mounted the other computer's network share to drive > z:\ > > > (That's in Control Panel/Network Connections/Tools/Map Network > Drive > > > - > > > you enter a drive letter and a network path eg. > > > "\\OTHERCOMP\myshare\") > > > Then I entered in BoxBackup's config file the path for a backed > up > > > item (eg. z:\files) > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > From boxbackup at fluffy.co.uk Thu Jul 20 19:16:36 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 11:16:36 -0700 (PDT) Subject: [Box Backup] ExcludeFile command being ignored with Win32 client. In-Reply-To: <7.0.1.0.2.20060719095550.04bf4178@elehost.com> Message-ID: <20060720181636.63587.qmail@web60615.mail.yahoo.com> Sorry, John, all I can recommend for the moment is the workaround to fake an ExcludeFile with a carefully crafted ExcludeFilesRegex. Good luck. Let me know if you need help with that. Pete --- John MacKenzie wrote: > Thanks for the suggestions Pete. > > BackupLocations > { > CDrive > { > Path = C:\ > ExcludeFile = C:\Program Files\Box > Backup\85AB23A-FileEncKeys.raw > ExcludeFile = C:\pagefile.sys > ExcludeDir = C:\windows\Temp\ > ExcludeDir = C:\Temp\ > ExcludeDir = C:\windows\system32\LogFiles\ > ExcludeDir = C:\Documents and Settings\Default > User.WINNT\Local Settings\Temporary Internet Files\ > } > > } > > Do you see any problems? The ExcludeDir commands seem to work fine > but both ExcludeFile commands do not (every time it stops and starts > the service it tries to backup the pagefile.sys which gives an > error). From boxbackup at fluffy.co.uk Thu Jul 20 21:46:41 2006 From: boxbackup at fluffy.co.uk (Chris Wilson) Date: Thu, 20 Jul 2006 21:46:41 +0100 (BST) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060720165223.83845.qmail@web60621.mail.yahoo.com> References: <20060720165223.83845.qmail@web60621.mail.yahoo.com> Message-ID: Hi Pete, On Thu, 20 Jul 2006, E.W. Peter Jalajas wrote: > We noticed that, due to some kind of permissions issue) we had read, > but not write, access to the remote directory (the z:\files directory, > e.g.)--that shouldn't matter, should it? It shouldn't, but I think there is a bug in the Win32 client where it opens files for writing even when only read permissions are necessary. What is the exact error that you get when Box tries to back up this location? Cheers, Chris. -- _ ___ __ _ / __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK | / (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer | \ _/_/_/_//_/___/ | We are GNU-free your mind-and your software | From boxbackup at fluffy.co.uk Thu Jul 20 22:06:17 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Thu, 20 Jul 2006 14:06:17 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: Message-ID: <20060720210617.74829.qmail@web60618.mail.yahoo.com> Hi Chris, We'll be working on it again tonight (10pm Eastern US). The exception was: (1/9) - Common OSFileError - Error accessing a file. Check permissions. So, in the meantime, inspired by Andrei and your comment, we'll work tonight on getting write permissions to the remote box. Thanks, Chris. Let me know how I can help. Pete --- Chris Wilson wrote: > > It shouldn't, but I think there is a bug in the Win32 client where it > opens files for writing even when only read permissions are > necessary. > What is the exact error that you get when Box tries to back up this > location? From boxbackup at fluffy.co.uk Fri Jul 21 08:18:11 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Fri, 21 Jul 2006 08:18:11 +0100 Subject: [Box Backup] Can Win32 client backup files from another machine? Message-ID: <20060721071811.GA23770@hickinbottom.demon.co.uk> I've not been following this thread too closely, so apologies if I'm covering old ground. If you've got Box Backup running as a Windows service then it will not see any shares that you have created from your windows account since shares are a per-account setting. That might account for the service apparently being unable to read any files over the share. A workaround is to configure the service to "run as" the account where you set the share up, then when the service starts it logs on as that user and thus has access to his shares. Just an idea in case it helps... Stuart From boxbackup at fluffy.co.uk Fri Jul 21 18:26:20 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Fri, 21 Jul 2006 10:26:20 -0700 (PDT) Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060721071811.GA23770@hickinbottom.demon.co.uk> Message-ID: <20060721172620.2138.qmail@web60624.mail.yahoo.com> Hi Stuart, Thanks for your thoughts. I just got off the phone with my partner and he had the idea to setup a new admin-group user called "backup" on all windows machines in the workgroup/domain, and setup all the services and shares as that admin user; we're going to try that early next week. That may be essentially what you're suggesting, Stuart. But I wonder if your "run as" trick could be extendible to reach out to several different machines in the workgroup/domain? Your thoughts on that? I guess, twisting this idea back on itself, could we set up the "central box backup client machine admin user" as an admin with Full Access rights on the other machines in the workgroup/domain (kinda like a Domain Admin having Full Access rights on the local machines). (I just want to remind any newer Box Backup readers out there that this configuration is explicitly NOT recommended. I believe this dis-recommendation is primarily because it will most assuredly be extremely wasteful due to repetitive "mark as deleted" and re-uploads of entire backup stores as the workgroup computers disappear and re-appear from the view of the central Box Backup client.) Thanks again, Stuart, Pete --- Stuart Hickinbottom wrote: > A workaround is to configure the service to "run as" the account > where > you set the share up, then when the service starts it logs on as that > user and thus has access to his shares. From boxbackup at fluffy.co.uk Wed Jul 26 03:23:54 2006 From: boxbackup at fluffy.co.uk (Dave Thomas) Date: Tue, 25 Jul 2006 19:23:54 -0700 Subject: [Box Backup] Problems with compile on RH9 Message-ID: <200607251923.54146.dave@linuxsense.net> Greetings list! I have about 20 linux boxes that I need to settle on a backup solution for, and so far I really like box backup, but I have a few issues. Main issue is we have a few old RH7.2 boxes and some RH9 boxes I need to install box backup on but I am having problems getting it to compile. I'll skip the RH 7.2 details for now and get into the RH9 issues since I can just rsync the 7.2 systems if I need to. The issue I am having is when trying to 'make'..I get: : undefined reference to `X509_STORE_CTX_cleanup' collect2: ld returned 1 exit status make[1]: *** [../../release/bin/bbackupd/bbackupd] Error 1 make[1]: Leaving directory `/root/boxbackup-0.10/bin/bbackupd' make: *** [parcels/boxbackup-0.10-backup-client-linux-gnu.tgz] Error 2 I am using a manual install of OpenSSL (0.9.7a) with the following config: ./configure --enable-gnu-readline --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/lib Anyone have any info on this problem? Thanks, Dave Thomas dave at linuxsense.net From boxbackup at fluffy.co.uk Thu Jul 27 00:30:23 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Thu, 27 Jul 2006 00:30:23 +0100 Subject: [Box Backup] Problems with compile on RH9 In-Reply-To: <200607251923.54146.dave@linuxsense.net> References: <200607251923.54146.dave@linuxsense.net> Message-ID: <1153956624.25020.2.camel@avenin.ebourne.me.uk> On Tue, 2006-07-25 at 19:23 -0700, Dave Thomas wrote: > The issue I am having is when trying to 'make'..I get: > > : undefined reference to `X509_STORE_CTX_cleanup' > collect2: ld returned 1 exit status > make[1]: *** [../../release/bin/bbackupd/bbackupd] Error 1 > make[1]: Leaving directory `/root/boxbackup-0.10/bin/bbackupd' > make: *** [parcels/boxbackup-0.10-backup-client-linux-gnu.tgz] Error 2 > > I am using a manual install of OpenSSL (0.9.7a) with the following config: > > ./configure --enable-gnu-readline --with-ssl-headers=/usr/local/ssl/include --with-ssl-lib=/usr/local/ssl/lib > > Anyone have any info on this problem? Probably mismatch between header files and found library when linking. Check the compile and link lines, make sure the -I and -L options for ssl match up. Also try taking the gcc link line where it failed and adding -Xlinker --verbose. Should tell you what files it finds where. Cheers, Martin. From boxbackup at fluffy.co.uk Thu Jul 27 06:33:11 2006 From: boxbackup at fluffy.co.uk (Dave Thomas) Date: Wed, 26 Jul 2006 22:33:11 -0700 Subject: [Box Backup] speed up initial sync? Message-ID: <200607262233.11187.dave@linuxsense.net> I am experiencing very long initial sync times with box backup that I suspect is due to the extremely high number of small files we use in our application. Its not unusual for us to have well over 1 million small files in a dir (yea...we need to...don't ask..its some crazy legacy stuff) and while backup times have always been long due to all those small files I am seeing initial sync times over well over 24 hours with box backup on some of these systems. I can deal with the long initial sync times of these systems if I have to, but anything longer than 12 hours is eating into 'work time', so I would like to speed things up if possible. My question: Short of tossing cubic dollars at new hardware, is there anyway I can speed up the initial sync process? Thanks for your time, Dave From boxbackup at fluffy.co.uk Thu Jul 27 08:48:26 2006 From: boxbackup at fluffy.co.uk (Martin Ebourne) Date: Thu, 27 Jul 2006 08:48:26 +0100 Subject: [Box Backup] speed up initial sync? In-Reply-To: <200607262233.11187.dave@linuxsense.net> References: <200607262233.11187.dave@linuxsense.net> Message-ID: <1153986506.25020.5.camel@avenin.ebourne.me.uk> On Wed, 2006-07-26 at 22:33 -0700, Dave Thomas wrote: > I am experiencing very long initial sync times with box backup that I suspect > is due to the extremely high number of small files we use in our application. > Its not unusual for us to have well over 1 million small files in a dir > (yea...we need to...don't ask..its some crazy legacy stuff) and while backup > times have always been long due to all those small files I am seeing initial > sync times over well over 24 hours with box backup on some of these systems. > I can deal with the long initial sync times of these systems if I have to, > but anything longer than 12 hours is eating into 'work time', so I would like > to speed things up if possible. > > My question: Short of tossing cubic dollars at new hardware, is there anyway I > can speed up the initial sync process? Hack code? Cheaper than new hardware. :) This is a known problem with current versions of Box which has been covered on the list before. One day it will get addressed, but I don't think any of the contributors have experienced this so there's not been a lot of incentive to fix it. Patches welcome. :) Cheers, Martin. From boxbackup at fluffy.co.uk Thu Jul 27 23:04:24 2006 From: boxbackup at fluffy.co.uk (Stuart Hickinbottom) Date: Thu, 27 Jul 2006 23:04:24 +0100 Subject: [Box Backup] Can Win32 client backup files from another machine? In-Reply-To: <20060721172620.2138.qmail@web60624.mail.yahoo.com> References: <20060721172620.2138.qmail@web60624.mail.yahoo.com> Message-ID: <44C93868.5000309@hickinbottom.demon.co.uk> Yes, I think that is pretty much the same thing as I was trying to describe. I'm not too sure what you're asking about the "run as" trick. If you mean whether one client could draw files from many other machines rather than just one then yes it can if you connect to multiple shares from your "backup" user then add those connected drives as separate roots to the bbackupd.conf file. If your "backup" user is a full admin user then you might be able to get away without having to set up any shares at all by using the normal "hidden drive" shares, eg: \\somemachine\c$ Or \\somemachine\c$\somefolder These are present on every Windows machine so you'll not need to set up anything at all on the client as long as the "backup" user is an administrator (domain or otherwise, I think). Again, not too sure if that's what you're asking. Also, as you say, not recommended because bbackupd might think you've deleted all those files and mark them as deleted on the server (I've encountered similar things for mounted drives in Linux, and it's not too entertaining when your 500GB archive is at the other end of a DSL link!). Stuart E.W. Peter Jalajas wrote: > Hi Stuart, > > Thanks for your thoughts. I just got off the phone with my partner and > he had the idea to setup a new admin-group user called "backup" on all > windows machines in the workgroup/domain, and setup all the services > and shares as that admin user; we're going to try that early next week. > > > That may be essentially what you're suggesting, Stuart. But I wonder > if your "run as" trick could be extendible to reach out to several > different machines in the workgroup/domain? Your thoughts on that? > > I guess, twisting this idea back on itself, could we set up the > "central box backup client machine admin user" as an admin with Full > Access rights on the other machines in the workgroup/domain (kinda like > a Domain Admin having Full Access rights on the local machines). > > (I just want to remind any newer Box Backup readers out there that this > configuration is explicitly NOT recommended. I believe this > dis-recommendation is primarily because it will most assuredly be > extremely wasteful due to repetitive "mark as deleted" and re-uploads > of entire backup stores as the workgroup computers disappear and > re-appear from the view of the central Box Backup client.) > > Thanks again, Stuart, > Pete > --- Stuart Hickinbottom wrote: > > >> A workaround is to configure the service to "run as" the account >> where >> you set the share up, then when the service starts it logs on as that >> user and thus has access to his shares. >> > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > From boxbackup at fluffy.co.uk Sat Jul 29 16:26:41 2006 From: boxbackup at fluffy.co.uk (W. Chris Shank) Date: Sat, 29 Jul 2006 11:26:41 -0400 (EDT) Subject: [Box Backup] still don't understand what happens when the store is full Message-ID: <29104438.571154186801911.JavaMail.root@banshee> I have an account that has 90% usage with 46% in old files and 14% in deleted files. The soft limit is 95% of the hard limit. I get notices once per day that the store is full and files aren't being backed up. Since my backup will go off every hour or so, I assumed that as long as I only go this message once a day I was probably ok. Expecting that the limit was reached, the housekeeper then purges some stuff out, then the next several backups succeed. Am I making a wriong assumption here? Is there any way to tell how old the files are in the store? How do I figure out if I've got the newest of stuff? Do I just need to keep growing the store for this account? Thanks in advance -- W. Chris Shank ACE Technology Group, LLC www.myremoteITdept.com (610) 640-4223 -------------------------------- Security Note: To protect against computer viruses, e-mail programs may prevent sending or receiving certain types of file attachments. Check your e-mail security settings to determine how attachments are handled. From boxbackup at fluffy.co.uk Sun Jul 30 14:24:01 2006 From: boxbackup at fluffy.co.uk (Ben Summers) Date: Sun, 30 Jul 2006 14:24:01 +0100 Subject: [Box Backup] still don't understand what happens when the store is full In-Reply-To: <29104438.571154186801911.JavaMail.root@banshee> References: <29104438.571154186801911.JavaMail.root@banshee> Message-ID: <52A2281D-4528-49CD-A3BD-22A20785936C@fluffy.co.uk> Do you have any files which are larger than the difference between the soft and hard limits? Are they updated regularly? Have you tried making that difference greater than the largest file? Ben On 29 Jul 2006, at 16:26, W. Chris Shank wrote: > I have an account that has 90% usage with 46% in old files and 14% > in deleted files. The soft limit is 95% of the hard limit. I get > notices once per day that the store is full and files aren't being > backed up. Since my backup will go off every hour or so, I assumed > that as long as I only go this message once a day I was probably > ok. Expecting that the limit was reached, the housekeeper then > purges some stuff out, then the next several backups succeed. Am I > making a wriong assumption here? Is there any way to tell how old > the files are in the store? How do I figure out if I've got the > newest of stuff? Do I just need to keep growing the store for this > account? > > Thanks in advance > > > -- > W. Chris Shank > ACE Technology Group, LLC > www.myremoteITdept.com > (610) 640-4223 > > -------------------------------- > Security Note: To protect against computer viruses, > e-mail programs may prevent sending or receiving > certain types of file attachments. Check your e-mail > security settings to determine how attachments are > handled. > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup From boxbackup at fluffy.co.uk Sun Jul 30 16:28:44 2006 From: boxbackup at fluffy.co.uk (E.W. Peter Jalajas) Date: Sun, 30 Jul 2006 08:28:44 -0700 (PDT) Subject: [Box Backup] still don't understand what happens when the store is full In-Reply-To: <52A2281D-4528-49CD-A3BD-22A20785936C@fluffy.co.uk> Message-ID: <20060730152844.10646.qmail@web60614.mail.yahoo.com> FWIW, for the newer Box Backup users out there, I set my hard limits HUGELY higher than my soft limits by several GB. I was using 10% based on the original how-to page, http://boxbackup.hostworks.ca/index.php/Certificates_and_accounts_management , but the percentage number is irrelevant, you need to think of it in terms of the size in GB of your largest new file that might be added to your substantially full store. That freeboard between the soft and hard limits is only used for very brief periods of time, and so one can manage the very slight risk of having multiple clients using large portions of their freeboard at the same time (kinda like overbooking an airplane). That is, the total sum of your soft limits MUST NEVER exceed your store server hard drive space, but if you have a large number of accounts, the total sum of your hard limits may exceed your store server hard drive space. While the first client is temporarily using up all of your hard drive space, the second client(s) will simply try again later. Ben, re my statement above: "you need to think of it in terms of the size in GB of your largest new file that might be added to your substantially full store", should I change that to: "you need to think of it in terms of the size in GB of your largest SET OF new fileS that might be added to your substantially full store DURING A SINGLE BACKUP CONNECTION SESSION"? That is, if there is a chance that you might have 3 separate new 1GB files that are scheduled to be uploaded to the substantially full server during a single backup connection session, you need at least 3GB of freeboard between your soft and hard limits so that the store server can hold ALL OF THEM while it makes room for them under the soft limit by deleting old and deleted files, or is 1GB enough, because it'll process them one at a time? Ben, also, I've been asked several times about the algorithm for the timing of deleting old and deleted files, and I always mumble my way through it. Has the precise detailed algorithm been documented (except in the code)? Something like, and I'm making this up: "Files marked as Deleted are deleted first, before any files marked as Old are deleted; then, if there are no (more) files marked as Deleted, then files marked as Old are deleted next, starting with the oldest of the files marked as Old." Maybe you can point me to the code file with the algorithm and then I can translate it to English and put it on the wiki. Thanks, Pete --- Ben Summers wrote: > > Do you have any files which are larger than the difference between > the soft and hard limits? Are they updated regularly? > > Have you tried making that difference greater than the largest file? > > Ben > > > On 29 Jul 2006, at 16:26, W. Chris Shank wrote: > > > I have an account that has 90% usage with 46% in old files and 14% > > > in deleted files. The soft limit is 95% of the hard limit. I get > > notices once per day that the store is full and files aren't being > > > backed up. Since my backup will go off every hour or so, I assumed > > > that as long as I only go this message once a day I was probably > > ok. Expecting that the limit was reached, the housekeeper then > > purges some stuff out, then the next several backups succeed. Am I > > > making a wriong assumption here? Is there any way to tell how old > > the files are in the store? How do I figure out if I've got the > > newest of stuff? Do I just need to keep growing the store for this > > > account? > > > > Thanks in advance > > > > > > -- > > W. Chris Shank > > ACE Technology Group, LLC > > www.myremoteITdept.com > > (610) 640-4223 > > > > -------------------------------- > > Security Note: To protect against computer viruses, > > e-mail programs may prevent sending or receiving > > certain types of file attachments. Check your e-mail > > security settings to determine how attachments are > > handled. > > > > _______________________________________________ > > boxbackup mailing list > > boxbackup at fluffy.co.uk > > http://lists.warhead.org.uk/mailman/listinfo/boxbackup > > _______________________________________________ > boxbackup mailing list > boxbackup at fluffy.co.uk > http://lists.warhead.org.uk/mailman/listinfo/boxbackup >