[Box Backup-dev] Re: [Box Backup] bbackupd - read errors on database files

Chris Wilson boxbackup-dev at fluffy.co.uk
Thu Apr 12 21:38:02 BST 2007


Hi Gary,

On Tue, 10 Apr 2007, G. wrote:

> The plain-text MD5 checksum in question (I mean one MD5 checksum per 
> file, not one MD5 checksum per block) could become a part of a file's 
> attribute stream, which is already decrypted and used for comparison of 
> file attributes. It could also be used to generate/compare stronger 
> folder-level checksum. So, you can just use such an MD5 directly to 
> verify whether last-known file or folder content on the server matches 
> file or folder content locally. Much less work than compare -aq, since 
> there is no need to compare checksums on block-by-block basis, and there 
> is no need to re-download MD5s in the first place (they are all already 
> in-memory after a backup cycle, and could be preserved by 
> StoreObjectInfoFile; if not, bbackupd already downloads all file 
> attribute stream objects anyway the first time around).

I'd say it uses less Internet bandwidth use than compare -a, but not less 
CPU or disk activity.

We can't cache the checksums of local files on disk, otherwise we'd have 
the same problem that we do now :-(

compare -aq does not compare checksums of anything, as far as I know, it 
just checks the file attributes. compare -a checks the file contents. A 
new mode might be compare -ac, which checks the checksums of remote blocks 
against their re-encrypted local block checksums.

>> Checksums of the encrypted data have been discussed, but that would
>> require new commands on the server to return the IV and checksum of each
>> block, which is more complex again.
>
> Yup, you are absolutely right. I should have been more clear; I meant 
> "remote checksum to local checksum" compare, not "remote checksum to 
> remote disk content" (bbstored RAID-style integrity check) compare.

Actually I didn't mean "remote checksum to remote disk content", but 
rather "local disk checksums to remote disk checksums", which is almost 
exactly as strong as "local disk data to remote disk data".

If the encrypted block checksums are stored on the server rather than 
recomputed when necessary, this would be weaker (local disk checksums to 
remove saved checksums) but require less resources on the server.

I think that the mode you describe, "remote checksum to remote disk 
content" would be better achieved by the client uploading the unencrypted 
checksum of the encrypted data, which is saved by the server as an 
unencrypted attribute, which bbstoreaccounts check can reverify at any 
time.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |



More information about the Boxbackup-dev mailing list