[Box Backup] Notes on trunk_3047 upgrade today

[Chris Wilson]

Hi Pete,

On Sun, 11 Dec 2011, Pete Jalajas wrote:

> I couldn't make it compile because of the doc Errors, so, being fairly 
> ignorant of these things, I tweaked the Makefile to take out the 
> doc-related dependencies from this line:

I tried to make a Makefile target that doesn't build the docs, but it's 
quite complicated because the install script also references them, and I'd 
prefer not to have two ways to build backup-server-linux-gnu.tgz, one with 
docs and one without, because it could lead to more confusion in future. 
Perhaps if you tell me the errors that you experienced, I can help you to 
fix them? It's usually just a case of installing xsltproc.

> Dec 11 19:39:43 gl8 bbstored client=0x10008027[19468]: NOTICE:
> Connection statistics for BACKUP-10008027: IN=514 OUT=36910
> NET_IN=-36396 TOTAL=37424
[...]
> So, at 19:39, it looks like 2 files were backed up with very small
> changes only (<514 bytes total, per "IN=514" in "Connection
> statistics" log entry above).  Correct?  Not sure what the OUT=36910
> bytes represents (file listings? checksums?)   What else would make up
> part of the "IN=" value?   If "IN=" is greater than zero, does that
> mean at least one file was backed up?

It just measures the total number of bytes sent by the client to the 
server. That would include commands and their arguments, as well as file 
data uploaded (complete and patched). Without more detailed logs, it's 
only possible to say that less than 514 bytes of changes were uploaded. 
You can enable LogAllFileAccess if you want logging with that amount of 
detail.

> If the client connected at all in Read/Write mode, perhaps that means by 
> definition at least one file needed to be backed up?

I think so, yes, provided that the client had a working state already in 
memory (either bbackupd hadn't been restarted since the last backup, or 
StoreObjectInfoFile was enabled and working).

> In lib/backupstore/BackupCommands.cpp, it would be very nice to have the 
> "mAccountName" (or at least the first 8 characters or so) listed in the 
> "Login from Client ID " and "Session finished for " log records, so I 
> don't have to grep mClientID in my little local lookup table every time.

Added.

> Similarly, and lastly:
> $ grep -nRi -e "Connection statistics"  bin/bbstored/*
> bin/bbstored/BackupStoreDaemon.cpp:367:	BOX_NOTICE("Connection
> statistics for " << commonName << ":"
>
> Not sure why this log record is in a separate code tree source file
> from the "Login from Client" and "Session finished for " entries
> above.

Because the protocol command handlers in BackupCommands.cpp don't have 
direct access to the raw stream, so they can't see the byte counters; and 
the logging happens at the end of the connection, even if it was aborted 
by a network error, in which case no protocol command is executed at all.

> Maybe it could be tweaked to match the other log entries,
> like BOX_FORMAT_ACCOUNT(mClientID), perhaps with "mAccountName" as
> well?

Done.

> PS:  For extra credit, it would be nice to have the mClientID and
> mAccountName in this log record:
> bbstored[19139]: NOTICE: Message from child process 20111: Incoming
> connection from 24.62.185.131 port 39402

Unfortunately it's too early. We don't know that information when the 
message is logged, because the client hasn't sent us anything yet, not 
even an SSL client certificate. I've added them to the post-login message 
instead.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <chris+sig at qwirx.com> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |