[Box Backup-dev] Fwd: Bandwidth usage patch
Ben Summers
boxbackup-dev at fluffy.co.uk
Mon Feb 13 09:37:53 GMT 2006
--Apple-Mail-1-454999898
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
delsp=yes;
format=flowed
Pascal has sent me the patch below. While it's not a completely
correct solution (as it ignored SSL overhead), shall we add it in to
the next release?
Ben
Begin forwarded message:
> From: Pascal Lalonde <plalonde at overnet.qc.ca>
> Date: 11 February 2006 18:08:08 GMT
> To: Ben Summers <ben at fluffy.co.uk>
> Subject: Bandwidth usage patch
>
> Hi Ben,
>
> I created a small patch to make the server log the transferred size
> after each connection. If the code meets your standards, I was hoping
> you could include this. The calculations seem to be pretty accurate,
> almost matching the ones found in the client's logs.
>
> Knowing the total transferred size lets me know who eats all my
> bandwidth as well as which clients are not uploading as they should
> due
> to configuration errors.
>
> I wrote a small script to sum up the transfer sizes and report the
> total. Since my boxbackup logs rotate monthly, I can get the monthly
> bandwidth usage. I'm sending it too (I'm no perl expert though!).
>
> I've been running my patch for a little while now with no problems.
> The
> runtest.PL reports all tests as PASSED for OpenBSD 3.8 and Debian 3.1.
>
> By the way, I haven't had any trouble with 0.09-plus2.
> Keep up the good work.
>
> Pascal
> Index: lib/server/SocketStream.cpp
> ===================================================================
> --- lib/server/SocketStream.cpp (revision 412)
> +++ lib/server/SocketStream.cpp (working copy)
> @@ -35,7 +35,9 @@
> SocketStream::SocketStream()
> : mSocketHandle(-1),
> mReadClosed(false),
> - mWriteClosed(false)
> + mWriteClosed(false),
> + mBytesRead(0),
> + mBytesWritten(0)
> {
> }
>
> @@ -50,7 +52,9 @@
> SocketStream::SocketStream(int socket)
> : mSocketHandle(socket),
> mReadClosed(false),
> - mWriteClosed(false)
> + mWriteClosed(false),
> + mBytesRead(0),
> + mBytesWritten(0)
> {
> if(socket < 0)
> {
> @@ -69,7 +73,9 @@
> SocketStream::SocketStream(const SocketStream &rToCopy)
> : mSocketHandle(::dup(rToCopy.mSocketHandle)),
> mReadClosed(rToCopy.mReadClosed),
> - mWriteClosed(rToCopy.mWriteClosed)
> + mWriteClosed(rToCopy.mWriteClosed),
> + mBytesRead(rToCopy.mBytesRead),
> + mBytesWritten(rToCopy.mBytesWritten)
>
> {
> if(rToCopy.mSocketHandle < 0)
> @@ -111,6 +117,7 @@
> if(mSocketHandle != -1) {THROW_EXCEPTION(ServerException,
> SocketAlreadyOpen)}
>
> mSocketHandle = socket;
> + ResetCounters();
> }
>
>
> @@ -151,6 +158,7 @@
> mSocketHandle = -1;
> THROW_EXCEPTION(ConnectionException, Conn_SocketConnectError)
> }
> + ResetCounters();
> }
>
> //
> ----------------------------------------------------------------------
> ----
> @@ -222,6 +230,7 @@
> mReadClosed = true;
> }
>
> + mBytesRead += r;
> return r;
> }
>
> @@ -263,6 +272,8 @@
> bytesLeft -= sent;
> // Move buffer pointer
> buffer += sent;
> +
> + mBytesWritten += sent;
>
> // Need to wait until it can send again?
> if(bytesLeft > 0)
> Index: lib/server/SocketStream.h
> ===================================================================
> --- lib/server/SocketStream.h (revision 412)
> +++ lib/server/SocketStream.h (working copy)
> @@ -56,6 +56,15 @@
> tOSSocketHandle mSocketHandle;
> bool mReadClosed;
> bool mWriteClosed;
> +
> +protected:
> + off_t mBytesRead;
> + off_t mBytesWritten;
> +
> +public:
> + off_t GetBytesRead() const {return mBytesRead;}
> + off_t GetBytesWritten() const {return mBytesWritten;}
> + void ResetCounters() {mBytesRead = mBytesWritten = 0;}
> };
>
> #endif // SOCKETSTREAM__H
> Index: lib/server/SocketStreamTLS.cpp
> ===================================================================
> --- lib/server/SocketStreamTLS.cpp (revision 412)
> +++ lib/server/SocketStreamTLS.cpp (working copy)
> @@ -40,6 +40,7 @@
> SocketStreamTLS::SocketStreamTLS()
> : mpSSL(0), mpBIO(0)
> {
> + ResetCounters();
> }
>
> //
> ----------------------------------------------------------------------
> ----
> @@ -101,6 +102,7 @@
> {
> SocketStream::Open(Type, Name, Port);
> Handshake(rContext);
> + ResetCounters();
> }
>
>
> @@ -295,6 +297,7 @@
> {
> case SSL_ERROR_NONE:
> // No error, return number of bytes read
> + mBytesRead += r;
> return r;
> break;
>
> @@ -358,6 +361,7 @@
> {
> case SSL_ERROR_NONE:
> // No error, data sent, return success
> + mBytesWritten += r;
> return;
> break;
>
> Index: bin/bbstored/BackupStoreDaemon.cpp
> ===================================================================
> --- bin/bbstored/BackupStoreDaemon.cpp (revision 412)
> +++ bin/bbstored/BackupStoreDaemon.cpp (working copy)
> @@ -278,7 +278,25 @@
> BackupProtocolServer server(rStream);
> server.SetLogToSysLog(mExtendedLogging);
> server.SetTimeout(BACKUP_STORE_TIMEOUT);
> - server.DoServer(context);
> + try
> + {
> + server.DoServer(context);
> + LogConnectionStats(clientCommonName.c_str(), rStream);
> + }
> + catch(...)
> + {
> + LogConnectionStats(clientCommonName.c_str(), rStream);
> + throw;
> + }
> context.CleanUp();
> }
>
> +void BackupStoreDaemon::LogConnectionStats(const char *commonName,
> + const SocketStreamTLS &s)
> +{
> + // Log the amount of data transferred
> + ::syslog(LOG_INFO, "Connection statistics for %s: "
> + "IN=%lld OUT=%lld TOTAL=%lld\n", commonName,
> + s.GetBytesRead(), s.GetBytesWritten(),
> + s.GetBytesRead() + s.GetBytesWritten());
> +}
> Index: bin/bbstored/BackupStoreDaemon.h
> ===================================================================
> --- bin/bbstored/BackupStoreDaemon.h (revision 412)
> +++ bin/bbstored/BackupStoreDaemon.h (working copy)
> @@ -61,6 +61,8 @@
> void HousekeepingProcess();
> bool CheckForInterProcessMsg(int AccountNum = 0, int
> MaximumWaitTime = 0);
>
> + void LogConnectionStats(const char *commonName, const
> SocketStreamTLS &s);
> +
> private:
> BackupStoreAccountDatabase *mpAccountDatabase;
> BackupStoreAccounts *mpAccounts;
--Apple-Mail-1-454999898
Content-Transfer-Encoding: 7bit
Content-Type: application/x-perl;
x-unix-mode=0666;
name="boxusage.pl"
Content-Disposition: attachment;
filename=boxusage.pl
#!/usr/bin/perl
use bigint;
$accfile = "/etc/box/bbstored/accounts.txt";
if (!open(ACCFILE, $accfile)) {
&error("Could not open accounts file");
}
while ($line = <ACCFILE>) {
$line =~ /(\d*):.*/;
push(@accounts, $1);
}
close(ACCFILE);
foreach $account (@accounts) {
$total{$account} = 0;
}
$now = localtime();
print("BoxBackup bandwidth usage on ", $now, ":\n");
while ($line = <>) {
if ($line =~ /Connection statistics for BACKUP-([0-9A-Fa-f]*): IN=([0-9]*) OUT=([0-9]*) TOTAL=([0-9]*)/) {
$total{$1} = $total{$1} + $4;
}
}
foreach $account (@accounts) {
print("BACKUP-", $account, ":\t", $total{$account}, " (Megs: ", $total{$account} / 1024 / 1024, ")\n");
}
exit(0);
--Apple-Mail-1-454999898
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
format=flowed
--Apple-Mail-1-454999898--
More information about the Boxbackup-dev
mailing list