[Box Backup] Memory usage house keeping through the roof

Chris Wilson chris at qwirx.com
Wed Feb 13 00:33:14 GMT 2013


Hi Jan,

On Mon, 11 Feb 2013, Jan Haastrecht wrote:

>> I think it uses more than it used to since I added reference counting, 
>> which is experimental. I am planning to memory-map this database from disk, 
>> to reduce in-core memory usage, but haven't implemented that yet. In the 
>> mean time, swap should work just as well.
>> 
>> It uses 4 bytes per object, so 4 MB if you have 1 million objects (files 
>> and directories) in your account. How many do you have?
>
> I have about 5 million (more than I expected). It's using around 140MB (of 
> which a part is swap). I can't see whether it has completed housekeeping.

I can't tell you immediately where the memory use is, but I can find out, 
if you're willing to help.

I've just added some code to the latest trunk that allows printing out a 
memory usage report by sending a USR1 signal to the process. You'd need to 
download the latest trunk from Subversion onto your bbstored server, 
configure and make like this:

 	./configure CXXFLAGS=-DDEBUG_LEAKS
 	make

and run the binary in release/bin/bbstored/bbstored instead of the usual 
bbstored, with the correct path to your bbstored.conf file if necessary.

Then you can identify the housekeeping process, wait until it's using a 
lot of memory, and use "kill -USR1" to send the signal to it. You should 
see a memory usage report in /var/log/messages or /var/log/syslog.

If your syslogd truncates the report due to the number of messages, you 
can also run bbstored from the console with the "-kV" option, or instead 
with "-o <logfile>" to write the logs to a file.

If you could let me know the output of that process, that would be much 
appreciated.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <chris+sig at qwirx.com> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |



More information about the Boxbackup mailing list