[Box Backup] High CPU usage without any work available

Jan Haastrecht jan.haastrecht at gmail.com
Thu Nov 22 22:22:04 GMT 2012


Hi Chris,
> Hi Jan,
>
> On Thu, 22 Nov 2012, Jan Haastrecht wrote:
>
>> First of all, thank you for answering my previous questions regarding the 
>> certificates.
>>
>> bbstored uses 88% CPU (as much as it can possibly use) continuously when no 
>> backup client is connected, even 5 minutes after which such a client has 
>> disconnected. lsof -p 4534 showed that there were only a few files open and 
>> since the memory of the backup server is limited, and the client hadn't added 
>> any large files in backed up directories in the few hours before this event; 
>> the CPU usage should have been close to zero.
>
> Does this always happen after a client disconnects?
No, I disconnected the client as an experiment. The client was connected when it 
happened.
> What OS is on the server?
Linux 3.3.8

> Did you see multiple bbstored processes? Was one of them a housekeeping 
> process? (If you can't tell from the "ps aux" output, it usually has a PID 1 
> or 2 higher than the main server process). Was it that one that was using a 
> lot of CPU? Do you have any very large accounts that might take a long time to 
> housekeep? 
I don't recall whether it was a housekeeping process. What is very large? It's 
about 50GB and the available space is much larger. Doesn't house keeping only 
run when there is a shortage of space?
>
>> The memory usage also appears to be much higher (3 to 4 times higher when 
>> compared to the startup memory) when CPU usage is higher. This seems to 
>> suggest there is a memory leak too.
>
> Housekeeping can use a lot of memory, especially for the reference count 
> database if you have a large account, that is normal.
There is about 50MB of memory available in total (for exclusive use by 
bbstored). I intend to store millions of files (less than 5 million) using about 
500GB. Is that going to work?
>
>> A command which would send a message to the daemon to show what it is 
>> currently doing would also be most welcome. Just running it with verbose 
>> options probably wouldn't work, because it would have to create 10+MB of log 
>> files before anything would come up and perhaps it even changes the behavior.
>
> You might be able to get some useful information by stracing the process 
> that's using a lot of CPU. Even if you don't understand the output, we might 
> be able to work it out.
I will do that when it happens next time.
>
> I recommend that you run bbstored with the -V option anyway, despite the large 
> volume of logs that it will create. This may cause strace to show more 
> information even if your syslogd doesn't write debug-level messages to 
> anywhere. You could also try attaching gdb to the CPU-using process and 
> generating a backtrace with the "bt" command, especially if bbstored is 
> compiled with debug symbols. If it isn't, then we might need you to build one 
> with debugging enabled to test it.
The wiki contains:

*DO NOT DO THIS* for bbstored or bbstoreaccounts, as the debug versions use a 
different store file layout and they will NOT be compatible with your existing 
store, and may well corrupt it.

Why doesn't the debug version do the same thing as the standard version, except 
with more debug information? I would like to receive the instructions to build a 
debug version anyway.

Finally, is anyone using boxbackup? One would expect that if such problems come 
up within days that others would have also noticed these issues before me.

Jan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.boxbackup.org/pipermail/boxbackup/attachments/20121122/167198f5/attachment.html>


More information about the Boxbackup mailing list