[Box Backup] High CPU usage without any work available

Chris Wilson chris at qwirx.com
Fri Nov 23 00:12:35 GMT 2012


Hi Jan,

On Thu, 22 Nov 2012, Jan Haastrecht wrote:

>             bbstored uses 88% CPU (as much as it can possibly use) continuously when no backup client is connected,
>
>       Does this always happen after a client disconnects?
> 
> No, I disconnected the client as an experiment. The client was connected when it happened.

Sorry, I don't understand. I thought you said NO client was connected and 
you were still seeing high CPU usage? Which is it? And also, can you 
repeat it, for example by getting the client to connect and disconnect?

>       What OS is on the server?
> 
> Linux 3.3.8

What distro? Where did you get the bbstored software from and how did you 
install it?

>       Did you see multiple bbstored processes? Was one of them a 
> housekeeping process? (If you can't tell from the "ps aux" output, it 
> usually has a PID 1 or 2 higher than the main server process).
>       Was it that one that was using a lot of CPU? Do you have any very 
> large accounts that might take a long time to housekeep?
> 
> I don't recall whether it was a housekeeping process. What is very 
> large? It's about 50GB and the available space is much larger. Doesn't 
> house keeping only run when there is a shortage of space?

50GB is probably reasonable, but might take 10-30 minutes for housekeeping 
depending on CPU and disk. You can see for yourself by running 
"bbstoreaccounts housekeep <id>" on the account if you have a recent 
enough version of bbstored (but I don't think you do).

It runs every interval configured in the bbstored.conf file. It rebuilds 
and corrects the reference count database even if it doesn't need to 
remove any files. We could probably make it only run when the account is 
over its soft limit, that seems like a good idea.

>       Housekeeping can use a lot of memory, especially for the reference 
> count database if you have a large account, that is normal.
> 
> There is about 50MB of memory available in total (for exclusive use by 
> bbstored). I intend to store millions of files (less than 5 million) 
> using about 500GB. Is that going to work?

I think it would take 4 bytes per file, i.e. 20 MB for the refcount 
database.

> DO NOT DO THIS for bbstored or bbstoreaccounts, as the debug versions 
> use a different store file layout and they will NOT be compatible with 
> your existing store, and may well corrupt it.
> 
> Why doesn't the debug version do the same thing as the standard version, 
> except with more debug information? I would like to receive the 
> instructions to build a debug version anyway.

I don't mean that you should run a debug build from the "debug" directory, 
but rather a release build that was built with debugging CFLAGS. The debug 
build contains modifications to stress some parts of the system, which 
make it incompatible.

If you just download and build the latest sources (you don't have to 
install them), and look in release/bin/bbstored, that version will be 
built with debugging symbols.

> Finally, is anyone using boxbackup? One would expect that if such 
> problems come up within days that others would have also noticed these 
> issues before me.

I think everyone expects that housekeeping is slow. It's not ideal but we 
have little time and higher priorities than improving it right now. Slow 
housekeeping has come up before on the mailing list, for accounts with 
hundreds of GB in them taking several hours.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <chris+sig at qwirx.com> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |



More information about the Boxbackup mailing list