[Box Backup] 100% CPU on big directories

[Ben Summers]

On 12 Feb 2006, at 17:28, Jamie Neil wrote:
>
[snip]

> Both the bbstored server and the server with the large directory have
> plenty of horsepower (3GHz Xeon and Dual 1GHz Xeon respectively),
> neither seem to be running out of memory, and they are talking over a
> 100Mbit switched LAN, so I assume that this is a limitation in  
> boxbackup.
>
> Yes, I know it's not clever to have a flat directory with 150,000  
> files
> in it :), but I've noticed the effect to a smaller degree on  
> directories
> with only 1000 files. Does anyone know if it's fixable?

I can think of a couple of places in the code which wouldn't  
appreciate this. It might be possible to get a speed-up by using a  
hash_map instead of a map in critical places.

I've added a note to the wiki about this to make sure we think about  
it in the next round of changes.

Sorry not to be more helpful now.

Ben