[Box Backup] Housekeeping hogging the server

[Jan Rychter]

>>>>> "Ben" == Ben Summers <ben at fluffy.co.uk>:
 Ben> On 25 Oct 2006, at 12:23, Jamie Neil wrote:

 >> I'm in the process of setting up a new server for backing up the
 >> other servers in our rack, and I was planning on using Boxbackup.
 >> Unfortunately it looks like it's not going to be practical due to
 >> the extremely high CPU loads that we are seeing.
 >>
 >> Server is a 2.8GHz Xeon with 1Gb RAM and RAID 1 SATA. Currently it's
 >> backing up 7 other servers with a total of 50Gb. None of the
 >> accounts is even close to their soft limits and yet the CPU has been
 >> pegged at close to 100% for the last 2 days.
 >>
 >> I saw this behaviour before when we were experimenting with
 >> Boxbackup and it turned out to be a couple of folders with thousands
 >> of small files in them. Once we removed them then the problem went
 >> away. I suspect the same problem this time as one of the servers is
 >> storing mail in maildirs (Cyrus) and may have several folders
 >> containing a few thousand files each.
 >>
 >> I assume that this is a housekeeping issue because there are
 >> currently no clients connected.

 Ben> Yes, this is most likely a housekeeping issue. The client is also
 Ben> not happy with directories with thousands of files in them. I do
 Ben> want to address this in a future version, it has come up a few
 Ben> times.

Incidentally, this is also my use case (I started this thread about
peformance issues). I use boxbackup to backup my E-mail storage, which
is in nnml format -- with directories reaching over 10,000 files
sometimes.

--J.