[Box Backup-commit] Re: #8: Improve handling of directories with many files

boxbackup-dev at fluffy.co.uk boxbackup-dev at fluffy.co.uk
Thu Oct 26 12:13:08 BST 2006

#8: Improve handling of directories with many files
  Reporter:  martin         |       Owner:      
      Type:  task           |      Status:  new 
  Priority:  normal         |   Milestone:  0.20
 Component:  box libraries  |     Version:  0.10
Resolution:                 |    Keywords:      
Comment (by ben):

 bbackupd : When a directory has lots of files, there are rather too many
 compares going on when searching for it's entry in the directory listing
 retrieved from the server. This scales logarithmically.

 bbstored : When housekeeping, bbstored reads every directory. Within each
 directory, the contents are only scanned linearly a couple of times, so
 overall it should scale linearly with the number of directories (where
 number of files in them are far less important).

 We do need to move to a ref counted store, to avoid all this scanning. But
 on review of the code, I don't think the excessive CPU usage on the server
 is due to inefficient handling of large directories.

Ticket URL: <http://bbdev.fluffy.co.uk/trac/ticket/8#comment:1>
Box Backup <http://www.fluffy.co.uk/boxbackup/>
An open source, completely automatic on-line backup system for UNIX.

More information about the Boxbackup-commit mailing list