[Box Backup-commit] [boxbackup/boxbackup] b08bd5: Add ListBucket command support to S3Simulator and ...

GitHub noreply at github.com
Sat Oct 7 20:32:19 BST 2017


  Branch: refs/heads/s3_support_merge
  Home:   https://github.com/boxbackup/boxbackup
  Commit: b08bd5fbfe52192ad18ea38c45fed671a64ca282
      https://github.com/boxbackup/boxbackup/commit/b08bd5fbfe52192ad18ea38c45fed671a64ca282
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/httpserver/S3Client.cpp
    M lib/httpserver/S3Client.h
    M lib/httpserver/S3Simulator.cpp
    M lib/httpserver/S3Simulator.h
    M test/httpserver/testhttpserver.cpp

  Log Message:
  -----------
  Add ListBucket command support to S3Simulator and S3Client


  Commit: 64405ac2b6f0a183e404525fd5aae52157daad56
      https://github.com/boxbackup/boxbackup/commit/64405ac2b6f0a183e404525fd5aae52157daad56
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/common/ReadLoggingStream.cpp

  Log Message:
  -----------
  Fix negative estimated remaining time due to 64-bit integer overflow on large files.

(cherry picked from commit b27d4e7c61b04db054399435b5c57df1f9b1832b)


  Commit: f4661d62640be5b6c7e7b046df713502f42ebad9
      https://github.com/boxbackup/boxbackup/commit/f4661d62640be5b6c7e7b046df713502f42ebad9
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/httpserver/S3Client.cpp
    M lib/httpserver/S3Client.h
    M lib/httpserver/S3Simulator.cpp
    M lib/httpserver/S3Simulator.h
    M test/httpserver/testhttpserver.cpp

  Log Message:
  -----------
  Add support for the HEAD and DELETE methods to the S3Simulator and S3Client.


  Commit: 2269177efbcec9a3735e94994cee367467c2a789
      https://github.com/boxbackup/boxbackup/commit/2269177efbcec9a3735e94994cee367467c2a789
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/common/BoxPlatform.h

  Log Message:
  -----------
  Enable memory testing under Darwin, as it does work now

(cherry picked from commit 86dc5ff3666c934dcf4fce261a93431b78bf018f)


  Commit: 819f4679f898c865ae06e7ff2e7e5885f444329c
      https://github.com/boxbackup/boxbackup/commit/819f4679f898c865ae06e7ff2e7e5885f444329c
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/HousekeepStoreAccount.cpp

  Log Message:
  -----------
  Rename local variable

First part of refactoring HousekeepStoreAccount to use BackupFileSystem.


  Commit: 116bb2600cb3b603adffe56233599c7f3efd8f38
      https://github.com/boxbackup/boxbackup/commit/116bb2600cb3b603adffe56233599c7f3efd8f38
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreException.txt
    M lib/backupstore/BackupStoreRefCountDatabase.cpp
    M lib/backupstore/BackupStoreRefCountDatabase.h

  Log Message:
  -----------
  Fix conflict over unlocked refcount DB

If BackupStoreCheck is are read-only, then it should not open a "temporary"
database in the old sense, because that does actually change the store: the
temporary file would conflict with any other process which wants to do the same
thing at the same time (e.g. housekeeping), and if neither process locks the
store, they will break each other.

Instead, we now distinguish between "potential" refcount DBs (of which there
can only be one at a time, and which require a lock to create) and really
"temporary" ones which are created in a temporary directory, can never become
permanent via Commit(), do not require a lock, and are unlimited in number.
Commit() will rename them, but still in the temporary directory, so it has no
effect on the store.

This requires some changes to the BackupStoreRefCountDatabase::Create interface
to add new flags.


  Commit: b010bdf48e99dd9d08ceb630dddeb005a4d70ca4
      https://github.com/boxbackup/boxbackup/commit/b010bdf48e99dd9d08ceb630dddeb005a4d70ca4
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreRefCountDatabase.cpp
    M lib/backupstore/BackupStoreRefCountDatabase.h

  Log Message:
  -----------
  Refactor BackupStoreRefCountDatabase to an abstract interface

This makes it easier to cleanly subclass in BackupFileSystem, to change its
behaviour by hooking into its Commit() method.


  Commit: 2674466232de994d6192e08a25c8eb26d94eed62
      https://github.com/boxbackup/boxbackup/commit/2674466232de994d6192e08a25c8eb26d94eed62
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreException.txt
    M lib/backupstore/BackupStoreRefCountDatabase.cpp
    M lib/backupstore/BackupStoreRefCountDatabase.h

  Log Message:
  -----------
  Move management of BackupStoreRefCountDatabase into BackupFileSystem

This is needed to allow BackupFileSystem to commit the database to storage on
request, and to cache it, which is needed for S3 support.


  Commit: e6ef4d9e1d545da3a0a3eeee79d57eed9fb9738a
      https://github.com/boxbackup/boxbackup/commit/e6ef4d9e1d545da3a0a3eeee79d57eed9fb9738a
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupAccountControl.h
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreAccounts.cpp
    M lib/backupstore/BackupStoreAccounts.h
    M lib/backupstore/BackupStoreException.txt
    M lib/backupstore/BackupStoreInfo.cpp
    M lib/backupstore/BackupStoreInfo.h

  Log Message:
  -----------
  Initial refactor of BackupAccountControl to use BackupFileSystem

Abstract away some account operations using BackupFileSystem so that they can
be implemented differently for S3 accounts. In particular, account opening and
NamedLock usage were targets for this refactor. BackupStoreInfo creation and
opening ended up being partially addressed as well. This significantly reduces
the diffs to the s3_support branch.


  Commit: b40dc419531b47e5393970366e3d7710da70b6ae
      https://github.com/boxbackup/boxbackup/commit/b40dc419531b47e5393970366e3d7710da70b6ae
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M infrastructure/m4/boxbackup_tests.m4
    M lib/backupstore/BackupProtocol.h
    M lib/backupstore/BackupStoreFile.h
    M lib/backupstore/BackupStoreFileCmbIdx.cpp
    M lib/backupstore/BackupStoreFileEncodeStream.h
    M lib/common/BufferedStream.h
    M lib/common/BufferedWriteStream.h
    M lib/common/CollectInBufferStream.h
    M lib/common/FileStream.h
    M lib/common/MemBlockStream.h
    M lib/common/PartialReadStream.h
    M lib/common/RateLimitingStream.h
    M lib/common/ReadGatherStream.h
    M lib/common/ReadLoggingStream.h
    M lib/common/SelfFlushingStream.h
    M lib/common/ZeroStream.h
    M lib/compress/CompressStream.h
    M lib/raidfile/RaidFileRead.h
    M lib/raidfile/RaidFileWrite.h
    M lib/server/ProtocolUncertainStream.h
    M lib/server/SocketStream.h
    M lib/server/SocketStreamTLS.h
    M lib/server/TcpNice.h
    M lib/server/WinNamedPipeStream.h
    M test/bbackupd/testbbackupd.cpp
    M test/compress/testcompress.cpp

  Log Message:
  -----------
  Enable compiler warning -Werror=overloaded-virtual, and fix


  Commit: c5466cc85a83d095388c28dc18a1b48788f64dc2
      https://github.com/boxbackup/boxbackup/commit/c5466cc85a83d095388c28dc18a1b48788f64dc2
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/HousekeepStoreAccount.cpp
    M lib/backupstore/HousekeepStoreAccount.h
    M lib/backupstore/StoreTestUtils.cpp
    M lib/backupstore/StoreTestUtils.h
    M lib/bbstored/BBStoreDHousekeeping.cpp

  Log Message:
  -----------
  Refactor HousekeepStoreAccount to use BackupFileSystem

This involved moving big chunks of functionality into RaidBackupFileSystem, for
example CombineFile, CombineDiffs and RaidPutFileCompleteTransaction. Not much
is implemented for S3BackupFileSystem yet.


  Commit: 7b8a48fe0d4e141f5fad89818e1b9f10bcf028bc
      https://github.com/boxbackup/boxbackup/commit/7b8a48fe0d4e141f5fad89818e1b9f10bcf028bc
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreCheck.h
    M lib/backupstore/BackupStoreCheck2.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 1

Pass the BackupFileSystem from BackupAccountControl to BackupStoreCheck, and
use it for locking the account (actually a no-op because it's already locked).
Trivial, but it's a start.


  Commit: c597742a73de567f278c83deb87872326e9109ad
      https://github.com/boxbackup/boxbackup/commit/c597742a73de567f278c83deb87872326e9109ad
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreCheck.h
    M lib/backupstore/BackupStoreCheck2.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 2

Use BackupFileSystem to access the refcount database.


  Commit: dd7ecdf470933e660ca8ed585fa11233d4652609
      https://github.com/boxbackup/boxbackup/commit/dd7ecdf470933e660ca8ed585fa11233d4652609
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreCheck2.cpp
    M lib/backupstore/BackupStoreInfo.cpp
    M lib/backupstore/BackupStoreInfo.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 3

Use BackupFileSystem to access the per-account BackupStoreInfo.


  Commit: fc4621738ba7fb19a2bab95b3c84f2253a3a9391
      https://github.com/boxbackup/boxbackup/commit/fc4621738ba7fb19a2bab95b3c84f2253a3a9391
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreCheck.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 4

Use BackupFileSystem to EnsureObjectIsPermanent.


  Commit: 78b117f4b5c18210b04592b6f61d975091ccef01
      https://github.com/boxbackup/boxbackup/commit/78b117f4b5c18210b04592b6f61d975091ccef01
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreCheck.h
    M lib/backupstore/BackupStoreCheck2.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 5

Use BackupFileSystem to Get and Put Files and Directories.


  Commit: d33181dd7f4a874656e02d3d12e33849b1ae27ea
      https://github.com/boxbackup/boxbackup/commit/d33181dd7f4a874656e02d3d12e33849b1ae27ea
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreCheck.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 6

Use BackupFileSystem to delete corrupt objects.


  Commit: 65de4e02377a5f8a3ba5b1085fa3011b504ca198
      https://github.com/boxbackup/boxbackup/commit/65de4e02377a5f8a3ba5b1085fa3011b504ca198
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreCheck2.cpp
    M lib/backupstore/BackupStoreException.txt

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 7

Move most low-level filesystem structure check code into BackupFileSystem, and
add specialised implementation for S3.


  Commit: 1eab33a21ad84f084b78d62631e155bbe9ad64f4
      https://github.com/boxbackup/boxbackup/commit/1eab33a21ad84f084b78d62631e155bbe9ad64f4
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupStoreCheck.cpp
    M lib/backupstore/BackupStoreCheck.h
    M lib/backupstore/BackupStoreCheck2.cpp

  Log Message:
  -----------
  Refactor BackupStoreCheck to use BackupFileSystem, part 8

Remove last vestiges of RaidFile from BackupStoreCheck.


  Commit: c610b3c76540425fdd494cf8d17c4b3bf89872b5
      https://github.com/boxbackup/boxbackup/commit/c610b3c76540425fdd494cf8d17c4b3bf89872b5
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreCheck2.cpp

  Log Message:
  -----------
  Rename some local variables in BackupStoreCheck


  Commit: 0f10d38df43a9e34f456a5dc64cbe0926995da30
      https://github.com/boxbackup/boxbackup/commit/0f10d38df43a9e34f456a5dc64cbe0926995da30
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupclient/BackupDaemonConfigVerify.cpp
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreException.txt
    M test/s3store/testfiles/bbackupd.conf

  Log Message:
  -----------
  Add configuration for S3 local cache directory


  Commit: 4f65983af40f5bac2567db65959c796f493e9fe9
      https://github.com/boxbackup/boxbackup/commit/4f65983af40f5bac2567db65959c796f493e9fe9
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp

  Log Message:
  -----------
  Fix repeated calls to TryGetLock() when already holding lock


  Commit: c88763565b2f2bdef0b8012139cb9bbb2bbe280f
      https://github.com/boxbackup/boxbackup/commit/c88763565b2f2bdef0b8012139cb9bbb2bbe280f
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h

  Log Message:
  -----------
  Implement RefCountDatabase methods in S3BackupFileSystem


  Commit: 32a4c7b5b4f55e9d5e352b1140da093b28701bc2
      https://github.com/boxbackup/boxbackup/commit/32a4c7b5b4f55e9d5e352b1140da093b28701bc2
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.h

  Log Message:
  -----------
  Fix writeback of refcount databases on BackupFileSystem destruction

Call ReleaseLock in the child destructor (not the parent) to allow the refcount
database to clean itself up (normally involving calling
BackupFileSystem::SaveRefCountDatabase) before the BackupFileSystem is too
destroyed to be able to fulfil the request.


  Commit: f7483393c5cb7c6e9f2135a4efd57f4eb958d1c1
      https://github.com/boxbackup/boxbackup/commit/f7483393c5cb7c6e9f2135a4efd57f4eb958d1c1
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/bbstored/BBStoreDHousekeeping.cpp
    M lib/bbstored/BackupStoreDaemon.cpp

  Log Message:
  -----------
  Wait for housekeeping to finish before bbstored exits

Should fix random test failures due to housekeeping process realising that the
master has asked it to terminate, and logging that it has, after the master has
already died and the test finished, confusing the test runner with its extra
output. Should also help make the occasionally-reported housekeeping crashes
more obvious and easier to debug.


  Commit: d9f32eb2253db868d71afafae974c7558b982ca7
      https://github.com/boxbackup/boxbackup/commit/d9f32eb2253db868d71afafae974c7558b982ca7
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h

  Log Message:
  -----------
  Remove useless Get/Put/HeadObject methods from S3BackupFileSystem

These should not have been part of the API, and are not needed since mrClient
has the same methods.


  Commit: 807c9502a4da53e294d829cc803b8a9179160c7f
      https://github.com/boxbackup/boxbackup/commit/807c9502a4da53e294d829cc803b8a9179160c7f
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupAccountControl.h
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreInfo.cpp
    M lib/backupstore/BackupStoreInfo.h
    R test/s3store/testfiles/store/subdir/dirs/create-me.txt
    M test/s3store/tests3store.cpp

  Log Message:
  -----------
  Refactor BackupAccountControl to use BackupFileSystem


  Commit: e6aa835e88bc47742ee1da8d69d52657be66cfd0
      https://github.com/boxbackup/boxbackup/commit/e6aa835e88bc47742ee1da8d69d52657be66cfd0
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupStoreFile.cpp
    M lib/backupstore/BackupStoreFile.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Refactor BackupStoreFile::VerifyStream from write to read side

This allows us to wrap an existing stream (not necessarily a file, e.g. a
streaming HTTPRequest) in a VerifyStream and pass that stream to a
S3PutFileCompleteTransaction, which isn't just going to CopyStreamTo()
somewhere, but do something more complex to it.


  Commit: 0ab2508cfd01a32dc39ce689d5b6b7e58a460e38
      https://github.com/boxbackup/boxbackup/commit/0ab2508cfd01a32dc39ce689d5b6b7e58a460e38
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/StoreTestUtils.cpp
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Fix a long-standing bug in the set_refcount() test helper function

Previously it would truncate the expected refcount list at the first zero entry,
instead of just removing zero entries from the end.

Also move some test code to test_server_commands which should have been there,
since they were causing issues with the check for the refcount database in
test_server_housekeeping. It's hard to disentangle these two changes.


  Commit: 1556500604d988635b9623ec9aa0b4418ed5efe4
      https://github.com/boxbackup/boxbackup/commit/1556500604d988635b9623ec9aa0b4418ed5efe4
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Add extra refcount checks to test_server_housekeeping


  Commit: 9474ba0f15502ecc278fe9dfcee007db2c24c842
      https://github.com/boxbackup/boxbackup/commit/9474ba0f15502ecc278fe9dfcee007db2c24c842
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupCommands.cpp
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreContext.cpp
    M lib/backupstore/BackupStoreContext.h
    M lib/backupstore/BackupStoreDirectory.h
    M lib/backupstore/BackupStoreException.txt
    M lib/backupstore/StoreTestUtils.cpp
    A lib/common/ByteCountingStream.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Refactor BackupStoreContext to use BackupFileSystem


  Commit: 3209e5a9f5b3f379a7b833b304932c77a2deabce
      https://github.com/boxbackup/boxbackup/commit/3209e5a9f5b3f379a7b833b304932c77a2deabce
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M infrastructure/makebuildenv.pl.in

  Log Message:
  -----------
  Enable and fix warnings in infrastructure/makebuildenv.pl


  Commit: 994526017764f89d7759de30988a81582dcada1d
      https://github.com/boxbackup/boxbackup/commit/994526017764f89d7759de30988a81582dcada1d
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M infrastructure/makebuildenv.pl.in

  Log Message:
  -----------
  Fix Makefile dependency tracking of configured files

This allows protocol files to be regenerated automatically if
makeprotocol.pl.in changes, and similarly exception files and
makeexception.pl.in.


  Commit: 20ce9cabcdf180d2063f1d40f402289bf0601887
      https://github.com/boxbackup/boxbackup/commit/20ce9cabcdf180d2063f1d40f402289bf0601887
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M bin/bbackupctl/bbackupctl.cpp
    M bin/bbackupquery/bbackupquery.cpp
    M lib/bbackupd/BackupDaemon.cpp
    M lib/bbstored/BBStoreDHousekeeping.cpp
    M lib/common/CommonException.txt
    M lib/common/FdGetLine.cpp
    M lib/common/FdGetLine.h
    R lib/common/GetLine.cpp
    R lib/common/GetLine.h
    M lib/common/IOStream.cpp
    M lib/common/IOStreamGetLine.cpp
    M lib/common/IOStreamGetLine.h
    A lib/common/LineBuffer.cpp
    A lib/common/LineBuffer.h
    M lib/httpserver/HTTPException.txt
    M lib/httpserver/HTTPHeaders.cpp
    M lib/httpserver/HTTPRequest.cpp
    M lib/httpserver/HTTPResponse.cpp
    M lib/httpserver/S3Simulator.cpp
    M lib/httpserver/S3Simulator.h
    M lib/server/ServerControl.cpp
    M lib/server/SocketStream.cpp
    A test/backupstore/testfiles/bbackupd.conf
    A test/backupstore/testfiles/s3simulator.conf
    M test/backupstorefix/testbackupstorefix.cpp
    M test/basicserver/testbasicserver.cpp
    M test/bbackupd/testbbackupd.cpp
    M test/common/testcommon.cpp

  Log Message:
  -----------
  Refactor GetLine

The class name was a problem, because the method that we want to call on it is
also called GetLine, which looks too much like a constructor in C++, so I
renamed it to LineBuffer.  I haven't yet renamed the child classes.

Also, it was impossible to distinguish between the different causes of
GetLine() returning false (timeout and signal) and EOF could only be checked by
calling a different function. Now all of these causes throw different
exceptions.

This really fixes the problem with attaching a debugger to a process waiting in
GetLine, which would previously cause the read() to return immediately with
EINTR, which we never retried because it looked like a timeout.


  Commit: 40cf96207139a31466b98a93298cef4f436eea6b
      https://github.com/boxbackup/boxbackup/commit/40cf96207139a31466b98a93298cef4f436eea6b
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/common/DebugMemLeakFinder.cpp
    M lib/common/MemLeakFinder.h

  Log Message:
  -----------
  Implement placement operator delete

"If the object is being created as part of a new expression, and an exception
is thrown, the object’s memory is deallocated by calling the appropriate
deallocation function. If the object is being created with a placement new
operator, the corresponding placement delete operator is called—that is, the
delete function that takes the same additional parameters as the placement new
operator. If no matching placement delete is found, no deallocation takes
place."

So to avoid memory leaks, we need to implement placement delete operators that
correspond to our placement new, which we use for leak detection (ironically)
in debug builds.


  Commit: fe65b06be724228a5932a5143ca7eec26374ef82
      https://github.com/boxbackup/boxbackup/commit/fe65b06be724228a5932a5143ca7eec26374ef82
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupclient/BackupDaemonConfigVerify.cpp
    M lib/backupstore/BackupAccountControl.cpp
    M lib/httpserver/S3Client.h

  Log Message:
  -----------
  Enable support for S3 virtual hosts in BackupAccountControl

This enables the S3 bucket name to be included in the virtual hostname, for
example bucketname.s3.amazonaws.com, when connecting to a different hostname
such as localhost. This makes bucket names usable in tests with
BackupAccountControl.


  Commit: 5cf84ed245a4a461c76361da7b1182a8713d7c84
      https://github.com/boxbackup/boxbackup/commit/5cf84ed245a4a461c76361da7b1182a8713d7c84
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreException.txt

  Log Message:
  -----------
  Properly implement S3BackupFileSystem::GetPermanentRefCountDatabase


  Commit: d76865fe3d787d246a803b1dd697f261e07d2607
      https://github.com/boxbackup/boxbackup/commit/d76865fe3d787d246a803b1dd697f261e07d2607
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp

  Log Message:
  -----------
  Remove always-false assertion in S3BackupFileSystem::GetDirectory

This was designed to stop untested code from running. Now we are ready to use
this code, with tests soon to come.


  Commit: 0326e3233b42ceeb579005564c93fc82222b6d04
      https://github.com/boxbackup/boxbackup/commit/0326e3233b42ceeb579005564c93fc82222b6d04
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/StoreTestUtils.cpp
    M lib/backupstore/StoreTestUtils.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Add store test helpers for specialised tests


  Commit: 8c809e1289cb3abb3d2b2e5706233ed5780508cd
      https://github.com/boxbackup/boxbackup/commit/8c809e1289cb3abb3d2b2e5706233ed5780508cd
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M lib/backupstore/BackupStoreContext.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Make BackupStoreContext::ClearDirectoryCache() public

Needed to clear the cache in backupstore tests specialised for S3.


  Commit: 5fad0fd1cb9a1a39756de69ce9348ca692e35bda
      https://github.com/boxbackup/boxbackup/commit/5fad0fd1cb9a1a39756de69ce9348ca692e35bda
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-10-07 (Sat, 07 Oct 2017)

  Changed paths:
    M test/httpserver/testhttpserver.cpp

  Log Message:
  -----------
  Test that 100-continue headers are sent and received correctly

This tests that HTTPRequest::SendHeaders includes the "Expect: 100-continue"
header only when expected. A separate test calls HTTPRequest::SendWithStream
which sends these headers, and therefore checks that both sides correctly
handle such exchanges.


Compare: https://github.com/boxbackup/boxbackup/compare/2279450b287a...5fad0fd1cb9a


More information about the Boxbackup-commit mailing list