[Box Backup-commit] [boxbackup/boxbackup] 63226a: Fix a long-standing bug in the set_refcount() test...

GitHub noreply at github.com
Sat Nov 11 19:52:39 GMT 2017


  Branch: refs/tags/BoxBackup-0.12.s3_support_merge.171111.1eb2d96
  Home:   https://github.com/boxbackup/boxbackup
  Commit: 63226abec0be13b6de6cd6e8901dbbf04464e63a
      https://github.com/boxbackup/boxbackup/commit/63226abec0be13b6de6cd6e8901dbbf04464e63a
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/StoreTestUtils.cpp
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Fix a long-standing bug in the set_refcount() test helper function

Previously it would truncate the expected refcount list at the first zero entry,
instead of just removing zero entries from the end.

Also move some test code to test_server_commands which should have been there,
since they were causing issues with the check for the refcount database in
test_server_housekeeping. It's hard to disentangle these two changes.


  Commit: 2ad2652c73b797e78781e66c682a94adc506a2b4
      https://github.com/boxbackup/boxbackup/commit/2ad2652c73b797e78781e66c682a94adc506a2b4
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Add extra refcount checks to test_server_housekeeping


  Commit: ce8f02e1a99ea5a05b19bf88303ee18dbc2d0b56
      https://github.com/boxbackup/boxbackup/commit/ce8f02e1a99ea5a05b19bf88303ee18dbc2d0b56
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupCommands.cpp
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreContext.cpp
    M lib/backupstore/BackupStoreContext.h
    M lib/backupstore/BackupStoreDirectory.h
    M lib/backupstore/BackupStoreException.txt
    M lib/backupstore/StoreTestUtils.cpp
    A lib/common/ByteCountingStream.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Refactor BackupStoreContext to use BackupFileSystem


  Commit: 8dca633bedeb380e792eebdbc12e4afcba426819
      https://github.com/boxbackup/boxbackup/commit/8dca633bedeb380e792eebdbc12e4afcba426819
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M infrastructure/makebuildenv.pl.in

  Log Message:
  -----------
  Enable and fix warnings in infrastructure/makebuildenv.pl


  Commit: 86aff1b0fffd2c16cc2db2039662038b41e2c339
      https://github.com/boxbackup/boxbackup/commit/86aff1b0fffd2c16cc2db2039662038b41e2c339
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M infrastructure/makebuildenv.pl.in

  Log Message:
  -----------
  Fix Makefile dependency tracking of configured files

This allows protocol files to be regenerated automatically if
makeprotocol.pl.in changes, and similarly exception files and
makeexception.pl.in.


  Commit: bde5cba76568dc4daff83456b8fd0bac675ebf94
      https://github.com/boxbackup/boxbackup/commit/bde5cba76568dc4daff83456b8fd0bac675ebf94
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M bin/bbackupctl/bbackupctl.cpp
    M bin/bbackupquery/bbackupquery.cpp
    M lib/bbackupd/BackupDaemon.cpp
    M lib/bbstored/BBStoreDHousekeeping.cpp
    M lib/common/CommonException.txt
    M lib/common/FdGetLine.cpp
    M lib/common/FdGetLine.h
    R lib/common/GetLine.cpp
    R lib/common/GetLine.h
    M lib/common/IOStream.cpp
    M lib/common/IOStreamGetLine.cpp
    M lib/common/IOStreamGetLine.h
    A lib/common/LineBuffer.cpp
    A lib/common/LineBuffer.h
    M lib/httpserver/HTTPException.txt
    M lib/httpserver/HTTPHeaders.cpp
    M lib/httpserver/HTTPRequest.cpp
    M lib/httpserver/HTTPResponse.cpp
    M lib/httpserver/S3Simulator.cpp
    M lib/httpserver/S3Simulator.h
    M lib/server/ServerControl.cpp
    M lib/server/SocketStream.cpp
    A test/backupstore/testfiles/bbackupd.conf
    A test/backupstore/testfiles/s3simulator.conf
    M test/backupstorefix/testbackupstorefix.cpp
    M test/basicserver/testbasicserver.cpp
    M test/bbackupd/testbbackupd.cpp
    M test/common/testcommon.cpp

  Log Message:
  -----------
  Refactor GetLine

The class name was a problem, because the method that we want to call on it is
also called GetLine, which looks too much like a constructor in C++, so I
renamed it to LineBuffer.  I haven't yet renamed the child classes.

Also, it was impossible to distinguish between the different causes of
GetLine() returning false (timeout and signal) and EOF could only be checked by
calling a different function. Now all of these causes throw different
exceptions.

This really fixes the problem with attaching a debugger to a process waiting in
GetLine, which would previously cause the read() to return immediately with
EINTR, which we never retried because it looked like a timeout.


  Commit: 7ccb056be0d5c9b005e868ffdcb64d44cc416a3b
      https://github.com/boxbackup/boxbackup/commit/7ccb056be0d5c9b005e868ffdcb64d44cc416a3b
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/common/DebugMemLeakFinder.cpp
    M lib/common/MemLeakFinder.h

  Log Message:
  -----------
  Implement placement operator delete

"If the object is being created as part of a new expression, and an exception
is thrown, the object’s memory is deallocated by calling the appropriate
deallocation function. If the object is being created with a placement new
operator, the corresponding placement delete operator is called—that is, the
delete function that takes the same additional parameters as the placement new
operator. If no matching placement delete is found, no deallocation takes
place."

So to avoid memory leaks, we need to implement placement delete operators that
correspond to our placement new, which we use for leak detection (ironically)
in debug builds.


  Commit: 84cbf95da2940ad9b7d7b4a5a22a7a8bfc56ca40
      https://github.com/boxbackup/boxbackup/commit/84cbf95da2940ad9b7d7b4a5a22a7a8bfc56ca40
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupclient/BackupDaemonConfigVerify.cpp
    M lib/backupstore/BackupAccountControl.cpp
    M lib/httpserver/S3Client.h

  Log Message:
  -----------
  Enable support for S3 virtual hosts in BackupAccountControl

This enables the S3 bucket name to be included in the virtual hostname, for
example bucketname.s3.amazonaws.com, when connecting to a different hostname
such as localhost. This makes bucket names usable in tests with
BackupAccountControl.


  Commit: 89910b216737a11272458bd30193f75d0a018b8f
      https://github.com/boxbackup/boxbackup/commit/89910b216737a11272458bd30193f75d0a018b8f
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupStoreException.txt

  Log Message:
  -----------
  Properly implement S3BackupFileSystem::GetPermanentRefCountDatabase


  Commit: 0ed728db33c2db9aad8872888a8792efd8863b56
      https://github.com/boxbackup/boxbackup/commit/0ed728db33c2db9aad8872888a8792efd8863b56
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp

  Log Message:
  -----------
  Remove always-false assertion in S3BackupFileSystem::GetDirectory

This was designed to stop untested code from running. Now we are ready to use
this code, with tests soon to come.


  Commit: cdb8fade4664e5c3376f2ee70761574ccce23cfe
      https://github.com/boxbackup/boxbackup/commit/cdb8fade4664e5c3376f2ee70761574ccce23cfe
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/StoreTestUtils.cpp
    M lib/backupstore/StoreTestUtils.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Add store test helpers for specialised tests


  Commit: bee97b67987c025a297dffbd6f5d03dc084b4d66
      https://github.com/boxbackup/boxbackup/commit/bee97b67987c025a297dffbd6f5d03dc084b4d66
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupStoreContext.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Make BackupStoreContext::ClearDirectoryCache() public

Needed to clear the cache in backupstore tests specialised for S3.


  Commit: effd39f798eaec59149cdb73b23dc7897c3ecd4d
      https://github.com/boxbackup/boxbackup/commit/effd39f798eaec59149cdb73b23dc7897c3ecd4d
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M test/httpserver/testhttpserver.cpp

  Log Message:
  -----------
  Test that 100-continue headers are sent and received correctly

This tests that HTTPRequest::SendHeaders includes the "Expect: 100-continue"
header only when expected. A separate test calls HTTPRequest::SendWithStream
which sends these headers, and therefore checks that both sides correctly
handle such exchanges.


  Commit: 34327a0b7d6d8065f63d4dadf5d88e1ab07b964b
      https://github.com/boxbackup/boxbackup/commit/34327a0b7d6d8065f63d4dadf5d88e1ab07b964b
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.h
    M lib/backupstore/BackupFileSystem.cpp
    M lib/backupstore/BackupFileSystem.h
    M lib/backupstore/BackupProtocol.h
    M lib/common/Test.cpp
    M lib/common/Test.h
    M lib/httpserver/S3Client.cpp
    M lib/server/makeprotocol.pl.in
    M modules.txt
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  Test that the S3 backend properly implements basic commands

This extends test_server_commands() to the S3 backend implementation, using
test specialisation.


  Commit: 0a54ae008567b192ef49a2e74ec8366bf4000aad
      https://github.com/boxbackup/boxbackup/commit/0a54ae008567b192ef49a2e74ec8366bf4000aad
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M infrastructure/buildenv-testmain-template.cpp

  Log Message:
  -----------
  Completely disable syslog logging in tests

(cherry picked from commit 3a8ca0c97e7346912889df72a1a87b80b430fe7a)


  Commit: 6e00d08156cd9d996e07b4f214afe5aed58cb3aa
      https://github.com/boxbackup/boxbackup/commit/6e00d08156cd9d996e07b4f214afe5aed58cb3aa
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    R infrastructure/cmake/build/bin_bbackupd.vcxproj.user
    R infrastructure/cmake/build/bin_bbstored.vcxproj.user
    R infrastructure/cmake/build/test_backupstore.vcxproj.user
    R infrastructure/cmake/build/test_backupstorefix.vcxproj.user
    R infrastructure/cmake/build/test_bbackupd.vcxproj.user
    R infrastructure/cmake/build/test_common.vcxproj.user
    R infrastructure/cmake/build/test_httpserver.vcxproj.user
    R infrastructure/cmake/build/test_raidfile.vcxproj.user

  Log Message:
  -----------
  Remove *.vcxproj.user from disposable CMake build directory

(cherry picked from commit 9df71c89a0fc935ee9965ddd8c4bcc2e1ad769a4)


  Commit: e893723ac597ce4b61e399b2501dcdfdea6e2c44
      https://github.com/boxbackup/boxbackup/commit/e893723ac597ce4b61e399b2501dcdfdea6e2c44
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M infrastructure/cmake/CMakeLists.txt
    M infrastructure/m4/boxbackup_tests.m4
    M lib/bbackupd/BackupClientContext.cpp
    M lib/bbackupd/BackupClientContext.h
    M lib/server/TcpNice.cpp
    M lib/server/TcpNice.h

  Log Message:
  -----------
  Add support for TcpNice on OSX, remove on Windows

I'm not convinced that it does anything on Windows without an RTT estimate,
which we can't get without administrator access:
https://msdn.microsoft.com/en-us/library/windows/desktop/bb485738(v=vs.85).aspx

(cherry picked from commit b646d22f415bbc8f5b5b065ed7462166591404f2)


  Commit: a29fe8922cc1c35aef9cba20110eb5b81762ba7a
      https://github.com/boxbackup/boxbackup/commit/a29fe8922cc1c35aef9cba20110eb5b81762ba7a
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp

  Log Message:
  -----------
  Move functions around to reduce diffs to s3_support


  Commit: 3fcc2df48d1370c01d781622b1fbbab3e135c03d
      https://github.com/boxbackup/boxbackup/commit/3fcc2df48d1370c01d781622b1fbbab3e135c03d
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupAccountControl.cpp
    M lib/backupstore/BackupAccountControl.h
    M lib/win32/getopt_long.cpp

  Log Message:
  -----------
  Fix compile error in BSD getopt_long.cpp

error: '&&' within '||' [-Werror,-Wlogical-op-parentheses]


  Commit: 7ceff38471b436a7da771907c0e28ed84d6f98fb
      https://github.com/boxbackup/boxbackup/commit/7ceff38471b436a7da771907c0e28ed84d6f98fb
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/httpserver/SimpleDBClient.h

  Log Message:
  -----------
  Fix SimpleDB timeouts: should be in milliseconds, not seconds

(cherry picked from commit 24bc49f30b9ce54d5024777f9c34d70a476d9663)


  Commit: 948674efde350daf9f4e57835d7adb62822dadd8
      https://github.com/boxbackup/boxbackup/commit/948674efde350daf9f4e57835d7adb62822dadd8
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/server/ServerStream.h
    M test/backupstore/testbackupstore.cpp

  Log Message:
  -----------
  ServerStream and test/backupstore: improve comments slightly

(cherry picked from commit c50aac67ee6ce6af9d63d5b96290585f2c5b7c21)


  Commit: b0ac03ce961955e37352223e138392e7ff273497
      https://github.com/boxbackup/boxbackup/commit/b0ac03ce961955e37352223e138392e7ff273497
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/httpserver/S3Simulator.cpp

  Log Message:
  -----------
  S3Simulator: add some TRACE logging to help debug 100-continue responses

(cherry picked from commit ca39b88d4a73b64de68400dfa29270f203fd0f14)


  Commit: 74621e10b94519f2b3feb3426ef8fb94e15bf024
      https://github.com/boxbackup/boxbackup/commit/74621e10b94519f2b3feb3426ef8fb94e15bf024
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/httpserver/S3Client.h

  Log Message:
  -----------
  Increase timeouts in S3Client to help with debugging

(cherry picked from commit cc14b3b4b9ec509233000c7d0a995580afd97f15)


  Commit: b35de723233e6235eb9d3376504c8a0f54c5b2ef
      https://github.com/boxbackup/boxbackup/commit/b35de723233e6235eb9d3376504c8a0f54c5b2ef
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/server/SocketStream.cpp

  Log Message:
  -----------
  SocketStream: use THROW_SOCKET_ERROR for better messages on Windows

(cherry picked from commit feddbef8052208508a2e542a1a70c23138f476b8)


  Commit: 550ac542e9bfe2885a79825fd03d6b8b54ea6d2d
      https://github.com/boxbackup/boxbackup/commit/550ac542e9bfe2885a79825fd03d6b8b54ea6d2d
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/common/Logging.cpp
    M lib/common/Logging.h

  Log Message:
  -----------
  Enable overriding log level by category as well as filename

This extends the -L command-line option to allow overriding by
filename/category=level as well as the existing support for overriding by
filename=level.

(cherry picked from commit 4421c53ab59adb494308f9c9f881a6a57bfd571c)


  Commit: d118c46b79ef531107f300d2ed630927832ce8ff
      https://github.com/boxbackup/boxbackup/commit/d118c46b79ef531107f300d2ed630927832ce8ff
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/win32/emu.cpp

  Log Message:
  -----------
  Windows syslog() emulation: improve error messages

If syslog can't log to the Windows event log for some reason, instead of
just printing the log message to the console, include some explanatory
text about what went wrong.

(cherry picked from commit c492c90062d2e88db1016f13bd912629477a8a46)


  Commit: 7d0d27123fa1108f68a88b7c5a6dd7e30467db93
      https://github.com/boxbackup/boxbackup/commit/7d0d27123fa1108f68a88b7c5a6dd7e30467db93
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/server/ServerStream.h

  Log Message:
  -----------
  Improve exception handling in single-process servers

Don't allow an uncaught exception to kill the whole server. This makes it
consistent with multi-process servers, where death of a child worker does
not stop the main server process, so it continues to serve requests.

This makes the HTTPServer robust to exceptions (such as the child
disconnecting while the server is streaming a response) on Windows.

(cherry picked from commit 4a515b9ce8e616b357dc07bee27aa7b3692c3602)


  Commit: 1eb2d9671456e9ccf635b9d11620ef73c7356cd8
      https://github.com/boxbackup/boxbackup/commit/1eb2d9671456e9ccf635b9d11620ef73c7356cd8
  Author: Chris Wilson <chris+github at qwirx.com>
  Date:   2017-11-11 (Sat, 11 Nov 2017)

  Changed paths:
    M lib/backupstore/BackupFileSystem.cpp

  Log Message:
  -----------
  Fix rare crash while tearing down S3BackupFileSystem

ReleaseLock() could fail if it can't contact the S3/SimpleDB server, but
destructors aren't allowed to throw exceptions, so catch this case and log
a warning instead (because we failed to release the lock that we were
holding).

(cherry picked from commit 6226c014dd2988a6a29cd21f6cc29fa0e7816607)


Compare: https://github.com/boxbackup/boxbackup/compare/63226abec0be^...1eb2d9671456


More information about the Boxbackup-commit mailing list