[Box Backup-commit] [boxbackup/boxbackup] 312afe: S3Simulator: log HTTP requests at TRACE level, not...
Chris Wilson
noreply at github.com
Fri Jun 21 00:04:46 BST 2019
Branch: refs/heads/s3_support
Home: https://github.com/boxbackup/boxbackup
Commit: 312afed730c278b40d2895dd8d4fea08d387a812
https://github.com/boxbackup/boxbackup/commit/312afed730c278b40d2895dd8d4fea08d387a812
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-20 (Thu, 20 Jun 2019)
Changed paths:
M lib/httpserver/S3Simulator.cpp
Log Message:
-----------
S3Simulator: log HTTP requests at TRACE level, not NOTICE
We don't want to see all of them! They are no longer high priority messages.
Commit: 53da812112eed1a58eb805c64a22caf3472a7e54
https://github.com/boxbackup/boxbackup/commit/53da812112eed1a58eb805c64a22caf3472a7e54
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-20 (Thu, 20 Jun 2019)
Changed paths:
M lib/server/ServerControl.cpp
Log Message:
-----------
WaitForServerStartup: don't require the pidFile parameter
Add support for waiting for a server to start even if it doesn't write PID
files (e.g. forked local daemons in test/bbackupd). In this case we know the
PID from the fork() return code, so we don't need a PID file to determine it.
Commit: be2cb4766aaddb63e52bf929ad9fdb66d50af166
https://github.com/boxbackup/boxbackup/commit/be2cb4766aaddb63e52bf929ad9fdb66d50af166
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-20 (Thu, 20 Jun 2019)
Changed paths:
M lib/backupstore/StoreTestUtils.cpp
M lib/backupstore/StoreTestUtils.h
Log Message:
-----------
StopSimulator: add a wait_for_process argument
Useful for waiting for a forked simulator daemon in test/bbackupd to shut down.
Commit: b3e327f9d500d8b328b6ed01d42d975bd7df1d13
https://github.com/boxbackup/boxbackup/commit/b3e327f9d500d8b328b6ed01d42d975bd7df1d13
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-20 (Thu, 20 Jun 2019)
Changed paths:
M .gitignore
M configure.ac
M infrastructure/cmake/CMakeLists.txt
M lib/bbackupd/BackupDaemon.cpp
M lib/common/Test.cpp
M lib/common/Test.h
M lib/httpserver/HTTPException.txt
M lib/httpserver/HTTPServer.cpp
M lib/httpserver/S3Client.h
M lib/httpserver/S3Simulator.cpp
M lib/server/ServerControl.cpp
M lib/server/ServerControl.h
M test/bbackupd/testbbackupd.cpp
A test/bbackupd/testfiles/bbackupd-snapshot.s3.conf.in
Log Message:
-----------
test/bbackupd: add specialised versions of some tests for S3
Not all tests are specialised because many are quite slow, and would behave in
the same way, not expected to exercise any additional S3-related code paths
that are not covered by the tests that have been implemented.
Commit: a9be6f8ff88864e640ca3451f61de061ce939d92
https://github.com/boxbackup/boxbackup/commit/a9be6f8ff88864e640ca3451f61de061ce939d92
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/common/Test.cpp
M lib/server/ServerControl.h
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
test/bbackupd: add a test for bbackupd-config and bbstored-config scripts
Test creating new clients and servers, signing their certificates and running a
test backup.
Commit: 8aef1bcfc4affb101b478f50c4f48b10bbbcfb19
https://github.com/boxbackup/boxbackup/commit/8aef1bcfc4affb101b478f50c4f48b10bbbcfb19
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
test/bbackupd: fix test_interrupted_restore_can_be_recovered for Windows
Commit: 7b1fd6846e0d6b7408c0bfae049af8065988cdc5
https://github.com/boxbackup/boxbackup/commit/7b1fd6846e0d6b7408c0bfae049af8065988cdc5
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupFileSystem.cpp
Log Message:
-----------
BackupFileSystem: add details to "Failed to lock account" error message
Commit: 9b88ffc301b2b45311a032941102d0a12ad77780
https://github.com/boxbackup/boxbackup/commit/9b88ffc301b2b45311a032941102d0a12ad77780
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupFileSystem.cpp
Log Message:
-----------
S3BackupFileSystem: Add warning when downloading entire files from S3
This should serve as a more visible reminder to implement block index caching,
and reduce wasteful re-downloading of entire files.
Commit: 8fcef7d3f9ca8d90079293050a1a9e467abfdbab
https://github.com/boxbackup/boxbackup/commit/8fcef7d3f9ca8d90079293050a1a9e467abfdbab
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupStoreRefCountDatabase.cpp
M lib/common/BufferedStream.cpp
M lib/common/BufferedStream.h
M lib/common/BufferedWriteStream.cpp
M lib/common/BufferedWriteStream.h
M lib/common/CollectInBufferStream.cpp
M lib/common/CollectInBufferStream.h
M lib/common/FileStream.cpp
M lib/common/FileStream.h
M lib/common/IOStream.cpp
M lib/common/IOStream.h
M lib/common/MemBlockStream.cpp
M lib/common/MemBlockStream.h
M lib/common/RateLimitingStream.h
M lib/common/ReadLoggingStream.cpp
M lib/common/ReadLoggingStream.h
M lib/common/TeeStream.h
M lib/common/ZeroStream.cpp
M lib/common/ZeroStream.h
M lib/crypto/MD5Digest.h
M lib/raidfile/RaidFileRead.cpp
M lib/raidfile/RaidFileWrite.cpp
M lib/raidfile/RaidFileWrite.h
M lib/server/TcpNice.h
Log Message:
-----------
Convert second argument of IOStream::Seek to a typed seek_type value
Should prevent accidentally passing arguments to Seek() the wrong way around.
Remove unused (duplicate) static functions, and change the type of
ConvertSeekTypeToOSWhence's argument to match.
Commit: 149eb4ebc794d28f1f6f35d770a0bdb4ceea715a
https://github.com/boxbackup/boxbackup/commit/149eb4ebc794d28f1f6f35d770a0bdb4ceea715a
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/common/Test.cpp
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
test/bbackupd: loosen timing tolerances on test_continuously_updated_file
Commit: 42541cbc55db6e3c3d9417b3bd2b095b107c99a5
https://github.com/boxbackup/boxbackup/commit/42541cbc55db6e3c3d9417b3bd2b095b107c99a5
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M infrastructure/makebuildenv.pl.in
Log Message:
-----------
makebuildenv.pl: rename @items to @sources, for code readability
Commit: 151b624da162dd78e390569ff766d2f9432315f2
https://github.com/boxbackup/boxbackup/commit/151b624da162dd78e390569ff766d2f9432315f2
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupAccountControl.cpp
Log Message:
-----------
BackupAccountControl: hide expected exception warnings during account creation
Commit: 228442756abc1d566f18f7df134cbbbee21256d9
https://github.com/boxbackup/boxbackup/commit/228442756abc1d566f18f7df134cbbbee21256d9
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupAccountControl.cpp
M lib/backupstore/BackupAccountControl.h
M lib/backupstore/BackupFileSystem.cpp
M lib/backupstore/BackupFileSystem.h
Log Message:
-----------
BackupAccountControl: add CloseAccount, and use it to explicitly unlock ASAP
Add an overload of DiscardBackupStoreInfo() which doesn't need a reference
passed in, for convenience.
Commit: 8bf442058c078af1a1823d441d642568a645217c
https://github.com/boxbackup/boxbackup/commit/8bf442058c078af1a1823d441d642568a645217c
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupCommands.cpp
M lib/backupstore/BackupProtocol.txt
Log Message:
-----------
Add a new protocol command, Finished2
This one returns the current (new) client store marker, which will be useful
when the BackupFileSystem updates it automatically, taking control away from
the BackupDaemon, which will then need to retrieve the current (new) value when
logging out, to use to ensure that its cache is still valid when logging in
again.
Commit: d59a09d0a404a731ab9de9d6940931459b8d11b9
https://github.com/boxbackup/boxbackup/commit/d59a09d0a404a731ab9de9d6940931459b8d11b9
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupCommands.cpp
M lib/backupstore/BackupConstants.h
M lib/bbackupd/BackupClientContext.cpp
M lib/bbackupd/BackupClientContext.h
M lib/bbackupd/BackupDaemon.cpp
Log Message:
-----------
Refactor ClientStoreMarker::NotKnown out of BackupClientContext.h
Commit: e2b0e00a54eedcbcab145fd8ea0c82db0c1cf18e
https://github.com/boxbackup/boxbackup/commit/e2b0e00a54eedcbcab145fd8ea0c82db0c1cf18e
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupFileSystem.cpp
M lib/backupstore/BackupStoreRefCountDatabase.cpp
M lib/backupstore/BackupStoreRefCountDatabase.h
M test/backupstore/testbackupstore.cpp
Log Message:
-----------
BackupStoreRefCountDatabase: new file format
This format includes an embedded client store marker, which is useful on S3
stores to verify that we haven't downloaded an old version of the refcount DB,
particularly if it was repaired by checking the account, and subsequently not
propagated due to Amazon S3's eventual (not immediate) consistency.
Commit: 3dd479680e349acf1841a1738f64b6aef9f52d3e
https://github.com/boxbackup/boxbackup/commit/3dd479680e349acf1841a1738f64b6aef9f52d3e
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/server/ServerControl.cpp
M lib/server/ServerControl.h
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
ServerControl: add signal checks and wait() wrapper
KillServer: add a parameter to allow the expected exit signal to be checked.
This allows us to determine if a process died for the wrong reason, e.g. if it
failed to catch SIGTERM, or was killed by SIGSEGV instead.
WaitForProcessExit: split out, to make it easier to clean up after forked
daemons, and check their exit status and signal.
test/bbackupd: use these to improve handling and failure mode checking of
forking daemon tests, reducing risk of hangs.
Commit: 2213d508cecf1c7ad19ffa69b76d35f261ee5d5b
https://github.com/boxbackup/boxbackup/commit/2213d508cecf1c7ad19ffa69b76d35f261ee5d5b
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/bbackupd/BackupDaemon.cpp
M lib/bbackupd/BackupDaemon.h
Log Message:
-----------
BackupDaemon: simplify DeserializeStoreObjectInfo
Reuse the existing ResetCachedState method, to reduce code duplication.
Commit: 621e6847d90cfadf461c69449274bb8125387bea
https://github.com/boxbackup/boxbackup/commit/621e6847d90cfadf461c69449274bb8125387bea
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/bbackupd/BackupClientContext.cpp
Log Message:
-----------
BackupClientContext: tidy up headers
Commit: f196cdc59f6cdd30b240a194c7e3219d2345a1f2
https://github.com/boxbackup/boxbackup/commit/f196cdc59f6cdd30b240a194c7e3219d2345a1f2
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/bbackupd/BackupDaemon.cpp
Log Message:
-----------
BackupDaemon: reduce logging noise when control socket client disconnects
Commit: b281299dea34dd64e2fab1ea201d2a4b28b7be1f
https://github.com/boxbackup/boxbackup/commit/b281299dea34dd64e2fab1ea201d2a4b28b7be1f
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M test/backupstorefix/testbackupstorefix.cpp
M test/backupstorefix/testfiles/bbackupd.bbstored.conf
Log Message:
-----------
test/backupstorefix: enable log output from bbackupd during test
Add extra logging to help debug random test failures on Travis due to inability
to connect to the daemon at StopClient() time. Because of this extra verbosity,
disable LogAllFileAccess to avoid bloating the logs even more.
Commit: 80a81a8b69722c97babd0b853521cec7bb08d148
https://github.com/boxbackup/boxbackup/commit/80a81a8b69722c97babd0b853521cec7bb08d148
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M .travis.yml
Log Message:
-----------
Travis: remove deprecated sudo: false option
"Container-based infrastructure is currently being deprecated. Please remove
any sudo: false keys in your .travis.yml file to use the default
fully-virtualized Linux infrastructure instead."
https://docs.travis-ci.com/user/reference/trusty/#container-based-infrastructure
Commit: 359888d803f0fccdaa2bbe6da02af25d71d68054
https://github.com/boxbackup/boxbackup/commit/359888d803f0fccdaa2bbe6da02af25d71d68054
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M infrastructure/cmake/CMakeLists.txt
Log Message:
-----------
CMake buildsystem: fix AC_SEARCH_LIBS emulation
When we detect a function in a library, we should continue to link against
that library for all future feature tests.
Commit: 28faae9b4a16712afeb8b5b8fc24a8840fa5a9c3
https://github.com/boxbackup/boxbackup/commit/28faae9b4a16712afeb8b5b8fc24a8840fa5a9c3
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/common/Logging.cpp
Log Message:
-----------
LogLevelOverrideByFileGuard: update lowest common log level when installed
Allows the global log level to be increased, so that it doesn't filter out the
messages that you requested to be logged at a higher level.
Commit: ae4f2124714337c6d604722418c6f4521890a3fe
https://github.com/boxbackup/boxbackup/commit/ae4f2124714337c6d604722418c6f4521890a3fe
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/bbackupd/BackupClientDirectoryRecord.cpp
M lib/bbackupd/BackupClientDirectoryRecord.h
M test/bbackupd/testbbackupd.cpp
A test/bbackupd/testfiles/bbackupd.logall.s3.conf
Log Message:
-----------
BackupClientDirectoryRecord: improve logging of file move decisions
Also improve logging of directory listing download decisions.
test/bbackupd: increase verbosity in test_backup_hardlinked_files to help debug
test failures on Travis which I have not been able to reproduce locally.
Commit: 6e6a43556252c2203a9df38ed818ed99ceabaf75
https://github.com/boxbackup/boxbackup/commit/6e6a43556252c2203a9df38ed818ed99ceabaf75
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M bin/bbackupquery/bbackupquery.cpp
M lib/backupclient/ConfiguredBackupClient.cpp
M lib/backupclient/ConfiguredBackupClient.h
M lib/bbackupd/BackupClientContext.cpp
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
Refactor GetConfiguredBackupClient to not Login automatically
This makes Login() a separate, publicly visible operation, which can also check
the client store marker against an expected value, and raise an exception if it
differs, making this code easily reusable.
Commit: 25fc64e43c51254a989cb003809353795ab3ea35
https://github.com/boxbackup/boxbackup/commit/25fc64e43c51254a989cb003809353795ab3ea35
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupProtocol.h
M lib/bbackupd/BackupClientDirectoryRecord.cpp
M lib/bbackupd/BackupClientDirectoryRecord.h
M lib/bbackupd/BackupDaemon.cpp
M lib/bbackupd/BackupDaemon.h
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
Fix failing test_backup_hardlinked_files
Remove assertion that inode numbers were not reused, which was incompatible
with hard links and violated under certain circumstances. Add a test that
reproduces those circumstances.
Commit: 2835ee6339457cd0102899e5b77bd7bb1d8c8994
https://github.com/boxbackup/boxbackup/commit/2835ee6339457cd0102899e5b77bd7bb1d8c8994
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/backupstore/BackupFileSystem.cpp
M lib/backupstore/BackupFileSystem.h
M lib/backupstore/BackupStoreCheck.cpp
M lib/backupstore/HousekeepStoreAccount.cpp
M lib/common/Logging.cpp
M lib/common/Logging.h
M lib/common/Test.cpp
Log Message:
-----------
Improve handling of exceptions in destructors
Destructors are not allowed to throw exceptions, and although we log a warning
if we catch one, it was easy to miss in automated tests. Now the exception is
recorded and causes a test failure during tearDown() instead.
Commit: a2fb2051eb502eb9a4c9ffc50e76ca3a9777bda7
https://github.com/boxbackup/boxbackup/commit/a2fb2051eb502eb9a4c9ffc50e76ca3a9777bda7
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M configure.ac
R infrastructure/m4/ax_check_bdb_v1.m4
R infrastructure/m4/ax_path_bdb.m4
M infrastructure/m4/boxbackup_tests.m4
Log Message:
-----------
Remove unused support for Berkeley DB
Replaced long ago by the built-in copy of QDBM.
Commit: eead8914d4a6d2f44fa5abcb5cf4bfc3440bb13d
https://github.com/boxbackup/boxbackup/commit/eead8914d4a6d2f44fa5abcb5cf4bfc3440bb13d
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/server/ServerControl.cpp
Log Message:
-----------
Fix KillServer on Windows
Don't hold the PID file open when trying to delete it, as this prevents
us from deleting it on Windows. If we still can't delete it, then
another process (a zombie server) must have it open.
Commit: c369e0417b6c8acefcc7bd8d965d8fb273614160
https://github.com/boxbackup/boxbackup/commit/c369e0417b6c8acefcc7bd8d965d8fb273614160
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M lib/win32/emu.cpp
M lib/win32/emu.h
Log Message:
-----------
Fix e7c3d2f144c76a15fa3bbdd330bcaa90f337da67 on Windows
Code now relies on being able to copy struct dirents, but our Windows
emulation library used a pointer to a static buffer for the filename,
which was overwritten while iterating over the directory. Now fixed by
replacing it with a static buffer in struct dirent, as Unixes seem to.
Commit: 45172b14b485a05c2adf55edcf82fd14154002cb
https://github.com/boxbackup/boxbackup/commit/45172b14b485a05c2adf55edcf82fd14154002cb
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
Simplify and rename test_bbackupd_exclusions
Commit: aa010f0f861e5bbbe3460022cff16113b02b5a3c
https://github.com/boxbackup/boxbackup/commit/aa010f0f861e5bbbe3460022cff16113b02b5a3c
Author: Chris Wilson <chris+github at qwirx.com>
Date: 2019-06-21 (Fri, 21 Jun 2019)
Changed paths:
M test/bbackupd/testbbackupd.cpp
Log Message:
-----------
Fix failing test/bbackupd tests
When did they get broken, and how?
Compare: https://github.com/boxbackup/boxbackup/compare/a8ab83214673...aa010f0f861e
More information about the Boxbackup-commit
mailing list