From noreply at github.com Wed Aug 2 20:20:56 2017 From: noreply at github.com (GitHub) Date: Wed, 02 Aug 2017 12:20:56 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 19032c: Add tests for S3Simulator and S3Client Message-ID: <59822618a6477_4f793fa6326bfc341088e@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 19032c5c79bd70898e5d935d52a8b16d4c71d189 https://github.com/boxbackup/boxbackup/commit/19032c5c79bd70898e5d935d52a8b16d4c71d189 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/httpserver/HTTPResponse.cpp M lib/httpserver/S3Client.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add tests for S3Simulator and S3Client Commit: 1ef9b6caf1787a81620ace6085dca6de04fa1fc8 https://github.com/boxbackup/boxbackup/commit/1ef9b6caf1787a81620ace6085dca6de04fa1fc8 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/crypto/MD5Digest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add MD5DigestStream and test (cherry picked from commit 3f0538bf1a2115c399f09a1735e7e9f3b426737e) Commit: 3463fcb2e4b9ae602e45557efbd2073f281a0be1 https://github.com/boxbackup/boxbackup/commit/3463fcb2e4b9ae602e45557efbd2073f281a0be1 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Refactor S3Simulator to better support merge of SimpleDB support (cherry picked from commit 35b581b3ac54769f5b343c995435a7c7b15041e7) Commit: 34adb89de4c7e823854023169ac256e18c868c83 https://github.com/boxbackup/boxbackup/commit/34adb89de4c7e823854023169ac256e18c868c83 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: R lib/backupclient/BackupStoreObjectDump.cpp A lib/backupstore/BackupStoreObjectDump.cpp Log Message: ----------- Move BackupStoreObjectDump.cpp to lib/backupstore It doesn't depend on anything in lib/backupclient, so there's no need for it to be there. (cherry picked from commit 85fda42a5164cc574b9a9b0335d0cb14bbb802b0) Commit: 5df14f56928a1a64661c44b7bc5fce6da807cca6 https://github.com/boxbackup/boxbackup/commit/5df14f56928a1a64661c44b7bc5fce6da807cca6 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Remove unused intercept code from testbbackupd. (cherry picked from commit 56631b8da4f8f22f291323d640560aca43f32b29) Commit: 65197c00af4ddf52d993b43ac5f2d0f5e88f4320 https://github.com/boxbackup/boxbackup/commit/65197c00af4ddf52d993b43ac5f2d0f5e88f4320 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/backupstore/BackupStoreContext.cpp M lib/common/CommonException.txt M lib/common/IOStream.cpp M lib/common/IOStream.h M lib/server/Protocol.cpp Log Message: ----------- Make IOStream::CopyStreamTo honour timeout Throw an exception on network timeout. This is safer than returning false, which isn't checked in many places, and allows us to return the number of bytes copied instead. (cherry picked from commit e38d3d771ebb303e75e635e618f4d1e41370aa23) Commit: 91592fdc1d5f4db8ea95a6a919ece51d3333711e https://github.com/boxbackup/boxbackup/commit/91592fdc1d5f4db8ea95a6a919ece51d3333711e Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/common/CollectInBufferStream.cpp Log Message: ----------- Add an assertion to CollectInBufferStream Ensure that we don't try to write a negative number of bytes. (cherry picked from commit 755242a46275384592281878ad6cce8dc7883be7) Commit: 8501574041dc3ac5383ee334b35b37d11b06909a https://github.com/boxbackup/boxbackup/commit/8501574041dc3ac5383ee334b35b37d11b06909a Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/httpserver/S3Client.cpp Log Message: ----------- Remove unwanted prototype for HeadObject() (cherry picked from commit 71d3d970b00ee85408e8ea5166fe46fdbac82ce1) Commit: 5bd70fc54ae85e6a2938c79f95f483500957240b https://github.com/boxbackup/boxbackup/commit/5bd70fc54ae85e6a2938c79f95f483500957240b Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/httpserver/HTTPResponse.cpp M lib/httpserver/HTTPResponse.h M lib/httpserver/HTTPServer.cpp M lib/httpserver/HTTPServer.h Log Message: ----------- Allow control of network timeout in HTTPResponse::Send() (cherry picked from commit aabb13634e2ef79eb2843f8ab2f0cbd7be267c38) Commit: b29474769eee7cca798440192c908ddd82f699c4 https://github.com/boxbackup/boxbackup/commit/b29474769eee7cca798440192c908ddd82f699c4 Author: Chris Wilson Date: 2017-07-31 (Mon, 31 Jul 2017) Changed paths: M lib/httpserver/S3Client.cpp Log Message: ----------- Log every request made using S3Client (for now) (cherry picked from commit 699a70598942c1422f568843176542b9b302c1dd) Commit: 36f0d51f77a65561bb136a2576a8cbd3938f5cde https://github.com/boxbackup/boxbackup/commit/36f0d51f77a65561bb136a2576a8cbd3938f5cde Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp Log Message: ----------- Add initial (non-functioning) keepalive support in S3Client (cherry picked from commit 3c62a5be57e0a7b7100b44c41cc8c69c20c92fa9) Commit: 8791ea9a7ac465677a3e95f9160c7fa221360ca3 https://github.com/boxbackup/boxbackup/commit/8791ea9a7ac465677a3e95f9160c7fa221360ca3 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M test/s3store/tests3store.cpp Log Message: ----------- Fix confusion about URIs and paths in S3 filesystem * Make it clear (with parameter names and comments) which functions take paths (relative to the configured base path), which functions take URIs (complete paths) and which functions take URLs (hostname, port and URI). * Put all objects (files and directories) in the same "directory" tree on S3, so that we can list them all together and easily identify the type of each object (from its extension). Previously they were separated into "directories" (delimited prefixes) called "files" and "dirs" (unlike BackupStore filesystems). * Reinstate hierarchical "directories" on S3, similar to BackupStore filesystems, since the reasons for doing this in RaidFile may well apply to S3 and other compatible stores too. However the objects are still named with their complete object ID (unlike RaidFile) and a .file or .dir extension. (cherry picked from commit 3dd21a3028cacea2ecf6ee3de2f32db3c5138589) Commit: 0cf501ea0aeceacfceb679da7e40271bc06b3e4f https://github.com/boxbackup/boxbackup/commit/0cf501ea0aeceacfceb679da7e40271bc06b3e4f Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Fix quoting of CXXFLAGS and LDFLAGS in Makefiles Previously, it was impossible to include an @ sign in these flags (for example, to link to support --with-ssl-headers=/usr/local/opt/openssl at 1.1/include/) because it was interpreted as a list variable name by Perl. Now the variables are evaluated single-quoted before being substituted into the Makefile output. Commit: ccaad16aee6e15586dd7a7f7a4f9fb177add8466 https://github.com/boxbackup/boxbackup/commit/ccaad16aee6e15586dd7a7f7a4f9fb177add8466 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/crypto/Random.cpp Log Message: ----------- Replace RAND_pseudo_bytes with RAND_bytes RAND_pseudo_bytes is less secure and is now deprecated. Commit: 53fa9606bf5f96c6a8bc2710e6f47485df3e4d3d https://github.com/boxbackup/boxbackup/commit/53fa9606bf5f96c6a8bc2710e6f47485df3e4d3d Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupStoreAccounts.cpp M lib/backupstore/BackupStoreAccounts.h Log Message: ----------- Move BackupStoreAccountControl code to BackupAccountControl file (cherry picked from commit 2891416b675cdc346713a26c6e357fba302bbc89) Commit: a67cb0e228c8754cb930e4ce05530a63ed8fabf9 https://github.com/boxbackup/boxbackup/commit/a67cb0e228c8754cb930e4ce05530a63ed8fabf9 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M .travis.yml M infrastructure/cmake/CMakeLists.txt A infrastructure/m4/ax_boost_base.m4 M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Add Boost, needed for XML support in S3Simulator and S3Client Boost will be needed on all platforms for S3 support at least. It's currently mandatory, but could be made optional if necessary. XML support of some kind is required both for S3's ListBucket command (which returns XML) and for SimpleDB support in the S3Simulator, since the SimpleDB API uses XML. I considered alternative XML libraries to Boost: * https://stackoverflow.com/questions/170686/what-is-the-best-open-xml-parser-for-c * https://stackoverflow.com/questions/9387610/what-xml-parser-should-i-use-in-c The most commonly used one (libxml2) is huge, ugly, slow and hard to use, and we don't need any of the extra features. TinyXML would meet our needs, but Boost is more likely to be available on more platforms, is well respected, and brings other (unrelated) features that we might want to use in future. (cherry picked from commit 1a9deec05151281e840b7df5a1244453c0111042) Commit: f79f99f8ed9b4c4c3baca5f4a46d5d764533aa5a https://github.com/boxbackup/boxbackup/commit/f79f99f8ed9b4c4c3baca5f4a46d5d764533aa5a Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/cdecode.cpp Log Message: ----------- Declare Base64 decoding array as signed char Need to use "signed char" explicitly to work around char being unsigned by default on ARM, causing compile errors: https://stackoverflow.com/a/31635045/648162 Commit: 6ffd63f166d5a7050e84ac6bb8cf5df636f42e31 https://github.com/boxbackup/boxbackup/commit/6ffd63f166d5a7050e84ac6bb8cf5df636f42e31 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 Commit: 385e36858e51dc771b89336b0d41fd0e6c9a7b24 https://github.com/boxbackup/boxbackup/commit/385e36858e51dc771b89336b0d41fd0e6c9a7b24 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/raidfile/RaidFileRead.cpp M lib/raidfile/RaidFileRead.h Log Message: ----------- Use a named enum type for DirReadType (RaidFileRead::ReadDirectoryContents) Allows checking for valid values at compile time, and allows the debugger to show the symbolic constant instead of just its numeric value. Commit: 33b1cf8c2683910c09f5cdc0d9f4103a2283feae https://github.com/boxbackup/boxbackup/commit/33b1cf8c2683910c09f5cdc0d9f4103a2283feae Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/S3Client.h Log Message: ----------- Add comments to explain the three S3Client constructors Commit: 0472617ee08972dc9a28dcf1591da7f179997736 https://github.com/boxbackup/boxbackup/commit/0472617ee08972dc9a28dcf1591da7f179997736 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M lib/common/BoxPlatform.h M lib/intercept/intercept.h M test/raidfile/testraidfile.cpp Log Message: ----------- Enable and fix raidfile intercepts on Mac OSX Commit: 34d29b0b000b19fd7f707d1a8cda1745097f8d89 https://github.com/boxbackup/boxbackup/commit/34d29b0b000b19fd7f707d1a8cda1745097f8d89 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/m4/ax_check_dirent_d_type.m4 Log Message: ----------- Fix check for a working struct dirent.d_type Which I broke in 2007 and never noticed was broken until now. Oops! Commit: 3b4c14851879cd37a9b171e01e6044332236ad66 https://github.com/boxbackup/boxbackup/commit/3b4c14851879cd37a9b171e01e6044332236ad66 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/buildenv-testmain-template.cpp Log Message: ----------- Fix exit status from tests: could previously return 0 even after failures Commit: fb8f8a3747ec4d10ed08485145c71273f4743e09 https://github.com/boxbackup/boxbackup/commit/fb8f8a3747ec4d10ed08485145c71273f4743e09 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Allow 1 extra second in test_store_error_reporting Hopefully this will solve the random test failures on Travis. Commit: f1ef4519a6bc1cf200dee57c8c7b02bc6225340e https://github.com/boxbackup/boxbackup/commit/f1ef4519a6bc1cf200dee57c8c7b02bc6225340e Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.h Log Message: ----------- Fix delete of incomplete type in BackupAccountControl.h This has been flagged up by MSVC for some time. I've been unable to make GCC or Clang do so. Commit: 2d51225149478de954c710c31f399883bcf607b9 https://github.com/boxbackup/boxbackup/commit/2d51225149478de954c710c31f399883bcf607b9 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Test.cpp Log Message: ----------- Log time taken by each test case in a test suite. Should help diagnose why test/backupstore sometimes takes >500 seconds on Travis, when it normally takes ~120 seconds. Commit: c249bab3fb624fd00362fe4841d4accf98fb8d3b https://github.com/boxbackup/boxbackup/commit/c249bab3fb624fd00362fe4841d4accf98fb8d3b Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt Log Message: ----------- Add support for RelWithDebInfo builds using CMake Debugging release tests is pretty difficult without debug info. This release-compatible build makes it easier. Commit: b930ba654659f9561ed10e554a318b6b980a5ece https://github.com/boxbackup/boxbackup/commit/b930ba654659f9561ed10e554a318b6b980a5ece Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt Log Message: ----------- CMake: make readline support configurable, not automatic Otherwise if it's detected on the system (e.g. if you install GnuWin) then it would always be enabled, but linking against GnuWin headers breaks the build. Commit: 7e0db0ae4f8fdd107c1f62056ca7f3628435ef14 https://github.com/boxbackup/boxbackup/commit/7e0db0ae4f8fdd107c1f62056ca7f3628435ef14 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupClientFileAttributes.cpp M lib/backupstore/BackupStoreFile.cpp M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/bbackupd/BackupDaemon.cpp M lib/bbackupquery/BackupQueries.cpp M lib/common/InvisibleTempFileStream.cpp M lib/common/NamedLock.cpp M lib/common/Test.cpp M lib/raidfile/RaidFileWrite.cpp M lib/server/Daemon.cpp M lib/server/ServerControl.cpp M lib/server/ServerStream.h M lib/win32/emu.h M test/backupstore/testbackupstore.cpp M test/backupstorefix/testbackupstorefix.cpp M test/backupstorepatch/testbackupstorepatch.cpp M test/basicserver/testbasicserver.cpp M test/bbackupd/testbbackupd.cpp M test/httpserver/testhttpserver.cpp M test/raidfile/testraidfile.cpp Log Message: ----------- Remove link and unlink macros on Windows. They provided source-code compatibility with the POSIX versions of these functions, but they conflict with Boost. Replace with EMU_LINK and EMU_UNLINK everywhere. Commit: a3065a4c98b6cad5e2e2880bef5992cdf9e31722 https://github.com/boxbackup/boxbackup/commit/a3065a4c98b6cad5e2e2880bef5992cdf9e31722 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/WaitForEvent.cpp M lib/common/WaitForEvent.h M lib/server/SocketListen.h M lib/server/SocketStream.cpp M lib/win32/emu.cpp M lib/win32/emu.h M lib/win32/emu_winver.h Log Message: ----------- Win32 build: upgrade to _WIN32_WINNT 0x600 (WinXP and above) This should enable us to use Windows dbghelp functions to get stack traces on exceptions, including demangled names. This required replacing our custom "struct pollfd" with EMU_STRUCT_POLLFD everywhere, to fix the conflict with newer versions of winsock2.h. Commit: f49f98c7e2d607d7219d2f381d8dbf6c7ed453cf https://github.com/boxbackup/boxbackup/commit/f49f98c7e2d607d7219d2f381d8dbf6c7ed453cf Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbackupquery/bbackupquery.cpp M infrastructure/buildenv-testmain-template.cpp A lib/common/MainHelper.cpp M lib/common/MainHelper.h M lib/server/Daemon.cpp Log Message: ----------- Refactor winsock initialisation on Windows Automatically initialise winsock in all binaries. Should fix issues with bbstoreaccounts not working on Windows. Commit: 068b9868ac11217e807cd2ef37f247fc8ebfe97a https://github.com/boxbackup/boxbackup/commit/068b9868ac11217e807cd2ef37f247fc8ebfe97a Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/DebugMemLeakFinder.cpp Log Message: ----------- Fix compile warnings about printf format strings in DebugMemLeakFinder Commit: 87bda4fb1083fa35dbb569c8da4e90a81626749c https://github.com/boxbackup/boxbackup/commit/87bda4fb1083fa35dbb569c8da4e90a81626749c Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Reduce precision of common timer test Ideally timers would be perfectly accurate and we could sleep for 1.0 seconds, but on OSX in particular they could fire 50-100 ms late (I've seen 4 ms in practice) and we don't want the tests to fail because of this, because we don't really need that kind of precision in practice. So we reduce the timer intervals by 100ms to be safe. (cherry picked from commit 8d02eebce553ed822e0fcd60d6e319384e15ba4b) Commit: 0286d7cae66fb312355bdc06fb9e854faf0484e4 https://github.com/boxbackup/boxbackup/commit/0286d7cae66fb312355bdc06fb9e854faf0484e4 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Logging.h M test/common/testcommon.cpp Log Message: ----------- Increase logging level and add timestamps in testcommon Should help to debug frequent test failures such as Condition [t2.HasExpired()] on OSX hosts on Travis. (cherry picked from commit 61a8e7514e01bc05ea3ea77d110c16bff37a50f4) Commit: 66a0f564b78ea6703521cd3dc756324c72331474 https://github.com/boxbackup/boxbackup/commit/66a0f564b78ea6703521cd3dc756324c72331474 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M .gitattributes Log Message: ----------- Stop Git from normalising line endings Although recommended, "text" mode causes constant and unfixable conficts that I give up fighting with. (cherry picked from commit 25e445a6848a870aabd89a3b8a013265aa3cf17b) Commit: c5d8173bf03ab9fa93d3b3ffd871b378b0819567 https://github.com/boxbackup/boxbackup/commit/c5d8173bf03ab9fa93d3b3ffd871b378b0819567 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M appveyor.yml M infrastructure/cmake/CMakeLists.txt M infrastructure/cmake/getversion.pl Log Message: ----------- Build Win32/64 binary packages automatically Every AppVeyor build on the specified branches that passes will be packaged into a ZIP file and installer using CPack and uploaded to GitHub Releases. Generate version number entirely in AppVeyor config, and build it into the binary, and use it in the names of the created binary packages, and compile it into the binaries. Commit: eb0f11fc51bb27c76ef0702bf14466c399e50d2c https://github.com/boxbackup/boxbackup/commit/eb0f11fc51bb27c76ef0702bf14466c399e50d2c Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbackupquery/bbackupquery.cpp M infrastructure/BoxPlatform.pm.in M lib/bbackupd/BackupDaemon.cpp M lib/bbstored/BackupStoreDaemon.cpp M lib/common/BannerText.h Log Message: ----------- Update banner text, and shorten compiled-in Git version numbers Shorten compiled-in version numbers based on Git revisions. Git revisions are often quoted as 7 hexadecimal digits, so let's do that here too. Add sanity to banner texts of command-line utilities. (cherry picked from commit 9afc3ca99d9cddd6f9a01fd5c0baa63ee16a449a) (cherry picked from commit c10913a35c155fb1efbc2c32ba0db45c580c4fad) (cherry picked from commit 0ce96b4c356b9f94b46e0c2842fdc81704c77f76) Commit: e7e52329a6569f8b02d0bfb62a6ea6c4a40b1a07 https://github.com/boxbackup/boxbackup/commit/e7e52329a6569f8b02d0bfb62a6ea6c4a40b1a07 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp Log Message: ----------- bbstoreaccounts: add support for standard command-line logging options Tidy up usage text, and fix erroneous memory leak reported after calling PrintUsageAndExit(). (cherry picked from commit 1a209b5c3c9cdd0ed3a92d2c884706a580b13844) Commit: 04d5327f16f616b9912d302fefb018377829c675 https://github.com/boxbackup/boxbackup/commit/04d5327f16f616b9912d302fefb018377829c675 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt Log Message: ----------- Make generated files depend on their generators under CMake If makeprotocol.pl.in or makeexception.pl changes, then we should rebuild the source files that are built by them (and therefore anything that depends on those source files, etc). (cherry picked from commit f9818932b31120f139500cc524563fe8483b072d) (cherry picked from commit 53d1a6c05ed4c39cb2eef34efb52a544ec4e02a1) Commit: 0b2f706ef23f576d46a8e183b1b5302e59833a84 https://github.com/boxbackup/boxbackup/commit/0b2f706ef23f576d46a8e183b1b5302e59833a84 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Whitespace fixes Commit: 019025deceff49485b03593f7a77a570d960546a https://github.com/boxbackup/boxbackup/commit/019025deceff49485b03593f7a77a570d960546a Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M appveyor.yml Log Message: ----------- Remove Xamarin to remove 500 lines of junk from build logs See http://help.appveyor.com/discussions/problems/4569-the-target-_convertpdbfiles-listed-in-a-beforetargets-attribute-at-c-does-not-exist-in-the-project-and-will-be-ignored for details. Commit: ea4f1728b529aff1dfc912d8f51567f8cd58ea77 https://github.com/boxbackup/boxbackup/commit/ea4f1728b529aff1dfc912d8f51567f8cd58ea77 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M appveyor.yml Log Message: ----------- Reorder lines in appveyor.yml to match actual execution order Commit: 4876a5573a8bcd3312e9804c27fe9f30f3369747 https://github.com/boxbackup/boxbackup/commit/4876a5573a8bcd3312e9804c27fe9f30f3369747 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M appveyor.yml M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Reduce AppVeyor build verbosity by passing extra args to sub-CMake Commit: 793d5f57f5f99f50bffadea674f58be79f9cb6ff https://github.com/boxbackup/boxbackup/commit/793d5f57f5f99f50bffadea674f58be79f9cb6ff Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M appveyor.yml Log Message: ----------- Disable DEBUG to reduce CMake build verbosity on AppVeyor Commit: 1ec75975e58cca35dc4ddc6e52583d0471c3e714 https://github.com/boxbackup/boxbackup/commit/1ec75975e58cca35dc4ddc6e52583d0471c3e714 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/backupstorefix/testbackupstorefix.cpp Log Message: ----------- Lock store while corrupting it in test/backupstorefix Should hopefully reduce risk of random test failures/race conditions. (cherry picked from commit 0f31359ca8ebbd6ddfd5f2d042c8c23af4d67997) Commit: 83c073852832deb513729dfecf4487dc0736509e https://github.com/boxbackup/boxbackup/commit/83c073852832deb513729dfecf4487dc0736509e Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt Log Message: ----------- Improve error messages when PCRE detection fails (or is abused) (cherry picked from commit 81528092fd51b6f81a21899bc205c369597c481d) Commit: 1c0d26fa62a6c9cc6e15146672df10667c4c0466 https://github.com/boxbackup/boxbackup/commit/1c0d26fa62a6c9cc6e15146672df10667c4c0466 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M lib/common/Box.h M lib/common/MainHelper.cpp M lib/common/MainHelper.h M lib/common/Utils.cpp Log Message: ----------- Win32: add experimental stack backtraces with symbol lookup (cherry picked from commit f4685e72c2c7e274533c1cb149180809631a1da7) Commit: 8198631273bfaaa7a03c857fd4fd2a3d5723ae00 https://github.com/boxbackup/boxbackup/commit/8198631273bfaaa7a03c857fd4fd2a3d5723ae00 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Logging.cpp M lib/common/Logging.h Log Message: ----------- Modify command-line -L option to override log level for a specific file This allows enabling TRACE logging for a specific file (e.g. to enable exception backtraces for that file) without enabling TRACE globally. Replace HideFileGuard with LogLevelOverrideByFileGuard. (cherry picked from commit 29f74be06351b6ce9a872765db63171ad902b596) Commit: f0dffbd0e4414583c20c109ba9796a3191b69cfa https://github.com/boxbackup/boxbackup/commit/f0dffbd0e4414583c20c109ba9796a3191b69cfa Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Box.h M lib/common/Logging.cpp M lib/common/Logging.h M lib/common/Utils.cpp M lib/common/Utils.h M lib/server/SocketStream.cpp Log Message: ----------- Use filename where exception was thrown in DumpStackBacktrace Allows us to turn on exceptions for specific files, rather that filtering them all as Utils.cpp. (cherry picked from commit 3590e9745dac2f25267bd755ed3d554423395330) Commit: 8b27144d1c42d55108d84f4ec3e51914818b5d84 https://github.com/boxbackup/boxbackup/commit/8b27144d1c42d55108d84f4ec3e51914818b5d84 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M runtest.pl.in Log Message: ----------- Enable backtraces and timestamps in all tests when building with CMake (cherry picked from commit 166e39c11d242ebed6b62fc3f126af48345a22ca) Commit: 975422bebd16ecd144dbafc2ef89b6481a269a7b https://github.com/boxbackup/boxbackup/commit/975422bebd16ecd144dbafc2ef89b6481a269a7b Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.h Log Message: ----------- Use the correct preprocessor define for release builds (cherry picked from commit 085149bf1c2c287ee744f2e094a99a77fc89dc60) Commit: cacfa120b266fbdb97ddd3f0326f4ef31c32a5a6 https://github.com/boxbackup/boxbackup/commit/cacfa120b266fbdb97ddd3f0326f4ef31c32a5a6 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/server/SocketStream.cpp Log Message: ----------- Show IP address as well as hostname for ServerException::SocketOpenError (cherry picked from commit 1a51adcd6be000f09fa280e42fb12e5631cd69b9) Commit: 008a3f9fcb88da5b15b30a566fdc75ac9e95fd1f https://github.com/boxbackup/boxbackup/commit/008a3f9fcb88da5b15b30a566fdc75ac9e95fd1f Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/backupstorefix/testbackupstorefix.cpp Log Message: ----------- test/backupstorefix: tag bbstoreaccounts check output in tests (cherry picked from commit c31d7143273f2d5257ec690e894442a8398aaf21) Commit: cb4491559cbce0e20a54ffba3b4a2b0b207d1cad https://github.com/boxbackup/boxbackup/commit/cb4491559cbce0e20a54ffba3b4a2b0b207d1cad Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Add support for M4 CXX flag detection in CMakeLists Enables all flags that are detected by M4 builds to be detected by CMake builds as well (except for -Wall which enables a bunch of junk). (cherry picked from commit 2ea8a9706d79a386729c7b2d814089e90bac6926) Commit: bf60b220e581f9c13406654c0947e2cbc1c2b49e https://github.com/boxbackup/boxbackup/commit/bf60b220e581f9c13406654c0947e2cbc1c2b49e Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/DebugMemLeakFinder.cpp M lib/common/Utils.cpp Log Message: ----------- Store and log full stack traces for leaked memory allocations Don't dump stack backtrace when logging memory leaks to a file Since the backtrace goes to the console instead, it's confusing to have them in different places. (cherry picked from commit 6e7df4dfdc06eaff2dd0ef51741df3cf6ba319ea) (cherry picked from commit 270cd1f16fa8a32b07ae0b9762a417db7b95b82f) (cherry picked from commit c627e9547fb8a191ea36780b140fd0516a0bffa8) Commit: 17caf65075d3b8ae86ef79b167642b50bde62959 https://github.com/boxbackup/boxbackup/commit/17caf65075d3b8ae86ef79b167642b50bde62959 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/buildenv-testmain-template.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/common/Test.h M test/backupstorefix/testbackupstorefix.cpp M test/bbackupd/testbbackupd.cpp Log Message: ----------- Increase default verbosity in some timing tests Commit: 4c6b6c6cfef8d801d308ee87b6eb01d2f23f5468 https://github.com/boxbackup/boxbackup/commit/4c6b6c6cfef8d801d308ee87b6eb01d2f23f5468 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in Log Message: ----------- Detect which debugger to use in t-gdb scripts: lldb or gdb Should make debugging on Mac and BSD less painful. Make t-gdb fail with an error when no debugger was detected. (cherry picked from commit da98a8183737fb1fcb658cebd9612791a4750c4c) (cherry picked from commit ea6bc722131222d1fad08521e61f552f16941b01) Commit: d45c885f8d93aae26e625a2a2f3125275dbc3f6a https://github.com/boxbackup/boxbackup/commit/d45c885f8d93aae26e625a2a2f3125275dbc3f6a Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Don't add CXXFLAGS to CFLAGS Some CXXFLAGS such as enabling C++11 don't make sense for CC and will be rejected, causing the compile to fail. We don't really test for flags being compatible with the C compiler anyway, and we don't have enough C files to care that much. (cherry picked from commit 2ad9ee9012d4a7354c33ff0272c0768b952a9064) Commit: 5241e85df0999166f7afed1f978e944bb2be1a0d https://github.com/boxbackup/boxbackup/commit/5241e85df0999166f7afed1f978e944bb2be1a0d Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Perl buildsystem: add test prepare targets Similar to the CMake prepare targets, these are useful when intending to debug tests or run a server using a standard test configuration, which requires the testfiles directory to be copied into the binary output tree. (cherry picked from commit 2afecdd57a3bcf42c116c8535843039a1596feac) Commit: 237a0ccde4d2f20ae838223687e739f81a05b63d https://github.com/boxbackup/boxbackup/commit/237a0ccde4d2f20ae838223687e739f81a05b63d Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Rename BackupStoreAccountsControl to BackupStoreAccountControl For consistency with the base class, BackupAccountControl, and grammatical reasons. Commit: d3f373b89748ef9bcd73748e2a09c0a5608ba992 https://github.com/boxbackup/boxbackup/commit/d3f373b89748ef9bcd73748e2a09c0a5608ba992 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Add missing implementations of RaidBackupFileSystem methods These should have been added whenever BackupFileSystem.h was added, which was 4db4deed3448b9c2b9e885a20cb1a777632464d6, although it didn't really belong in that commit either. But it's too late to change history now. Adding the missing method definitions makes it possible to instantiate a RaidBackupFileSystem. Commit: fc0491921512d5039d9c4df1017accc0e4db963c https://github.com/boxbackup/boxbackup/commit/fc0491921512d5039d9c4df1017accc0e4db963c Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Refactor BackupStoreAccountControl to be for a specific account ID This allows other BackupAccountControl implementations to implement the same interface without needing to support account IDs, which are very store-specific. Commit: 2642cd268b0b14cc93a23ec00bfa9705f91890c8 https://github.com/boxbackup/boxbackup/commit/2642cd268b0b14cc93a23ec00bfa9705f91890c8 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/backupstore/testbackupstore.cpp Log Message: ----------- Remove debugging code to list files during test_multiple_uploads Revert "Try listing files to help debug random failures of test_multiple_uploads()." This reverts commit e302ded5b7d40fe88d65808564f4db9df2a6f878. Commit: a28369d30fe2c464ceb0ef14a793d62386151fa6 https://github.com/boxbackup/boxbackup/commit/a28369d30fe2c464ceb0ef14a793d62386151fa6 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in M infrastructure/travis-build.sh M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupConstants.h M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupProtocol.txt M lib/backupstore/BackupStoreAccounts.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreFile.cpp M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/HousekeepStoreAccount.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/bbstored/BackupStoreDaemon.cpp M lib/common/CollectInBufferStream.cpp M lib/common/CommonException.txt M lib/common/NamedLock.cpp M lib/httpserver/S3Simulator.cpp M runtest.pl.in M test/backupstore/testbackupstore.cpp M test/backupstorefix/testbackupstorefix.cpp M test/backupstorepatch/testbackupstorepatch.cpp M test/bbackupd/testbbackupd.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace and comment cleanup Commit: 2ea3a376b0091dcc692bee8977569cf662b0bf80 https://github.com/boxbackup/boxbackup/commit/2ea3a376b0091dcc692bee8977569cf662b0bf80 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. Commit: e1d05ef068f77b92afcab0eb1af3c5cff5c03fab https://github.com/boxbackup/boxbackup/commit/e1d05ef068f77b92afcab0eb1af3c5cff5c03fab Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum Commit: f54861f60f50b924bfb878e1e7d73e57c665976b https://github.com/boxbackup/boxbackup/commit/f54861f60f50b924bfb878e1e7d73e57c665976b Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Add GetTempDirPath() for portability On Unix, check TEMP, TMP and TMPDIR, and if no env var is set, fall back to /tmp. For some reason it's becoming increasingly common for Unixes not to set TMP* to anything. Commit: c861174a9270789126f57f12044a1b34ec186196 https://github.com/boxbackup/boxbackup/commit/c861174a9270789126f57f12044a1b34ec186196 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Fix include ordering in testcommon.cpp Commit: e060804a0e1cf62efc7aa2d16948b19dbc1d7338 https://github.com/boxbackup/boxbackup/commit/e060804a0e1cf62efc7aa2d16948b19dbc1d7338 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Refactor testcommon to split into independent tests Remove redundant NamedLock tests. Commit: d309f0ec96925b1b1fa52efc4b60828a671272bb https://github.com/boxbackup/boxbackup/commit/d309f0ec96925b1b1fa52efc4b60828a671272bb Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h M test/common/testcommon.cpp Log Message: ----------- Add force parameter to RemovePrefix and RemoveSuffix Commit: 092d70cd92846060df442f156571e6c54290f479 https://github.com/boxbackup/boxbackup/commit/092d70cd92846060df442f156571e6c54290f479 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Remove BackupStoreContext::ReceivedFinishCommand Replace with CleanUp(). We don't need two methods that do almost the same thing. Commit: 990852ada835b2d628ecf0c7ad0c40397a5abcbf https://github.com/boxbackup/boxbackup/commit/990852ada835b2d628ecf0c7ad0c40397a5abcbf Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreContext::GetBackupStoreInfo Almost all access to mapStoreInfo is now via GetBackupStoreInfo(Internal), in preparation for the refactor that will move the main copy of the BackupStoreInfo to the BackupFileSystem instance instead. Commit: c2df6a1ec64be15cc72a8e61454b111b53c2fcc9 https://github.com/boxbackup/boxbackup/commit/c2df6a1ec64be15cc72a8e61454b111b53c2fcc9 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupStoreContext.cpp Log Message: ----------- Enable C++0x support We need C++0x features enabled for "auto" to work, and using Boost properly seems to require this. Use "auto" in a few places to prove that it works. Commit: cbe6357332c42f2b3b13af6356b1997a4c0b3698 https://github.com/boxbackup/boxbackup/commit/cbe6357332c42f2b3b13af6356b1997a4c0b3698 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor to create BackupFileSystem::GetBackupStoreInfo Extract common code from filesystem-specific implementations, and rename them to GetBackupStoreInfoInternal(). Commit: f266d223e50e2733349f5d3df93bbb8221c4935e https://github.com/boxbackup/boxbackup/commit/f266d223e50e2733349f5d3df93bbb8221c4935e Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor BackupFileSystem::GetLock() Call subclass implementations of TryGetLock(), and throw an exception if we fail to get a lock instead of returning false. Commit: 6e5e203e979b4f477c16dbf9def72f533cc402c0 https://github.com/boxbackup/boxbackup/commit/6e5e203e979b4f477c16dbf9def72f533cc402c0 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Refactor test code into process_is_running() We need the ability to check whether a process is still running to check for stale S3 account locks in SimpleDB, so move this code out of Test.cpp and into Utils.cpp, for use outside of tests. Commit: 734ce0e22d678caccf2bbcc25261b10e5339f3a7 https://github.com/boxbackup/boxbackup/commit/734ce0e22d678caccf2bbcc25261b10e5339f3a7 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Test.h M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Implement compare_lists and test_equal_maps in lib/common/Test.cpp Commit: 85d9049b426250456877a91b394c8dd04896f6f6 https://github.com/boxbackup/boxbackup/commit/85d9049b426250456877a91b394c8dd04896f6f6 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/HTTPQueryDecoder.cpp M lib/httpserver/HTTPQueryDecoder.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add static method HTTPQueryDecoder::URLEncode This will be needed for request parameter encoding and for SimpleDB. Commit: 0d24c7b8efda1e729278cb53e3c724072d9d4651 https://github.com/boxbackup/boxbackup/commit/0d24c7b8efda1e729278cb53e3c724072d9d4651 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add query parameter editing and encoding to HTTPRequest Commit: 85b33006910933310347a2c64425f7bfac37b20b https://github.com/boxbackup/boxbackup/commit/85b33006910933310347a2c64425f7bfac37b20b Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for virtualhost names in S3Client This is a precondition of support for both S3 and SimpleDB in the S3Simulator, since otherwise we can't tell the requests apart. Commit: 3710174592026f2fbde07baaeae38cf8b73b73ec https://github.com/boxbackup/boxbackup/commit/3710174592026f2fbde07baaeae38cf8b73b73ec Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 8a3636712e4f132d642069b563ab849a790b3d82 https://github.com/boxbackup/boxbackup/commit/8a3636712e4f132d642069b563ab849a790b3d82 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: dcc5fac199949a5d5c60ddfea3a7ff3974020272 https://github.com/boxbackup/boxbackup/commit/dcc5fac199949a5d5c60ddfea3a7ff3974020272 Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: c9939f3b7c0ff96c85e67fc990b4a6b4ac7f808a https://github.com/boxbackup/boxbackup/commit/c9939f3b7c0ff96c85e67fc990b4a6b4ac7f808a Author: Chris Wilson Date: 2017-08-02 (Wed, 02 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Compare: https://github.com/boxbackup/boxbackup/compare/18f9e1646e39...c9939f3b7c0f From noreply at github.com Wed Aug 2 21:44:10 2017 From: noreply at github.com (GitHub) Date: Wed, 02 Aug 2017 13:44:10 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5982399ac4cb8_7e43fd9f7873c30179d3@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170802.c9939f3 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Thu Aug 3 20:59:56 2017 From: noreply at github.com (GitHub) Date: Thu, 03 Aug 2017 12:59:56 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 1e8dec: Make GetBlockSize() use BackupFileSystem Message-ID: <598380bcd0947_34b3fe91659fc38505b@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 1e8dec3f3d2932e2d48fa872d695a46d56d1e87a https://github.com/boxbackup/boxbackup/commit/1e8dec3f3d2932e2d48fa872d695a46d56d1e87a Author: Chris Wilson Date: 2017-08-03 (Thu, 03 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. From noreply at github.com Thu Aug 3 21:21:24 2017 From: noreply at github.com (GitHub) Date: Thu, 03 Aug 2017 13:21:24 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 30650b: Protocol generator: record last error message rece... Message-ID: <598385c46110e_d793f7fc556dc347871f@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 30650b1c26037806753b5008fa03d6d84dc54620 https://github.com/boxbackup/boxbackup/commit/30650b1c26037806753b5008fa03d6d84dc54620 Author: Chris Wilson Date: 2017-08-03 (Thu, 03 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. From noreply at github.com Thu Aug 3 21:24:26 2017 From: noreply at github.com (GitHub) Date: Thu, 03 Aug 2017 13:24:26 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5983867a8efbc_73a03f7e4b0c9c44851d3@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170803.1e8dec3 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Thu Aug 3 21:48:47 2017 From: noreply at github.com (GitHub) Date: Thu, 03 Aug 2017 13:48:47 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59838c2fde15c_44cd3ff319cb5c2c8608d@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170803.30650b1 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Fri Aug 4 20:02:36 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 12:02:36 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 026b64: Custom build system: fix autogen dependencies Message-ID: <5984c4cc34888_42df3fbb1f7e5c38442b3@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 026b6456100f59528daee52c86ceaec6b176641d https://github.com/boxbackup/boxbackup/commit/026b6456100f59528daee52c86ceaec6b176641d Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. Commit: 459345a7bc210f04f1f89da13be8449ca1b77ca6 https://github.com/boxbackup/boxbackup/commit/459345a7bc210f04f1f89da13be8449ca1b77ca6 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum Commit: b7c3db508fbba143b46e0bb03ac370650a87cbb0 https://github.com/boxbackup/boxbackup/commit/b7c3db508fbba143b46e0bb03ac370650a87cbb0 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Add GetTempDirPath() for portability On Unix, check TEMP, TMP and TMPDIR, and if no env var is set, fall back to /tmp. For some reason it's becoming increasingly common for Unixes not to set TMP* to anything. Commit: 9f5163624c623ec4a7961299cff17893f645fa9c https://github.com/boxbackup/boxbackup/commit/9f5163624c623ec4a7961299cff17893f645fa9c Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Fix include ordering in testcommon.cpp Commit: 1247085900c9f4392110c9f99ab88759bdd86243 https://github.com/boxbackup/boxbackup/commit/1247085900c9f4392110c9f99ab88759bdd86243 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Refactor testcommon to split into independent tests Remove redundant NamedLock tests. Commit: e59336b5e4757a3fbc5ad5cdc5b165272b14624a https://github.com/boxbackup/boxbackup/commit/e59336b5e4757a3fbc5ad5cdc5b165272b14624a Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h M test/common/testcommon.cpp Log Message: ----------- Add force parameter to RemovePrefix and RemoveSuffix Commit: ea8bb140c99da731e06290ffdcf12f7a0aaf111a https://github.com/boxbackup/boxbackup/commit/ea8bb140c99da731e06290ffdcf12f7a0aaf111a Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Remove BackupStoreContext::ReceivedFinishCommand Replace with CleanUp(). We don't need two methods that do almost the same thing. Commit: e2c33c4457e290bc043ac135cd7e0c9227a85387 https://github.com/boxbackup/boxbackup/commit/e2c33c4457e290bc043ac135cd7e0c9227a85387 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreContext::GetBackupStoreInfo Almost all access to mapStoreInfo is now via GetBackupStoreInfo(Internal), in preparation for the refactor that will move the main copy of the BackupStoreInfo to the BackupFileSystem instance instead. Commit: 352f3d2c7f479e990c45dbb8c85905a634d28f6e https://github.com/boxbackup/boxbackup/commit/352f3d2c7f479e990c45dbb8c85905a634d28f6e Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupStoreContext.cpp Log Message: ----------- Enable C++0x support We need C++0x features enabled for "auto" to work, and using Boost properly seems to require this. Use "auto" in a few places to prove that it works. Commit: d2f7cff5ca0797823f212800e8886e7c808b527e https://github.com/boxbackup/boxbackup/commit/d2f7cff5ca0797823f212800e8886e7c808b527e Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor to create BackupFileSystem::GetBackupStoreInfo Extract common code from filesystem-specific implementations, and rename them to GetBackupStoreInfoInternal(). Commit: f5352e33065e92899afaffa574c1344ca8cb0d06 https://github.com/boxbackup/boxbackup/commit/f5352e33065e92899afaffa574c1344ca8cb0d06 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor BackupFileSystem::GetLock() Call subclass implementations of TryGetLock(), and throw an exception if we fail to get a lock instead of returning false. Commit: d10fbd8420384253a04b8e3babb50212b273286b https://github.com/boxbackup/boxbackup/commit/d10fbd8420384253a04b8e3babb50212b273286b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Refactor test code into process_is_running() We need the ability to check whether a process is still running to check for stale S3 account locks in SimpleDB, so move this code out of Test.cpp and into Utils.cpp, for use outside of tests. Commit: 407c594f87ed8bf26efaa1fd5fc9f4c3f311b1cf https://github.com/boxbackup/boxbackup/commit/407c594f87ed8bf26efaa1fd5fc9f4c3f311b1cf Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Test.h M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Implement compare_lists and test_equal_maps in lib/common/Test.cpp Commit: 0a02e2b338ec67510d773758169165ce997ff125 https://github.com/boxbackup/boxbackup/commit/0a02e2b338ec67510d773758169165ce997ff125 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPQueryDecoder.cpp M lib/httpserver/HTTPQueryDecoder.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add static method HTTPQueryDecoder::URLEncode This will be needed for request parameter encoding and for SimpleDB. Commit: f629990cf485bdb8d24fa94b534ad91134eeddc2 https://github.com/boxbackup/boxbackup/commit/f629990cf485bdb8d24fa94b534ad91134eeddc2 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add query parameter editing and encoding to HTTPRequest Commit: a8522430511d1883b4a3d16e650ba8988f19222d https://github.com/boxbackup/boxbackup/commit/a8522430511d1883b4a3d16e650ba8988f19222d Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for virtualhost names in S3Client This is a precondition of support for both S3 and SimpleDB in the S3Simulator, since otherwise we can't tell the requests apart. Commit: 832bc288e4341fd29bce245687b42411c262ea06 https://github.com/boxbackup/boxbackup/commit/832bc288e4341fd29bce245687b42411c262ea06 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 2c48caaedec53c66d7afc4623487521b439df658 https://github.com/boxbackup/boxbackup/commit/2c48caaedec53c66d7afc4623487521b439df658 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: cf369da1e2eebd4159a19bbdd61807037320c954 https://github.com/boxbackup/boxbackup/commit/cf369da1e2eebd4159a19bbdd61807037320c954 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: f1b4e632fed52c02ab92390343ac48b9fa7fb70c https://github.com/boxbackup/boxbackup/commit/f1b4e632fed52c02ab92390343ac48b9fa7fb70c Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: b3d782ebdc78cd4594ff0edd0b42768f729146c2 https://github.com/boxbackup/boxbackup/commit/b3d782ebdc78cd4594ff0edd0b42768f729146c2 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: 2f0f1d60c305b4cb2ed8f273cd159473760a501a https://github.com/boxbackup/boxbackup/commit/2f0f1d60c305b4cb2ed8f273cd159473760a501a Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Compare: https://github.com/boxbackup/boxbackup/compare/30650b1c2603...2f0f1d60c305 From noreply at github.com Fri Aug 4 20:26:54 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 12:26:54 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5984ca7ee381e_cc73ff2fada3c341095a6@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170804.2f0f1d6 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Fri Aug 4 20:37:50 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 12:37:50 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] fb598f: Increase default verbosity in some timing tests Message-ID: <5984cd0eb5977_7e1e3faad9b5fc34955f6@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: fb598faab5318586eee15bbd9accfb5a3a7278e3 https://github.com/boxbackup/boxbackup/commit/fb598faab5318586eee15bbd9accfb5a3a7278e3 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/buildenv-testmain-template.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/common/Test.h M test/backupstorefix/testbackupstorefix.cpp M test/bbackupd/testbbackupd.cpp Log Message: ----------- Increase default verbosity in some timing tests Commit: 1418657c5b974189cc5eabb737f590e00d8b22da https://github.com/boxbackup/boxbackup/commit/1418657c5b974189cc5eabb737f590e00d8b22da Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in Log Message: ----------- Detect which debugger to use in t-gdb scripts: lldb or gdb Should make debugging on Mac and BSD less painful. Make t-gdb fail with an error when no debugger was detected. (cherry picked from commit da98a8183737fb1fcb658cebd9612791a4750c4c) (cherry picked from commit ea6bc722131222d1fad08521e61f552f16941b01) Commit: eed9a5fcfd68e5cef4546ef734dfae1ab0648040 https://github.com/boxbackup/boxbackup/commit/eed9a5fcfd68e5cef4546ef734dfae1ab0648040 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Don't add CXXFLAGS to CFLAGS Some CXXFLAGS such as enabling C++11 don't make sense for CC and will be rejected, causing the compile to fail. We don't really test for flags being compatible with the C compiler anyway, and we don't have enough C files to care that much. (cherry picked from commit 2ad9ee9012d4a7354c33ff0272c0768b952a9064) Commit: 2c3254cd82cbf8eccfc6ba1152ab0a8cf63e1bbc https://github.com/boxbackup/boxbackup/commit/2c3254cd82cbf8eccfc6ba1152ab0a8cf63e1bbc Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Perl buildsystem: add test prepare targets Similar to the CMake prepare targets, these are useful when intending to debug tests or run a server using a standard test configuration, which requires the testfiles directory to be copied into the binary output tree. (cherry picked from commit 2afecdd57a3bcf42c116c8535843039a1596feac) Commit: c6179723c85755da9daf0dd8f03062a7d9a080fe https://github.com/boxbackup/boxbackup/commit/c6179723c85755da9daf0dd8f03062a7d9a080fe Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Rename BackupStoreAccountsControl to BackupStoreAccountControl For consistency with the base class, BackupAccountControl, and grammatical reasons. Commit: cbc39fc1081cdfbb208ebefa04ed966b674db82b https://github.com/boxbackup/boxbackup/commit/cbc39fc1081cdfbb208ebefa04ed966b674db82b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Add missing implementations of RaidBackupFileSystem methods These should have been added whenever BackupFileSystem.h was added, which was 4db4deed3448b9c2b9e885a20cb1a777632464d6, although it didn't really belong in that commit either. But it's too late to change history now. Adding the missing method definitions makes it possible to instantiate a RaidBackupFileSystem. Commit: 11137936f178604733e0ed4157105076bf8285c0 https://github.com/boxbackup/boxbackup/commit/11137936f178604733e0ed4157105076bf8285c0 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Refactor BackupStoreAccountControl to be for a specific account ID This allows other BackupAccountControl implementations to implement the same interface without needing to support account IDs, which are very store-specific. Commit: 12aa37fb07a92105c5a0234500aebc50cc9ce293 https://github.com/boxbackup/boxbackup/commit/12aa37fb07a92105c5a0234500aebc50cc9ce293 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/backupstore/testbackupstore.cpp Log Message: ----------- Remove debugging code to list files during test_multiple_uploads Revert "Try listing files to help debug random failures of test_multiple_uploads()." This reverts commit e302ded5b7d40fe88d65808564f4db9df2a6f878. Commit: 10d406a0ab96ad85bced36bc64104b0d4723e9eb https://github.com/boxbackup/boxbackup/commit/10d406a0ab96ad85bced36bc64104b0d4723e9eb Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in M infrastructure/travis-build.sh M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupConstants.h M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupProtocol.txt M lib/backupstore/BackupStoreAccounts.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreFile.cpp M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/HousekeepStoreAccount.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/bbstored/BackupStoreDaemon.cpp M lib/common/CollectInBufferStream.cpp M lib/common/CommonException.txt M lib/common/NamedLock.cpp M lib/httpserver/S3Simulator.cpp M runtest.pl.in M test/backupstore/testbackupstore.cpp M test/backupstorefix/testbackupstorefix.cpp M test/backupstorepatch/testbackupstorepatch.cpp M test/bbackupd/testbbackupd.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace and comment cleanup Commit: feedab815e2fa093129486c78bd5bd19bc620779 https://github.com/boxbackup/boxbackup/commit/feedab815e2fa093129486c78bd5bd19bc620779 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. Commit: b2a333867404500c3253fe1a14fe0867388ce128 https://github.com/boxbackup/boxbackup/commit/b2a333867404500c3253fe1a14fe0867388ce128 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum Commit: 929b84c023b99ebca220bfad384199070f10d3ff https://github.com/boxbackup/boxbackup/commit/929b84c023b99ebca220bfad384199070f10d3ff Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Add GetTempDirPath() for portability On Unix, check TEMP, TMP and TMPDIR, and if no env var is set, fall back to /tmp. For some reason it's becoming increasingly common for Unixes not to set TMP* to anything. Commit: 266991212b6b9ee9f93a508feb74978c2d4d091b https://github.com/boxbackup/boxbackup/commit/266991212b6b9ee9f93a508feb74978c2d4d091b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Fix include ordering in testcommon.cpp Commit: 720cd08242c43a2cd0bb027e115797b995615dab https://github.com/boxbackup/boxbackup/commit/720cd08242c43a2cd0bb027e115797b995615dab Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Refactor testcommon to split into independent tests Remove redundant NamedLock tests. Commit: f56d57a8c141f18d2b3dc322ba938d59e067023a https://github.com/boxbackup/boxbackup/commit/f56d57a8c141f18d2b3dc322ba938d59e067023a Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h M test/common/testcommon.cpp Log Message: ----------- Add force parameter to RemovePrefix and RemoveSuffix Commit: 9e10bdf4e0177a90bcf514aa33961d3076677c82 https://github.com/boxbackup/boxbackup/commit/9e10bdf4e0177a90bcf514aa33961d3076677c82 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Remove BackupStoreContext::ReceivedFinishCommand Replace with CleanUp(). We don't need two methods that do almost the same thing. Commit: 04b03a3b7576002ec02e27624bcf6e367bb792fb https://github.com/boxbackup/boxbackup/commit/04b03a3b7576002ec02e27624bcf6e367bb792fb Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreContext::GetBackupStoreInfo Almost all access to mapStoreInfo is now via GetBackupStoreInfo(Internal), in preparation for the refactor that will move the main copy of the BackupStoreInfo to the BackupFileSystem instance instead. Commit: 16441b5b4cf58ff2d58d278af87a0428f0771321 https://github.com/boxbackup/boxbackup/commit/16441b5b4cf58ff2d58d278af87a0428f0771321 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupStoreContext.cpp Log Message: ----------- Enable C++0x support We need C++0x features enabled for "auto" to work, and using Boost properly seems to require this. Use "auto" in a few places to prove that it works. Commit: e51b4f3a6debd16855267fbd748f87c56fb6bae6 https://github.com/boxbackup/boxbackup/commit/e51b4f3a6debd16855267fbd748f87c56fb6bae6 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor to create BackupFileSystem::GetBackupStoreInfo Extract common code from filesystem-specific implementations, and rename them to GetBackupStoreInfoInternal(). Commit: 1abaa9461a970460e176815056e60af381e564f8 https://github.com/boxbackup/boxbackup/commit/1abaa9461a970460e176815056e60af381e564f8 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor BackupFileSystem::GetLock() Call subclass implementations of TryGetLock(), and throw an exception if we fail to get a lock instead of returning false. Commit: 456f7fd144ebf29616e98c86a40d3e850566ba1e https://github.com/boxbackup/boxbackup/commit/456f7fd144ebf29616e98c86a40d3e850566ba1e Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Refactor test code into process_is_running() We need the ability to check whether a process is still running to check for stale S3 account locks in SimpleDB, so move this code out of Test.cpp and into Utils.cpp, for use outside of tests. Commit: 63ff1bb988081a800cc6fe498fb8cb990fa6681b https://github.com/boxbackup/boxbackup/commit/63ff1bb988081a800cc6fe498fb8cb990fa6681b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Test.h M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Implement compare_lists and test_equal_maps in lib/common/Test.cpp Commit: e595964133c7d09ee27df97d855b14eb969058ed https://github.com/boxbackup/boxbackup/commit/e595964133c7d09ee27df97d855b14eb969058ed Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPQueryDecoder.cpp M lib/httpserver/HTTPQueryDecoder.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add static method HTTPQueryDecoder::URLEncode This will be needed for request parameter encoding and for SimpleDB. Commit: 6ceb9d68cfb29b0314ef42a0fa0e88bf3fed00d2 https://github.com/boxbackup/boxbackup/commit/6ceb9d68cfb29b0314ef42a0fa0e88bf3fed00d2 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add query parameter editing and encoding to HTTPRequest Commit: 89d136d65e830eb363770c567ace21d251566164 https://github.com/boxbackup/boxbackup/commit/89d136d65e830eb363770c567ace21d251566164 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for virtualhost names in S3Client This is a precondition of support for both S3 and SimpleDB in the S3Simulator, since otherwise we can't tell the requests apart. Commit: 9f96bd69fcbe8740ed6a611cd625d928145432ef https://github.com/boxbackup/boxbackup/commit/9f96bd69fcbe8740ed6a611cd625d928145432ef Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: dd966cdd75f6497abe43665a3af6cc107327a686 https://github.com/boxbackup/boxbackup/commit/dd966cdd75f6497abe43665a3af6cc107327a686 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: c0552484babfd95bbb691d611be9203ec7f5c414 https://github.com/boxbackup/boxbackup/commit/c0552484babfd95bbb691d611be9203ec7f5c414 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: 27a65db476d900a863afa96b5a67e18334586992 https://github.com/boxbackup/boxbackup/commit/27a65db476d900a863afa96b5a67e18334586992 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: efcb9ad962a82ceb5631685bf0d2cb701c80bcd4 https://github.com/boxbackup/boxbackup/commit/efcb9ad962a82ceb5631685bf0d2cb701c80bcd4 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: 01e238b7ac10406ca215f957d4f280a09eb80c04 https://github.com/boxbackup/boxbackup/commit/01e238b7ac10406ca215f957d4f280a09eb80c04 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Compare: https://github.com/boxbackup/boxbackup/compare/2f0f1d60c305...01e238b7ac10 From noreply at github.com Fri Aug 4 20:47:27 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 12:47:27 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 4285f2: Improve error message on hostname lookup failure Message-ID: <5984cf4f2f039_7ac03fdf5f0bbc3c136411@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 4285f2560793dcb388c051cf72854bf8f3e2027d https://github.com/boxbackup/boxbackup/commit/4285f2560793dcb388c051cf72854bf8f3e2027d Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. From noreply at github.com Fri Aug 4 21:08:27 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 13:08:27 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 637605: Make narrowing conversions an error Message-ID: <5984d43b46015_6b9c3fe6a2323c281196f6@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 637605eb189e95949c7ef81a30f24daa3365b837 https://github.com/boxbackup/boxbackup/commit/637605eb189e95949c7ef81a30f24daa3365b837 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: b51e3b94f850b1e76dbabf94962b2d4a8a1a9ae7 https://github.com/boxbackup/boxbackup/commit/b51e3b94f850b1e76dbabf94962b2d4a8a1a9ae7 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat This was broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Compare: https://github.com/boxbackup/boxbackup/compare/4285f2560793...b51e3b94f850 From noreply at github.com Fri Aug 4 21:39:34 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 13:39:34 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5984db86a0cdd_116e3fb5fec31c401123f0@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170804.b51e3b9 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Fri Aug 4 21:44:19 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 13:44:19 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 24cea5: Increase default verbosity in some timing tests Message-ID: <5984dca31ba89_1920b3f7e56249c30153317@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 24cea5d8bff79da844aa0d05dcf79047a93b6aef https://github.com/boxbackup/boxbackup/commit/24cea5d8bff79da844aa0d05dcf79047a93b6aef Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/buildenv-testmain-template.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/common/Test.h M test/backupstorefix/testbackupstorefix.cpp M test/bbackupd/testbbackupd.cpp Log Message: ----------- Increase default verbosity in some timing tests Commit: 172a9cf24646e43455e2432bee8bfee303bf8992 https://github.com/boxbackup/boxbackup/commit/172a9cf24646e43455e2432bee8bfee303bf8992 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in Log Message: ----------- Detect which debugger to use in t-gdb scripts: lldb or gdb Should make debugging on Mac and BSD less painful. Make t-gdb fail with an error when no debugger was detected. (cherry picked from commit da98a8183737fb1fcb658cebd9612791a4750c4c) (cherry picked from commit ea6bc722131222d1fad08521e61f552f16941b01) Commit: accdbb8526138e05aec7dc77e2e7c509b5ba4050 https://github.com/boxbackup/boxbackup/commit/accdbb8526138e05aec7dc77e2e7c509b5ba4050 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Don't add CXXFLAGS to CFLAGS Some CXXFLAGS such as enabling C++11 don't make sense for CC and will be rejected, causing the compile to fail. We don't really test for flags being compatible with the C compiler anyway, and we don't have enough C files to care that much. (cherry picked from commit 2ad9ee9012d4a7354c33ff0272c0768b952a9064) Commit: 88ee5668bde131599129a16dc3de31cdde5833ce https://github.com/boxbackup/boxbackup/commit/88ee5668bde131599129a16dc3de31cdde5833ce Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Perl buildsystem: add test prepare targets Similar to the CMake prepare targets, these are useful when intending to debug tests or run a server using a standard test configuration, which requires the testfiles directory to be copied into the binary output tree. (cherry picked from commit 2afecdd57a3bcf42c116c8535843039a1596feac) Commit: 1a1141d26fd5a206b5255bc8b9c169ac49dec7cb https://github.com/boxbackup/boxbackup/commit/1a1141d26fd5a206b5255bc8b9c169ac49dec7cb Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Rename BackupStoreAccountsControl to BackupStoreAccountControl For consistency with the base class, BackupAccountControl, and grammatical reasons. Commit: fe52dc30df3a4e2017b03fbf408a1d91df93eb78 https://github.com/boxbackup/boxbackup/commit/fe52dc30df3a4e2017b03fbf408a1d91df93eb78 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Add missing implementations of RaidBackupFileSystem methods These should have been added whenever BackupFileSystem.h was added, which was 4db4deed3448b9c2b9e885a20cb1a777632464d6, although it didn't really belong in that commit either. But it's too late to change history now. Adding the missing method definitions makes it possible to instantiate a RaidBackupFileSystem. Commit: ff7cef1f0cbe229816b6ed990e3d9594877f2867 https://github.com/boxbackup/boxbackup/commit/ff7cef1f0cbe229816b6ed990e3d9594877f2867 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Refactor BackupStoreAccountControl to be for a specific account ID This allows other BackupAccountControl implementations to implement the same interface without needing to support account IDs, which are very store-specific. Commit: 6b9874151f8d9b19c7bb487578438c3539767611 https://github.com/boxbackup/boxbackup/commit/6b9874151f8d9b19c7bb487578438c3539767611 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/backupstore/testbackupstore.cpp Log Message: ----------- Remove debugging code to list files during test_multiple_uploads Revert "Try listing files to help debug random failures of test_multiple_uploads()." This reverts commit e302ded5b7d40fe88d65808564f4db9df2a6f878. Commit: 4abf84f59d66ef5ec231aaac4a67d228f28ae929 https://github.com/boxbackup/boxbackup/commit/4abf84f59d66ef5ec231aaac4a67d228f28ae929 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in M infrastructure/travis-build.sh M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupConstants.h M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupProtocol.txt M lib/backupstore/BackupStoreAccounts.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreFile.cpp M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/HousekeepStoreAccount.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/bbstored/BackupStoreDaemon.cpp M lib/common/CollectInBufferStream.cpp M lib/common/CommonException.txt M lib/common/NamedLock.cpp M lib/httpserver/S3Simulator.cpp M runtest.pl.in M test/backupstore/testbackupstore.cpp M test/backupstorefix/testbackupstorefix.cpp M test/backupstorepatch/testbackupstorepatch.cpp M test/bbackupd/testbbackupd.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace and comment cleanup Commit: 45b6827ffa3d0cf88946ecf87864faba31c293b5 https://github.com/boxbackup/boxbackup/commit/45b6827ffa3d0cf88946ecf87864faba31c293b5 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. Commit: cfa908c019fbafa765a0033597a5a1a82f44756b https://github.com/boxbackup/boxbackup/commit/cfa908c019fbafa765a0033597a5a1a82f44756b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum Commit: 7ff17a242fa140c50b272f6bc65f4850e06a76f4 https://github.com/boxbackup/boxbackup/commit/7ff17a242fa140c50b272f6bc65f4850e06a76f4 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Add GetTempDirPath() for portability On Unix, check TEMP, TMP and TMPDIR, and if no env var is set, fall back to /tmp. For some reason it's becoming increasingly common for Unixes not to set TMP* to anything. Commit: 29b82edb4fe9477681726f135c03cc6a16dbab70 https://github.com/boxbackup/boxbackup/commit/29b82edb4fe9477681726f135c03cc6a16dbab70 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Fix include ordering in testcommon.cpp Commit: 7d9ee0d1141eaf8f0293ccda78761379abe94a4a https://github.com/boxbackup/boxbackup/commit/7d9ee0d1141eaf8f0293ccda78761379abe94a4a Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Refactor testcommon to split into independent tests Remove redundant NamedLock tests. Commit: 787c0811406d0210e13b8673a3874fe7f2a0303c https://github.com/boxbackup/boxbackup/commit/787c0811406d0210e13b8673a3874fe7f2a0303c Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h M test/common/testcommon.cpp Log Message: ----------- Add force parameter to RemovePrefix and RemoveSuffix Commit: ca57fd20f556f6da63af44b9d537200db156be31 https://github.com/boxbackup/boxbackup/commit/ca57fd20f556f6da63af44b9d537200db156be31 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Remove BackupStoreContext::ReceivedFinishCommand Replace with CleanUp(). We don't need two methods that do almost the same thing. Commit: c6f30d197decc927c3383e30e405b9b5ce019aa8 https://github.com/boxbackup/boxbackup/commit/c6f30d197decc927c3383e30e405b9b5ce019aa8 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreContext::GetBackupStoreInfo Almost all access to mapStoreInfo is now via GetBackupStoreInfo(Internal), in preparation for the refactor that will move the main copy of the BackupStoreInfo to the BackupFileSystem instance instead. Commit: 667be66c228689c6da0d973577d633b1962caff5 https://github.com/boxbackup/boxbackup/commit/667be66c228689c6da0d973577d633b1962caff5 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupStoreContext.cpp Log Message: ----------- Enable C++0x support We need C++0x features enabled for "auto" to work, and using Boost properly seems to require this. Use "auto" in a few places to prove that it works. Commit: 2829e058bf5b50e3b17ae1d89602b63bf56beea3 https://github.com/boxbackup/boxbackup/commit/2829e058bf5b50e3b17ae1d89602b63bf56beea3 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor to create BackupFileSystem::GetBackupStoreInfo Extract common code from filesystem-specific implementations, and rename them to GetBackupStoreInfoInternal(). Commit: 2bdb3e01a8da58b589e30ea9bd532e116fc1ed39 https://github.com/boxbackup/boxbackup/commit/2bdb3e01a8da58b589e30ea9bd532e116fc1ed39 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor BackupFileSystem::GetLock() Call subclass implementations of TryGetLock(), and throw an exception if we fail to get a lock instead of returning false. Commit: 8970b6fbfee051e0d1f3e2fb7eafee8e66567880 https://github.com/boxbackup/boxbackup/commit/8970b6fbfee051e0d1f3e2fb7eafee8e66567880 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Refactor test code into process_is_running() We need the ability to check whether a process is still running to check for stale S3 account locks in SimpleDB, so move this code out of Test.cpp and into Utils.cpp, for use outside of tests. Commit: 09797280350f4b86266eb49fe1c66f419df4a7ff https://github.com/boxbackup/boxbackup/commit/09797280350f4b86266eb49fe1c66f419df4a7ff Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Test.h M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Implement compare_lists and test_equal_maps in lib/common/Test.cpp Commit: 7a161a4d18cb9ebbaa8c57ab214d4742a8599b0b https://github.com/boxbackup/boxbackup/commit/7a161a4d18cb9ebbaa8c57ab214d4742a8599b0b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPQueryDecoder.cpp M lib/httpserver/HTTPQueryDecoder.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add static method HTTPQueryDecoder::URLEncode This will be needed for request parameter encoding and for SimpleDB. Commit: 96f95536400ee21fade30560adf36f0fd8aab83b https://github.com/boxbackup/boxbackup/commit/96f95536400ee21fade30560adf36f0fd8aab83b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add query parameter editing and encoding to HTTPRequest Commit: 65d8023ac492d7463a1d25d5301dcbf9fe339152 https://github.com/boxbackup/boxbackup/commit/65d8023ac492d7463a1d25d5301dcbf9fe339152 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for virtualhost names in S3Client This is a precondition of support for both S3 and SimpleDB in the S3Simulator, since otherwise we can't tell the requests apart. Commit: cbe4870794e9b9356c3abec2f5a648822397cd73 https://github.com/boxbackup/boxbackup/commit/cbe4870794e9b9356c3abec2f5a648822397cd73 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 863a065436fb9d171779078f90fff21a425aacff https://github.com/boxbackup/boxbackup/commit/863a065436fb9d171779078f90fff21a425aacff Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: 76b2c2e8c504e1e4e5cb905d2e05959202aad398 https://github.com/boxbackup/boxbackup/commit/76b2c2e8c504e1e4e5cb905d2e05959202aad398 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: cd7c776dabd22fe53ae1c6eb4a4f61ed3c1a4824 https://github.com/boxbackup/boxbackup/commit/cd7c776dabd22fe53ae1c6eb4a4f61ed3c1a4824 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: bc6aea1db9f303a25836389b30204b1b35c8b9c5 https://github.com/boxbackup/boxbackup/commit/bc6aea1db9f303a25836389b30204b1b35c8b9c5 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: 917b09c2110406eabc55c5d54150d7427dca1de5 https://github.com/boxbackup/boxbackup/commit/917b09c2110406eabc55c5d54150d7427dca1de5 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: 5f4848700189f15191d21f618da772649ebaec69 https://github.com/boxbackup/boxbackup/commit/5f4848700189f15191d21f618da772649ebaec69 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: 0bbe6122114f752f18a41babb31e744799d9833b https://github.com/boxbackup/boxbackup/commit/0bbe6122114f752f18a41babb31e744799d9833b Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: 00c45656b68b80531d7ddebbe843bfcab4308018 https://github.com/boxbackup/boxbackup/commit/00c45656b68b80531d7ddebbe843bfcab4308018 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat This was broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Commit: 61e8a72b560292654364954eda71dad28308c7a6 https://github.com/boxbackup/boxbackup/commit/61e8a72b560292654364954eda71dad28308c7a6 Author: Chris Wilson Date: 2017-08-04 (Fri, 04 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Compare: https://github.com/boxbackup/boxbackup/compare/b51e3b94f850...61e8a72b5602 From noreply at github.com Fri Aug 4 22:28:55 2017 From: noreply at github.com (GitHub) Date: Fri, 04 Aug 2017 14:28:55 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5984e7178a41d_7c523fbe0e157c3024835@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170804.61e8a72 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sat Aug 5 11:22:51 2017 From: noreply at github.com (GitHub) Date: Sat, 05 Aug 2017 03:22:51 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 4f8144: Add simulator and client for SimpleDB Message-ID: <59859c7ba5d77_7cf33fb5d3e31c3c678c3@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 4f81440fbf1272c8691eb10963d28ffd460d0088 https://github.com/boxbackup/boxbackup/commit/4f81440fbf1272c8691eb10963d28ffd460d0088 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 89eaa2d878f54973d2ec5d3f7d8f0e793f99f703 https://github.com/boxbackup/boxbackup/commit/89eaa2d878f54973d2ec5d3f7d8f0e793f99f703 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: bdbe9055b8735dac00dcfcfd64049de5d0d59385 https://github.com/boxbackup/boxbackup/commit/bdbe9055b8735dac00dcfcfd64049de5d0d59385 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: 3e34d1685268b75b2606d525a4c8c0eaa759f65a https://github.com/boxbackup/boxbackup/commit/3e34d1685268b75b2606d525a4c8c0eaa759f65a Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: ef9322c3b116b8470d89d2dd7d362e8beec56eb3 https://github.com/boxbackup/boxbackup/commit/ef9322c3b116b8470d89d2dd7d362e8beec56eb3 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: ff54af64f1a5cf71f5bdc2d907e5d9245960ee73 https://github.com/boxbackup/boxbackup/commit/ff54af64f1a5cf71f5bdc2d907e5d9245960ee73 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: 42d5d94d9f05265bca1df86a827d04d869717456 https://github.com/boxbackup/boxbackup/commit/42d5d94d9f05265bca1df86a827d04d869717456 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: 1ec26b5252fcaf491e202fc427427b60fe873998 https://github.com/boxbackup/boxbackup/commit/1ec26b5252fcaf491e202fc427427b60fe873998 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: d2775d5b8e0dbcc4debb7c1434014cf77fcaf188 https://github.com/boxbackup/boxbackup/commit/d2775d5b8e0dbcc4debb7c1434014cf77fcaf188 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat This was broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Commit: 15c728d442baf401e3c83365e1f9757cccdd65c4 https://github.com/boxbackup/boxbackup/commit/15c728d442baf401e3c83365e1f9757cccdd65c4 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Compare: https://github.com/boxbackup/boxbackup/compare/61e8a72b5602...15c728d442ba From noreply at github.com Sat Aug 5 11:25:46 2017 From: noreply at github.com (GitHub) Date: Sat, 05 Aug 2017 03:25:46 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 6029e3: Switch PCRE back to an official release Message-ID: <59859d2a7490a_2b33ff0ea6fdc345619d@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 6029e32f21c194a0eb57f1b67b810e63d88935d6 https://github.com/boxbackup/boxbackup/commit/6029e32f21c194a0eb57f1b67b810e63d88935d6 Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release From noreply at github.com Sat Aug 5 11:38:43 2017 From: noreply at github.com (GitHub) Date: Sat, 05 Aug 2017 03:38:43 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 1a0b56: Remove BoxBackup install step from Windows unibuil... Message-ID: <5985a03317196_46923f92bd3efc3063937@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 1a0b568f5541632c3de048040abe9e59a2d5e79a https://github.com/boxbackup/boxbackup/commit/1a0b568f5541632c3de048040abe9e59a2d5e79a Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. From noreply at github.com Sat Aug 5 11:44:32 2017 From: noreply at github.com (GitHub) Date: Sat, 05 Aug 2017 03:44:32 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 4fa4ac: Switch from nmake to JOM for OpenSSL unibuild on W... Message-ID: <5985a190ab66b_70103fd86c11bc301350c7@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 4fa4ac7ebae865572d23dfdfe162d6e16ed86c9d https://github.com/boxbackup/boxbackup/commit/4fa4ac7ebae865572d23dfdfe162d6e16ed86c9d Author: Chris Wilson Date: 2017-08-05 (Sat, 05 Aug 2017) Changed paths: M appveyor.yml M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch from nmake to JOM for OpenSSL unibuild on Windows JOM should be faster, especially for multicore machines. From noreply at github.com Sat Aug 5 11:46:08 2017 From: noreply at github.com (GitHub) Date: Sat, 05 Aug 2017 03:46:08 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5985a1f04e23e_7cf33fb5d3e31c3c68021@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170805.15c728d Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 6 16:32:10 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 08:32:10 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 446e82: Fix incorrect detection of the need to intercept l... Message-ID: <5987367aea90f_553b3fd499d93c287512e@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 446e82f2e3e012f9f271b6f84097b258e2731cef https://github.com/boxbackup/boxbackup/commit/446e82f2e3e012f9f271b6f84097b258e2731cef Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.cpp M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat For some reason, #if defined __GNUC__ && __GNUC__ >= 2 was true on OSX, resulting in intercepts not working properly. It was also broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 Now we detect when to intercept Linux' lxstat syscall using autoconf macros. We also detect whether to intercept stat64 (instead of stat) based on whether the stat64 function exists. (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Fix d2775d5b Commit: c18627ea024e02998a9f2c39e2c4ffc2e500c877 https://github.com/boxbackup/boxbackup/commit/c18627ea024e02998a9f2c39e2c4ffc2e500c877 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Commit: 87519732ca9e06495bda4ab110bfe6a699d1e19e https://github.com/boxbackup/boxbackup/commit/87519732ca9e06495bda4ab110bfe6a699d1e19e Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release Commit: 2339dd4de149bd221d5c85d94f5125ceafb2f95f https://github.com/boxbackup/boxbackup/commit/2339dd4de149bd221d5c85d94f5125ceafb2f95f Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. Commit: f5e6bbc68807a6130a6437d5c65e728fa3738973 https://github.com/boxbackup/boxbackup/commit/f5e6bbc68807a6130a6437d5c65e728fa3738973 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M appveyor.yml M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch from nmake to JOM for OpenSSL unibuild on Windows JOM should be faster, especially for multicore machines. Compare: https://github.com/boxbackup/boxbackup/compare/4fa4ac7ebae8...f5e6bbc68807 From noreply at github.com Sun Aug 6 20:24:57 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 12:24:57 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] e9ca64: Increase default verbosity in some timing tests Message-ID: <59876d094104b_18d23fbe8af9dc3c18277d@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: e9ca6460239e0f2f33e658c27fcee88dbf044908 https://github.com/boxbackup/boxbackup/commit/e9ca6460239e0f2f33e658c27fcee88dbf044908 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/buildenv-testmain-template.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/common/Test.h M test/backupstorefix/testbackupstorefix.cpp M test/bbackupd/testbbackupd.cpp Log Message: ----------- Increase default verbosity in some timing tests Commit: df110155f8bf435af2359b2adc17e694682daa44 https://github.com/boxbackup/boxbackup/commit/df110155f8bf435af2359b2adc17e694682daa44 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in Log Message: ----------- Detect which debugger to use in t-gdb scripts: lldb or gdb Should make debugging on Mac and BSD less painful. Make t-gdb fail with an error when no debugger was detected. (cherry picked from commit da98a8183737fb1fcb658cebd9612791a4750c4c) (cherry picked from commit ea6bc722131222d1fad08521e61f552f16941b01) Commit: f37d2089cd0e3ae46cceaa91e388dec48f963608 https://github.com/boxbackup/boxbackup/commit/f37d2089cd0e3ae46cceaa91e388dec48f963608 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Don't add CXXFLAGS to CFLAGS Some CXXFLAGS such as enabling C++11 don't make sense for CC and will be rejected, causing the compile to fail. We don't really test for flags being compatible with the C compiler anyway, and we don't have enough C files to care that much. (cherry picked from commit 2ad9ee9012d4a7354c33ff0272c0768b952a9064) Commit: 621d3e2c547e1c0ff38a23fa1a9826aba8dc6c64 https://github.com/boxbackup/boxbackup/commit/621d3e2c547e1c0ff38a23fa1a9826aba8dc6c64 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Perl buildsystem: add test prepare targets Similar to the CMake prepare targets, these are useful when intending to debug tests or run a server using a standard test configuration, which requires the testfiles directory to be copied into the binary output tree. (cherry picked from commit 2afecdd57a3bcf42c116c8535843039a1596feac) Commit: 93f53a0f2bc5c0b20b70a3a5df5ed9237e8c14e0 https://github.com/boxbackup/boxbackup/commit/93f53a0f2bc5c0b20b70a3a5df5ed9237e8c14e0 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Rename BackupStoreAccountsControl to BackupStoreAccountControl For consistency with the base class, BackupAccountControl, and grammatical reasons. Commit: e3dacc32bdbd1caaed5c6134a626866a52119b75 https://github.com/boxbackup/boxbackup/commit/e3dacc32bdbd1caaed5c6134a626866a52119b75 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Add missing implementations of RaidBackupFileSystem methods These should have been added whenever BackupFileSystem.h was added, which was 4db4deed3448b9c2b9e885a20cb1a777632464d6, although it didn't really belong in that commit either. But it's too late to change history now. Adding the missing method definitions makes it possible to instantiate a RaidBackupFileSystem. Commit: 905f8444f41de01280ea3f5e7ab12e795acaea2a https://github.com/boxbackup/boxbackup/commit/905f8444f41de01280ea3f5e7ab12e795acaea2a Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/StoreTestUtils.cpp Log Message: ----------- Refactor BackupStoreAccountControl to be for a specific account ID This allows other BackupAccountControl implementations to implement the same interface without needing to support account IDs, which are very store-specific. Commit: 7eb7cb664d6645937553a589c7f10f20c03f4343 https://github.com/boxbackup/boxbackup/commit/7eb7cb664d6645937553a589c7f10f20c03f4343 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M test/backupstore/testbackupstore.cpp Log Message: ----------- Remove debugging code to list files during test_multiple_uploads Revert "Try listing files to help debug random failures of test_multiple_uploads()." This reverts commit e302ded5b7d40fe88d65808564f4db9df2a6f878. Commit: 963f8d0cb9f0c60c26a7d53c2dd1328ba9ac1117 https://github.com/boxbackup/boxbackup/commit/963f8d0cb9f0c60c26a7d53c2dd1328ba9ac1117 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M bin/bbstoreaccounts/bbstoreaccounts.cpp M infrastructure/m4/boxbackup_tests.m4 M infrastructure/makebuildenv.pl.in M infrastructure/travis-build.sh M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupConstants.h M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupProtocol.txt M lib/backupstore/BackupStoreAccounts.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreFile.cpp M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/HousekeepStoreAccount.cpp M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/bbstored/BackupStoreDaemon.cpp M lib/common/CollectInBufferStream.cpp M lib/common/CommonException.txt M lib/common/NamedLock.cpp M lib/httpserver/S3Simulator.cpp M runtest.pl.in M test/backupstore/testbackupstore.cpp M test/backupstorefix/testbackupstorefix.cpp M test/backupstorepatch/testbackupstorepatch.cpp M test/bbackupd/testbbackupd.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace and comment cleanup Commit: e80848efa754a1d1afcd9039608e98ce2e2143d1 https://github.com/boxbackup/boxbackup/commit/e80848efa754a1d1afcd9039608e98ce2e2143d1 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. Commit: 0486495c7dc2d134bf39b5f779338bf40d88e418 https://github.com/boxbackup/boxbackup/commit/0486495c7dc2d134bf39b5f779338bf40d88e418 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum Commit: 487dc5ff2bab291e45f056e709e17c2eefb8020b https://github.com/boxbackup/boxbackup/commit/487dc5ff2bab291e45f056e709e17c2eefb8020b Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Add GetTempDirPath() for portability On Unix, check TEMP, TMP and TMPDIR, and if no env var is set, fall back to /tmp. For some reason it's becoming increasingly common for Unixes not to set TMP* to anything. Commit: 4c5908b8ef31adb617296b5a4c28a7743e496819 https://github.com/boxbackup/boxbackup/commit/4c5908b8ef31adb617296b5a4c28a7743e496819 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Fix include ordering in testcommon.cpp Commit: d7019ca9782771be6ffffa87edc58148b7ac635d https://github.com/boxbackup/boxbackup/commit/d7019ca9782771be6ffffa87edc58148b7ac635d Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M test/common/testcommon.cpp Log Message: ----------- Refactor testcommon to split into independent tests Remove redundant NamedLock tests. Commit: f2331727148bf6650d973b029daa0c826f00a42e https://github.com/boxbackup/boxbackup/commit/f2331727148bf6650d973b029daa0c826f00a42e Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Utils.cpp M lib/common/Utils.h M test/common/testcommon.cpp Log Message: ----------- Add force parameter to RemovePrefix and RemoveSuffix Commit: f15ad172c01392241c44d2125015145bcc1902a0 https://github.com/boxbackup/boxbackup/commit/f15ad172c01392241c44d2125015145bcc1902a0 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Remove BackupStoreContext::ReceivedFinishCommand Replace with CleanUp(). We don't need two methods that do almost the same thing. Commit: 35b69ad8b77f4965c9fedacf2fa1f98b9beb0cf3 https://github.com/boxbackup/boxbackup/commit/35b69ad8b77f4965c9fedacf2fa1f98b9beb0cf3 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreContext::GetBackupStoreInfo Almost all access to mapStoreInfo is now via GetBackupStoreInfo(Internal), in preparation for the refactor that will move the main copy of the BackupStoreInfo to the BackupFileSystem instance instead. Commit: 91458e0124b68053623e350082481db4b20f8771 https://github.com/boxbackup/boxbackup/commit/91458e0124b68053623e350082481db4b20f8771 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupStoreContext.cpp Log Message: ----------- Enable C++0x support We need C++0x features enabled for "auto" to work, and using Boost properly seems to require this. Use "auto" in a few places to prove that it works. Commit: c54adcf104e678154e56e3ab1bacdcb980d5d31a https://github.com/boxbackup/boxbackup/commit/c54adcf104e678154e56e3ab1bacdcb980d5d31a Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor to create BackupFileSystem::GetBackupStoreInfo Extract common code from filesystem-specific implementations, and rename them to GetBackupStoreInfoInternal(). Commit: f0f72d72c43067ac9eb74fb3985b83cdf51725b7 https://github.com/boxbackup/boxbackup/commit/f0f72d72c43067ac9eb74fb3985b83cdf51725b7 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h Log Message: ----------- Refactor BackupFileSystem::GetLock() Call subclass implementations of TryGetLock(), and throw an exception if we fail to get a lock instead of returning false. Commit: 578a7ff44605992df365613a61901d41a34efb86 https://github.com/boxbackup/boxbackup/commit/578a7ff44605992df365613a61901d41a34efb86 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Refactor test code into process_is_running() We need the ability to check whether a process is still running to check for stale S3 account locks in SimpleDB, so move this code out of Test.cpp and into Utils.cpp, for use outside of tests. Commit: bd72422470696a16ca7a6e7b79881f809ceeb071 https://github.com/boxbackup/boxbackup/commit/bd72422470696a16ca7a6e7b79881f809ceeb071 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/common/Test.h M lib/common/Utils.cpp M lib/common/Utils.h Log Message: ----------- Implement compare_lists and test_equal_maps in lib/common/Test.cpp Commit: b6cc88d87751c0e42fe1445bc3310a23985ac9e5 https://github.com/boxbackup/boxbackup/commit/b6cc88d87751c0e42fe1445bc3310a23985ac9e5 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/httpserver/HTTPQueryDecoder.cpp M lib/httpserver/HTTPQueryDecoder.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add static method HTTPQueryDecoder::URLEncode This will be needed for request parameter encoding and for SimpleDB. Commit: 11a88c3d38719a83b5f8d59094811dacb8ce425a https://github.com/boxbackup/boxbackup/commit/11a88c3d38719a83b5f8d59094811dacb8ce425a Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add query parameter editing and encoding to HTTPRequest Commit: 4706cf9c7b2c6bd1ca7cebe7587c5ccf037871b9 https://github.com/boxbackup/boxbackup/commit/4706cf9c7b2c6bd1ca7cebe7587c5ccf037871b9 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for virtualhost names in S3Client This is a precondition of support for both S3 and SimpleDB in the S3Simulator, since otherwise we can't tell the requests apart. Commit: 1f6330c009190762849167ad88385c76ea0cb160 https://github.com/boxbackup/boxbackup/commit/1f6330c009190762849167ad88385c76ea0cb160 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 85de3d133f82d8999e4c4298d6d4026e184288ab https://github.com/boxbackup/boxbackup/commit/85de3d133f82d8999e4c4298d6d4026e184288ab Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: 4f9161ae445c12b677bef15d5042ed566be7e6f9 https://github.com/boxbackup/boxbackup/commit/4f9161ae445c12b677bef15d5042ed566be7e6f9 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: a91b9e1ce6eeef18fb9d7077521d6fa965db68f1 https://github.com/boxbackup/boxbackup/commit/a91b9e1ce6eeef18fb9d7077521d6fa965db68f1 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: 10879cbd43e984f9514888da101baf92d225a750 https://github.com/boxbackup/boxbackup/commit/10879cbd43e984f9514888da101baf92d225a750 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: cab466abf7d7a054fef016ab76f59b781a11e833 https://github.com/boxbackup/boxbackup/commit/cab466abf7d7a054fef016ab76f59b781a11e833 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: b9eac0da4580371e0e3ccc5672728d841993efce https://github.com/boxbackup/boxbackup/commit/b9eac0da4580371e0e3ccc5672728d841993efce Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: 40c98155629a22e241046ea514c91e9195ea5c90 https://github.com/boxbackup/boxbackup/commit/40c98155629a22e241046ea514c91e9195ea5c90 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: b7f68a8889560b75cc21969ad3356224c2b49a76 https://github.com/boxbackup/boxbackup/commit/b7f68a8889560b75cc21969ad3356224c2b49a76 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.cpp M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat For some reason, #if defined __GNUC__ && __GNUC__ >= 2 was true on OSX, resulting in intercepts not working properly. It was also broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 Now we detect when to intercept Linux' lxstat syscall using autoconf macros. We also detect whether to intercept stat64 (instead of stat) based on whether the stat64 function exists. (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Fix d2775d5b Commit: 0b10723c9ac0e77d1a21ca844818c99d5143ffa5 https://github.com/boxbackup/boxbackup/commit/0b10723c9ac0e77d1a21ca844818c99d5143ffa5 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Commit: 4bc54f160595dfeb40297cce13937b13441de06e https://github.com/boxbackup/boxbackup/commit/4bc54f160595dfeb40297cce13937b13441de06e Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release Commit: 3c91f74eb65bfd1d1b7a55786da7e84413cb1640 https://github.com/boxbackup/boxbackup/commit/3c91f74eb65bfd1d1b7a55786da7e84413cb1640 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. Commit: 2cb5ac91a9cebe596ba12eaa4bec9639e4cba0c1 https://github.com/boxbackup/boxbackup/commit/2cb5ac91a9cebe596ba12eaa4bec9639e4cba0c1 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Run OpenSSL tests after building it for Windows Unibuild Compare: https://github.com/boxbackup/boxbackup/compare/f5e6bbc68807...2cb5ac91a9ce From noreply at github.com Sun Aug 6 20:26:32 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 12:26:32 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 929e0b: Run OpenSSL tests after building it for Windows Un... Message-ID: <59876d68c08af_49193ffa03495c2c8156d@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 929e0b4a3fabb014e89c4539187a106aa197822a https://github.com/boxbackup/boxbackup/commit/929e0b4a3fabb014e89c4539187a106aa197822a Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Run OpenSSL tests after building it for Windows Unibuild From noreply at github.com Sun Aug 6 20:29:25 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 12:29:25 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] bd39b8: Upgrade to OpenSSL 1.0.2l Message-ID: <59876e155d38d_135d3fc5fc271c38132774@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: bd39b891928847048b428cacae37bc09ae629459 https://github.com/boxbackup/boxbackup/commit/bd39b891928847048b428cacae37bc09ae629459 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Upgrade to OpenSSL 1.0.2l Run OpenSSL tests after building it for Windows Unibuild From noreply at github.com Sun Aug 6 21:03:34 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 13:03:34 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] c8e3b1: Improve exception messages from RaidFileWrite Message-ID: <5987761654863_7ead3f92a357dc3877625@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: c8e3b1fb62c7445cc99ca04b542367bb6529ab7f https://github.com/boxbackup/boxbackup/commit/c8e3b1fb62c7445cc99ca04b542367bb6529ab7f Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/raidfile/RaidFileWrite.cpp Log Message: ----------- Improve exception messages from RaidFileWrite Use THROW_SYS_FILE_ERROR to include the system error message in the exception message. Replace calls to BOX_LOG_SYS_ERROR where previously used. From noreply at github.com Sun Aug 6 21:06:21 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 13:06:21 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <598776bd180ba_2d233fc08e853c3c487d8@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170806.bd39b89 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 6 21:30:44 2017 From: noreply at github.com (GitHub) Date: Sun, 06 Aug 2017 13:30:44 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59877c749742_77643fb6f522fc2c977a2@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170806.c8e3b1f Home: https://github.com/boxbackup/boxbackup From noreply at github.com Mon Aug 7 21:02:02 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 13:02:02 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 2820af: Add support for S3-compatible ETag headers to S3Si... Message-ID: <5988c73abd926_3b343fc36826fc301165f1@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 2820af7928807fc2c62fbc16e75da964e18fd673 https://github.com/boxbackup/boxbackup/commit/2820af7928807fc2c62fbc16e75da964e18fd673 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: 09c0e9e5119a26491bd06c3314fa5eb546d48966 https://github.com/boxbackup/boxbackup/commit/09c0e9e5119a26491bd06c3314fa5eb546d48966 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: 017642701d0de7669550866ec6124581a6c51296 https://github.com/boxbackup/boxbackup/commit/017642701d0de7669550866ec6124581a6c51296 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: 65aa45ce63ce38df99cc6e635f18679adf321529 https://github.com/boxbackup/boxbackup/commit/65aa45ce63ce38df99cc6e635f18679adf321529 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: d3d1add6aeab6c11483a5ddbbef433e4a8724b50 https://github.com/boxbackup/boxbackup/commit/d3d1add6aeab6c11483a5ddbbef433e4a8724b50 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: 14fdd6abe961514bb99a88500c5ce6dc2d6e006d https://github.com/boxbackup/boxbackup/commit/14fdd6abe961514bb99a88500c5ce6dc2d6e006d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: dc649a6f5bdc25576d603ad541e4c9a5bd79b202 https://github.com/boxbackup/boxbackup/commit/dc649a6f5bdc25576d603ad541e4c9a5bd79b202 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: 90315349c93559cdf577a2b77ab7ce0e975ab432 https://github.com/boxbackup/boxbackup/commit/90315349c93559cdf577a2b77ab7ce0e975ab432 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.cpp M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat For some reason, #if defined __GNUC__ && __GNUC__ >= 2 was true on OSX, resulting in intercepts not working properly. It was also broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 Now we detect when to intercept Linux' lxstat syscall using autoconf macros. We also detect whether to intercept stat64 (instead of stat) based on whether the stat64 function exists. (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Fix d2775d5b Commit: acea1e03e1a7808df6291a234b1715f0ab8cff0e https://github.com/boxbackup/boxbackup/commit/acea1e03e1a7808df6291a234b1715f0ab8cff0e Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Commit: 06303743d7f0c3bba2b56fb957ccb1abef8df418 https://github.com/boxbackup/boxbackup/commit/06303743d7f0c3bba2b56fb957ccb1abef8df418 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release Commit: 36acbe9dfcb77dc5d63d796e1514945a7b10a503 https://github.com/boxbackup/boxbackup/commit/36acbe9dfcb77dc5d63d796e1514945a7b10a503 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. Commit: 01589e4858e16e5cd0d91ed21cdf711b8e89bf56 https://github.com/boxbackup/boxbackup/commit/01589e4858e16e5cd0d91ed21cdf711b8e89bf56 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Upgrade to OpenSSL 1.0.2l Run OpenSSL tests after building it for Windows Unibuild Commit: e767f3d0bd4556a2ef4136f6bd368279a2a5b57d https://github.com/boxbackup/boxbackup/commit/e767f3d0bd4556a2ef4136f6bd368279a2a5b57d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/raidfile/RaidFileWrite.cpp Log Message: ----------- Improve exception messages from RaidFileWrite Use THROW_SYS_FILE_ERROR to include the system error message in the exception message. Replace calls to BOX_LOG_SYS_ERROR where previously used. Commit: 7683001bb37ed0769d6beaf7a440bf1202c8f0e9 https://github.com/boxbackup/boxbackup/commit/7683001bb37ed0769d6beaf7a440bf1202c8f0e9 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M .travis.yml Log Message: ----------- Travis CI: exclude (don't build) tags created by AppVeyor AppVeyor creates new tags for each release (binary build), which have the same git commit that we already built, so building it again is a waste. Commit: 646ab9f63c7546bde838bfd9a327ae1b1be41da0 https://github.com/boxbackup/boxbackup/commit/646ab9f63c7546bde838bfd9a327ae1b1be41da0 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPResponse.h M lib/httpserver/S3Client.cpp Log Message: ----------- Add ResponseCodeString shortcut to HTTPResponse Useful for debug logging and exception messages. Add the actual HTTP response to the exception message in S3Client::CheckResponse, to help with debugging. Commit: b3579582a8d9891edd8adfb3566a47cf4765201d https://github.com/boxbackup/boxbackup/commit/b3579582a8d9891edd8adfb3566a47cf4765201d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Add option to include parameters in HTTPRequest::GetRequestURI This is needed to support the ListBucket command. Compare: https://github.com/boxbackup/boxbackup/compare/c8e3b1fb62c7...b3579582a8d9 From noreply at github.com Mon Aug 7 21:12:32 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 13:12:32 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 0d50d1: Add simulator and client for SimpleDB Message-ID: <5988c9b04b847_2b33ff0ea6fdc3470945@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 0d50d1f43b110cdd5588ba186c40be3a67b660ef https://github.com/boxbackup/boxbackup/commit/0d50d1f43b110cdd5588ba186c40be3a67b660ef Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 10dcb33588cc7caa6c265a2f9bec3771576bbc19 https://github.com/boxbackup/boxbackup/commit/10dcb33588cc7caa6c265a2f9bec3771576bbc19 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: 931a618a4552ddeeba0ce8d835bd2f249bd78679 https://github.com/boxbackup/boxbackup/commit/931a618a4552ddeeba0ce8d835bd2f249bd78679 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: 763f4a6818e5a417dc2f3d90d7e22edfa462f33b https://github.com/boxbackup/boxbackup/commit/763f4a6818e5a417dc2f3d90d7e22edfa462f33b Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: 7700565d032190f68aac026345c6efb486ac0789 https://github.com/boxbackup/boxbackup/commit/7700565d032190f68aac026345c6efb486ac0789 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: 9fbc06ce47d662d7a7a8546046bbda4af01e84c3 https://github.com/boxbackup/boxbackup/commit/9fbc06ce47d662d7a7a8546046bbda4af01e84c3 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: 8e9ce7e0f5a593b701f71422e3fd9327376f0d50 https://github.com/boxbackup/boxbackup/commit/8e9ce7e0f5a593b701f71422e3fd9327376f0d50 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: 7c14015abb9677bac341a9ecff8cbb1f9a2f3a42 https://github.com/boxbackup/boxbackup/commit/7c14015abb9677bac341a9ecff8cbb1f9a2f3a42 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: f820bd70a201c3aef13a5f3a8645246b2061068b https://github.com/boxbackup/boxbackup/commit/f820bd70a201c3aef13a5f3a8645246b2061068b Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.cpp M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat For some reason, #if defined __GNUC__ && __GNUC__ >= 2 was true on OSX, resulting in intercepts not working properly. It was also broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 Now we detect when to intercept Linux' lxstat syscall using autoconf macros. We also detect whether to intercept stat64 (instead of stat) based on whether the stat64 function exists. (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Fix d2775d5b Commit: 359ec68a936fbdc6a8fcf6e98eedc9dcb9851237 https://github.com/boxbackup/boxbackup/commit/359ec68a936fbdc6a8fcf6e98eedc9dcb9851237 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Commit: 11bff52395875a182b5e84cf2e14d53ba06b1e2e https://github.com/boxbackup/boxbackup/commit/11bff52395875a182b5e84cf2e14d53ba06b1e2e Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release Commit: 61ca2325cdfab5ccf0b75930355855a784990e31 https://github.com/boxbackup/boxbackup/commit/61ca2325cdfab5ccf0b75930355855a784990e31 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. Commit: 36cbc34b46df8cd27a688a8373803dbbcca4f7bc https://github.com/boxbackup/boxbackup/commit/36cbc34b46df8cd27a688a8373803dbbcca4f7bc Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Upgrade to OpenSSL 1.0.2l Run OpenSSL tests after building it for Windows Unibuild Commit: 92d31043aaed661f1481a1e8fcea1d0f2a2838ec https://github.com/boxbackup/boxbackup/commit/92d31043aaed661f1481a1e8fcea1d0f2a2838ec Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/raidfile/RaidFileWrite.cpp Log Message: ----------- Improve exception messages from RaidFileWrite Use THROW_SYS_FILE_ERROR to include the system error message in the exception message. Replace calls to BOX_LOG_SYS_ERROR where previously used. Commit: 89d1a2e0d032a213bcf3acd3e0377e7fcdc7c04e https://github.com/boxbackup/boxbackup/commit/89d1a2e0d032a213bcf3acd3e0377e7fcdc7c04e Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M .travis.yml Log Message: ----------- Travis CI: exclude (don't build) tags created by AppVeyor AppVeyor creates new tags for each release (binary build), which have the same git commit that we already built, so building it again is a waste. Commit: e6b19d625c80b93553313130f1024cd93a823892 https://github.com/boxbackup/boxbackup/commit/e6b19d625c80b93553313130f1024cd93a823892 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPResponse.h M lib/httpserver/S3Client.cpp Log Message: ----------- Add ResponseCodeString shortcut to HTTPResponse Useful for debug logging and exception messages. Add the actual HTTP response to the exception message in S3Client::CheckResponse, to help with debugging. Commit: 1a84481484059b67bb910445d9f56baa70ed66a2 https://github.com/boxbackup/boxbackup/commit/1a84481484059b67bb910445d9f56baa70ed66a2 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Add option to include parameters in HTTPRequest::GetRequestURI This is needed to support the ListBucket command. Compare: https://github.com/boxbackup/boxbackup/compare/b3579582a8d9...1a8448148405 From noreply at github.com Mon Aug 7 21:26:56 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 13:26:56 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 1f6330: Add simulator and client for SimpleDB Message-ID: <5988cd10c0bc2_2a4e3faa17993c38102027@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170807.b357958 Home: https://github.com/boxbackup/boxbackup Commit: 1f6330c009190762849167ad88385c76ea0cb160 https://github.com/boxbackup/boxbackup/commit/1f6330c009190762849167ad88385c76ea0cb160 Author: Chris Wilson Date: 2017-08-06 (Sun, 06 Aug 2017) Changed paths: M lib/common/Test.cpp M lib/httpserver/HTTPException.txt M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h A lib/httpserver/SimpleDBClient.cpp A lib/httpserver/SimpleDBClient.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add simulator and client for SimpleDB This will be used to support reliable instant locking of S3 accounts. Commit: 2820af7928807fc2c62fbc16e75da964e18fd673 https://github.com/boxbackup/boxbackup/commit/2820af7928807fc2c62fbc16e75da964e18fd673 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Simulator.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for S3-compatible ETag headers to S3Simulator Commit: 09c0e9e5119a26491bd06c3314fa5eb546d48966 https://github.com/boxbackup/boxbackup/commit/09c0e9e5119a26491bd06c3314fa5eb546d48966 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M modules.txt M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for If-None-Match header to S3Client and S3Simulator This allows the client to only download a file if it doesn't match the one in the local cache, which makes it possible to use the cache very efficiently. Commit: 017642701d0de7669550866ec6124581a6c51296 https://github.com/boxbackup/boxbackup/commit/017642701d0de7669550866ec6124581a6c51296 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M test/bbackupd/testbbackupd.cpp Log Message: ----------- Rename setup_test_bbackupd() in test/bbackupd The name of this utility function was confusing with the SETUP_TEST_BBACKUPD macro which served an entirely different purpose, so I renamed it to prepare_test_with_client_daemon(). Commit: 65aa45ce63ce38df99cc6e635f18679adf321529 https://github.com/boxbackup/boxbackup/commit/65aa45ce63ce38df99cc6e635f18679adf321529 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupCommands.cpp M lib/backupstore/BackupStoreContext.cpp M lib/backupstore/BackupStoreContext.h Log Message: ----------- Make GetBlockSize() use BackupFileSystem Refactor BackupAccountControl, BackupStoreContext and BackupCommands to call BackupFileSystem to get the block size of the filesystem. It's a start. Commit: d3d1add6aeab6c11483a5ddbbef433e4a8724b50 https://github.com/boxbackup/boxbackup/commit/d3d1add6aeab6c11483a5ddbbef433e4a8724b50 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/backupstore/StoreTestUtils.h M lib/server/makeprotocol.pl.in Log Message: ----------- Protocol generator: record last error message received Store this as a string (as well as the error code) so that we can report it to the user, for example in test failure messages. Commit: 14fdd6abe961514bb99a88500c5ce6dc2d6e006d https://github.com/boxbackup/boxbackup/commit/14fdd6abe961514bb99a88500c5ce6dc2d6e006d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/Logging.h M lib/server/Socket.cpp Log Message: ----------- Improve error message on hostname lookup failure Throw exceptions with a message, including the OS error, which means retrieving it properly from the winsock library on Windows. Commit: dc649a6f5bdc25576d603ad541e4c9a5bd79b202 https://github.com/boxbackup/boxbackup/commit/dc649a6f5bdc25576d603ad541e4c9a5bd79b202 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 Log Message: ----------- Make narrowing conversions an error E.g. on ARM64 with unsigned chars, this is a real error, and causes the build to fail: https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=arm64&ver=0.11.1~r2837-4&stamp=1485057548&raw=0 (cherry picked from commit 3433291fc5bfbbdc87385796c26c44699edf5c0c) Commit: 90315349c93559cdf577a2b77ab7ce0e975ab432 https://github.com/boxbackup/boxbackup/commit/90315349c93559cdf577a2b77ab7ce0e975ab432 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/intercept/intercept.cpp M lib/intercept/intercept.h Log Message: ----------- Fix incorrect detection of the need to intercept lxstat For some reason, #if defined __GNUC__ && __GNUC__ >= 2 was true on OSX, resulting in intercepts not working properly. It was also broken on MIPS64, causing test/raidfile to fail with errors like: TEST FAILURE: Condition [TestGetFileSize(stripe1fn) == fs1] failed at testraidfile.cpp:415 https://buildd.debian.org/status/fetch.php?pkg=boxbackup&arch=mips64el&ver=0.11.1~r2837-4&stamp=1485060478&raw=0 Now we detect when to intercept Linux' lxstat syscall using autoconf macros. We also detect whether to intercept stat64 (instead of stat) based on whether the stat64 function exists. (cherry picked from commit a042c9951fabc1fb4f57e6d13d739192dda76025) Fix d2775d5b Commit: acea1e03e1a7808df6291a234b1715f0ab8cff0e https://github.com/boxbackup/boxbackup/commit/acea1e03e1a7808df6291a234b1715f0ab8cff0e Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/BoxTime.cpp Log Message: ----------- Increase precision of timing on Windows. Should hopefully reduce random failures of timing-dependent tests which rely on subsecond precision for accuracy. (cherry picked from commit c845b2e39fffeb560983a301d810616a6495469d) (cherry picked from commit ce7e000d43a91dcd0d39ee3a505521e700875556) Commit: 06303743d7f0c3bba2b56fb957ccb1abef8df418 https://github.com/boxbackup/boxbackup/commit/06303743d7f0c3bba2b56fb957ccb1abef8df418 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Switch PCRE back to an official release Commit: 36acbe9dfcb77dc5d63d796e1514945a7b10a503 https://github.com/boxbackup/boxbackup/commit/36acbe9dfcb77dc5d63d796e1514945a7b10a503 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Remove BoxBackup install step from Windows unibuild We don't need to install anything in C:\Program Files\BoxBackup, and often don't have permission to do that anyway. Commit: 01589e4858e16e5cd0d91ed21cdf711b8e89bf56 https://github.com/boxbackup/boxbackup/commit/01589e4858e16e5cd0d91ed21cdf711b8e89bf56 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M infrastructure/cmake/windows/CMakeLists.txt Log Message: ----------- Upgrade to OpenSSL 1.0.2l Run OpenSSL tests after building it for Windows Unibuild Commit: e767f3d0bd4556a2ef4136f6bd368279a2a5b57d https://github.com/boxbackup/boxbackup/commit/e767f3d0bd4556a2ef4136f6bd368279a2a5b57d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/raidfile/RaidFileWrite.cpp Log Message: ----------- Improve exception messages from RaidFileWrite Use THROW_SYS_FILE_ERROR to include the system error message in the exception message. Replace calls to BOX_LOG_SYS_ERROR where previously used. Commit: 7683001bb37ed0769d6beaf7a440bf1202c8f0e9 https://github.com/boxbackup/boxbackup/commit/7683001bb37ed0769d6beaf7a440bf1202c8f0e9 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M .travis.yml Log Message: ----------- Travis CI: exclude (don't build) tags created by AppVeyor AppVeyor creates new tags for each release (binary build), which have the same git commit that we already built, so building it again is a waste. Commit: 646ab9f63c7546bde838bfd9a327ae1b1be41da0 https://github.com/boxbackup/boxbackup/commit/646ab9f63c7546bde838bfd9a327ae1b1be41da0 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPResponse.h M lib/httpserver/S3Client.cpp Log Message: ----------- Add ResponseCodeString shortcut to HTTPResponse Useful for debug logging and exception messages. Add the actual HTTP response to the exception message in S3Client::CheckResponse, to help with debugging. Commit: b3579582a8d9891edd8adfb3566a47cf4765201d https://github.com/boxbackup/boxbackup/commit/b3579582a8d9891edd8adfb3566a47cf4765201d Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/HTTPRequest.cpp M lib/httpserver/HTTPRequest.h M lib/httpserver/S3Simulator.cpp Log Message: ----------- Add option to include parameters in HTTPRequest::GetRequestURI This is needed to support the ListBucket command. Compare: https://github.com/boxbackup/boxbackup/compare/1f6330c00919^...b3579582a8d9 From noreply at github.com Mon Aug 7 21:36:54 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 13:36:54 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] dde9c7: Fix FileExists in Utils.cpp Message-ID: <5988cf66ad7ff_12ab3f9789d91c2c14764@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: dde9c776733b7b75338b81531cde03999283c1f0 https://github.com/boxbackup/boxbackup/commit/dde9c776733b7b75338b81531cde03999283c1f0 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/common/Utils.cpp Log Message: ----------- Fix FileExists in Utils.cpp Testing for st.st_mode & S_IFLNK doesn't work on OSX, because IFLNK and IFREG share bits, so regular files have a nonzero result and appear to be links. It seems safer and more readable to use the mode test macros, e.g. S_ISLNK and S_ISDIR. Commit: ebb178b182836d616a34b2ac32afef2ae14b0e09 https://github.com/boxbackup/boxbackup/commit/ebb178b182836d616a34b2ac32afef2ae14b0e09 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add ListBucket command support to S3Simulator and S3Client Compare: https://github.com/boxbackup/boxbackup/compare/1a8448148405...ebb178b18283 From noreply at github.com Mon Aug 7 21:50:41 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 13:50:41 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5988d2a11d6a1_ec23fd224931c38155865@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170807.1a84481 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Mon Aug 7 22:15:30 2017 From: noreply at github.com (GitHub) Date: Mon, 07 Aug 2017 14:15:30 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5988d872578d3_1de63f9c03b09c38796ae@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170807.ebb178b Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sat Aug 12 22:21:40 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 14:21:40 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 9086a1: Fix negative estimated remaining time due to 64-bi... Message-ID: <598f71647f561_21c63ffae8e63c3c62135@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 9086a18820e44ccbdf1d35449fc3199d1fab8dd2 https://github.com/boxbackup/boxbackup/commit/9086a18820e44ccbdf1d35449fc3199d1fab8dd2 Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/common/ReadLoggingStream.cpp Log Message: ----------- Fix negative estimated remaining time due to 64-bit integer overflow on large files. (cherry picked from commit b27d4e7c61b04db054399435b5c57df1f9b1832b) From noreply at github.com Sat Aug 12 22:28:37 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 14:28:37 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] e1ddc9: Add ListBucket command support to S3Simulator and ... Message-ID: <598f7305b03dc_14a7d3ff190197c30883e5@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: e1ddc9360a1143f9447b9c0865ed6e2798121c6f https://github.com/boxbackup/boxbackup/commit/e1ddc9360a1143f9447b9c0865ed6e2798121c6f Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add ListBucket command support to S3Simulator and S3Client Commit: 10e256418eb8fb0cefe401ab2ae11352add87b9d https://github.com/boxbackup/boxbackup/commit/10e256418eb8fb0cefe401ab2ae11352add87b9d Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/common/ReadLoggingStream.cpp Log Message: ----------- Fix negative estimated remaining time due to 64-bit integer overflow on large files. (cherry picked from commit b27d4e7c61b04db054399435b5c57df1f9b1832b) Compare: https://github.com/boxbackup/boxbackup/compare/9086a18820e4...10e256418eb8 From noreply at github.com Sat Aug 12 22:36:29 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 14:36:29 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 7c0624: Add support for the HEAD and DELETE methods to the... Message-ID: <598f74ddf2f34_2ae83fb3420d3c2c89345@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 7c0624edb6d35dec47fa3e774d9a392fdf407d4e https://github.com/boxbackup/boxbackup/commit/7c0624edb6d35dec47fa3e774d9a392fdf407d4e Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add support for the HEAD and DELETE methods to the S3Simulator and S3Client. From noreply at github.com Sat Aug 12 22:46:24 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 14:46:24 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] ef6ea3: Enable memory testing under Darwin, as it does wor... Message-ID: <598f77307f8e0_55133feae0d4fc287311c@hookshot-fe1-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: ef6ea346a3341a9a76ceeba1dec259bce6168b12 https://github.com/boxbackup/boxbackup/commit/ef6ea346a3341a9a76ceeba1dec259bce6168b12 Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/common/BoxPlatform.h Log Message: ----------- Enable memory testing under Darwin, as it does work now (cherry picked from commit 86dc5ff3666c934dcf4fce261a93431b78bf018f) From noreply at github.com Sat Aug 12 22:47:12 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 14:47:12 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] ebb178: Add ListBucket command support to S3Simulator and ... Message-ID: <598f776071f00_35383f86f5f15c3c65060@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170812.9086a18 Home: https://github.com/boxbackup/boxbackup Commit: ebb178b182836d616a34b2ac32afef2ae14b0e09 https://github.com/boxbackup/boxbackup/commit/ebb178b182836d616a34b2ac32afef2ae14b0e09 Author: Chris Wilson Date: 2017-08-07 (Mon, 07 Aug 2017) Changed paths: M lib/httpserver/S3Client.cpp M lib/httpserver/S3Client.h M lib/httpserver/S3Simulator.cpp M lib/httpserver/S3Simulator.h M test/httpserver/testhttpserver.cpp Log Message: ----------- Add ListBucket command support to S3Simulator and S3Client Commit: 9086a18820e44ccbdf1d35449fc3199d1fab8dd2 https://github.com/boxbackup/boxbackup/commit/9086a18820e44ccbdf1d35449fc3199d1fab8dd2 Author: Chris Wilson Date: 2017-08-12 (Sat, 12 Aug 2017) Changed paths: M lib/common/ReadLoggingStream.cpp Log Message: ----------- Fix negative estimated remaining time due to 64-bit integer overflow on large files. (cherry picked from commit b27d4e7c61b04db054399435b5c57df1f9b1832b) Compare: https://github.com/boxbackup/boxbackup/compare/ebb178b18283^...9086a18820e4 From noreply at github.com Sat Aug 12 23:14:13 2017 From: noreply at github.com (GitHub) Date: Sat, 12 Aug 2017 15:14:13 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <598f7db5a707a_2d203fa31ba4dc3057250@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170812.ef6ea34 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 13 11:21:57 2017 From: noreply at github.com (GitHub) Date: Sun, 13 Aug 2017 03:21:57 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 9904f9: Rename local variable Message-ID: <59902845a5399_43c33f94f2999c44891a2@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 9904f9965ffb161f14f161bfe38afb4335ceafd9 https://github.com/boxbackup/boxbackup/commit/9904f9965ffb161f14f161bfe38afb4335ceafd9 Author: Chris Wilson Date: 2017-08-13 (Sun, 13 Aug 2017) Changed paths: M lib/backupstore/HousekeepStoreAccount.cpp Log Message: ----------- Rename local variable First part of refactoring HousekeepStoreAccount to use BackupFileSystem. Commit: f1ad4eb484f2333197ab7b04e57467b12de4cdb9 https://github.com/boxbackup/boxbackup/commit/f1ad4eb484f2333197ab7b04e57467b12de4cdb9 Author: Chris Wilson Date: 2017-08-13 (Sun, 13 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/BackupStoreRefCountDatabase.h Log Message: ----------- Fix conflict over unlocked refcount DB If BackupStoreCheck is are read-only, then it should not open a "temporary" database in the old sense, because that does actually change the store: the temporary file would conflict with any other process which wants to do the same thing at the same time (e.g. housekeeping), and if neither process locks the store, they will break each other. Instead, we now distinguish between "potential" refcount DBs (of which there can only be one at a time, and which require a lock to create) and really "temporary" ones which are created in a temporary directory, can never become permanent via Commit(), do not require a lock, and are unlimited in number. Commit() will rename them, but still in the temporary directory, so it has no effect on the store. This requires some changes to the BackupStoreRefCountDatabase::Create interface to add new flags. Compare: https://github.com/boxbackup/boxbackup/compare/ef6ea346a334...f1ad4eb484f2 From noreply at github.com Sun Aug 13 11:45:35 2017 From: noreply at github.com (GitHub) Date: Sun, 13 Aug 2017 03:45:35 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59902dcf89299_26dc3fd45d269c305522d@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170813.f1ad4eb Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 13 16:57:21 2017 From: noreply at github.com (GitHub) Date: Sun, 13 Aug 2017 08:57:21 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 761ad3: Refactor BackupStoreRefCountDatabase to an abstrac... Message-ID: <599076e1e955_38bc3ffc32063c40109367@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 761ad37d5671cf34404ad86465d3cd045eb50f46 https://github.com/boxbackup/boxbackup/commit/761ad37d5671cf34404ad86465d3cd045eb50f46 Author: Chris Wilson Date: 2017-08-13 (Sun, 13 Aug 2017) Changed paths: M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/BackupStoreRefCountDatabase.h Log Message: ----------- Refactor BackupStoreRefCountDatabase to an abstract interface This makes it easier to cleanly subclass in BackupFileSystem, to change its behaviour by hooking into its Commit() method. From noreply at github.com Sun Aug 13 17:20:18 2017 From: noreply at github.com (GitHub) Date: Sun, 13 Aug 2017 09:20:18 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59907c42425b4_35363fb0c0f17c2c2414e@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170813.761ad37 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Thu Aug 17 20:29:18 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 12:29:18 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] e5423d: Move management of BackupStoreRefCountDatabase int... Message-ID: <5995ee8e6dc62_7d13f994a8f7c30407d4@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: e5423d4b83fe89b96ace3962dd3e98ff52b337da https://github.com/boxbackup/boxbackup/commit/e5423d4b83fe89b96ace3962dd3e98ff52b337da Author: Chris Wilson Date: 2017-08-13 (Sun, 13 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreRefCountDatabase.cpp M lib/backupstore/BackupStoreRefCountDatabase.h Log Message: ----------- Move management of BackupStoreRefCountDatabase into BackupFileSystem This is needed to allow BackupFileSystem to commit the database to storage on request, and to cache it, which is needed for S3 support. Commit: a1146cf95ce864d9438d3d0a5545ae8bd875eaab https://github.com/boxbackup/boxbackup/commit/a1146cf95ce864d9438d3d0a5545ae8bd875eaab Author: Chris Wilson Date: 2017-08-17 (Thu, 17 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupAccountControl.h M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreAccounts.cpp M lib/backupstore/BackupStoreAccounts.h M lib/backupstore/BackupStoreException.txt M lib/backupstore/BackupStoreInfo.cpp M lib/backupstore/BackupStoreInfo.h Log Message: ----------- Initial refactor of BackupAccountControl to use BackupFileSystem Abstract away some account operations using BackupFileSystem so that they can be implemented differently for S3 accounts. In particular, account opening and NamedLock usage were targets for this refactor. BackupStoreInfo creation and opening ended up being partially addressed as well. This significantly reduces the diffs to the s3_support branch. Compare: https://github.com/boxbackup/boxbackup/compare/761ad37d5671...a1146cf95ce8 From noreply at github.com Thu Aug 17 20:59:23 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 12:59:23 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5995f59bb4074_6c383fc69f4a5c2c11433a@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170817.a1146cf Home: https://github.com/boxbackup/boxbackup From noreply at github.com Thu Aug 17 21:14:51 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 13:14:51 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] b0be6f: Enable compiler warning -Werror=overloaded-virtual... Message-ID: <5995f93b6168c_3d043feac922dc30142238@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: b0be6fb9cb147d71816fef46fe59ebb372442438 https://github.com/boxbackup/boxbackup/commit/b0be6fb9cb147d71816fef46fe59ebb372442438 Author: Chris Wilson Date: 2017-08-17 (Thu, 17 Aug 2017) Changed paths: M infrastructure/m4/boxbackup_tests.m4 M lib/backupstore/BackupProtocol.h M lib/backupstore/BackupStoreFile.h M lib/backupstore/BackupStoreFileCmbIdx.cpp M lib/backupstore/BackupStoreFileEncodeStream.h M lib/common/BufferedStream.h M lib/common/BufferedWriteStream.h M lib/common/CollectInBufferStream.h M lib/common/FileStream.h M lib/common/MemBlockStream.h M lib/common/PartialReadStream.h M lib/common/RateLimitingStream.h M lib/common/ReadGatherStream.h M lib/common/ReadLoggingStream.h M lib/common/SelfFlushingStream.h M lib/common/ZeroStream.h M lib/compress/CompressStream.h M lib/raidfile/RaidFileRead.h M lib/raidfile/RaidFileWrite.h M lib/server/ProtocolUncertainStream.h M lib/server/SocketStream.h M lib/server/SocketStreamTLS.h M lib/server/TcpNice.h M lib/server/WinNamedPipeStream.h M test/bbackupd/testbbackupd.cpp M test/compress/testcompress.cpp Log Message: ----------- Enable compiler warning -Werror=overloaded-virtual, and fix From noreply at github.com Thu Aug 17 21:48:54 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 13:48:54 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <5996013624d68_c373fe499a25c346104a@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170817.b0be6fb Home: https://github.com/boxbackup/boxbackup From noreply at github.com Thu Aug 17 22:01:24 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 14:01:24 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 50ca30: Refactor HousekeepStoreAccount to use BackupFileSy... Message-ID: <5996042437684_10053fbf41c81c44142150@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 50ca30ae6d58e8f3f8d4e15d5aca3dc1e0dedfcc https://github.com/boxbackup/boxbackup/commit/50ca30ae6d58e8f3f8d4e15d5aca3dc1e0dedfcc Author: Chris Wilson Date: 2017-08-17 (Thu, 17 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/HousekeepStoreAccount.cpp M lib/backupstore/HousekeepStoreAccount.h M lib/backupstore/StoreTestUtils.cpp M lib/backupstore/StoreTestUtils.h M lib/bbstored/BBStoreDHousekeeping.cpp Log Message: ----------- Refactor HousekeepStoreAccount to use BackupFileSystem This involved moving big chunks of functionality into RaidBackupFileSystem, for example CombineFile, CombineDiffs and RaidPutFileCompleteTransaction. Not much is implemented for S3BackupFileSystem yet. From noreply at github.com Thu Aug 17 22:36:33 2017 From: noreply at github.com (GitHub) Date: Thu, 17 Aug 2017 14:36:33 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59960c617d5dc_69973f993f1e1c3011529d@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170817.50ca30a Home: https://github.com/boxbackup/boxbackup From noreply at github.com Fri Aug 18 22:04:53 2017 From: noreply at github.com (GitHub) Date: Fri, 18 Aug 2017 14:04:53 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59975675b9834_39193f900b20fc3091627@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170818.50ca30a Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sat Aug 19 22:23:39 2017 From: noreply at github.com (GitHub) Date: Sat, 19 Aug 2017 14:23:39 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 39e13e: Whitespace and comment cleanup Message-ID: <5998ac5bb76cb_11c13ff911ec7c241365cd@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support Home: https://github.com/boxbackup/boxbackup Commit: 39e13ea37d403e51019403d52529f3f6abf66ed6 https://github.com/boxbackup/boxbackup/commit/39e13ea37d403e51019403d52529f3f6abf66ed6 Author: Chris Wilson Date: 2017-08-19 (Sat, 19 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in M lib/backupstore/BackupConstants.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/HousekeepStoreAccount.cpp M lib/common/CommonException.txt M lib/httpserver/S3Simulator.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace and comment cleanup (cherry picked from commit 963f8d0cb9f0c60c26a7d53c2dd1328ba9ac1117) Commit: 727125282872f2790d6d09e1a4af88ef651d0ac9 https://github.com/boxbackup/boxbackup/commit/727125282872f2790d6d09e1a4af88ef651d0ac9 Author: Chris Wilson Date: 2017-08-19 (Sat, 19 Aug 2017) Changed paths: M lib/backupclient/BackupClientRestore.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/common/Test.cpp M lib/httpserver/S3Simulator.cpp Log Message: ----------- Make ObjectExists and BackupStoreCheck use object_exists_t enum (cherry picked from commit 0486495c7dc2d134bf39b5f779338bf40d88e418) Commit: b646d22f415bbc8f5b5b065ed7462166591404f2 https://github.com/boxbackup/boxbackup/commit/b646d22f415bbc8f5b5b065ed7462166591404f2 Author: Chris Wilson Date: 2017-08-19 (Sat, 19 Aug 2017) Changed paths: M infrastructure/cmake/CMakeLists.txt M infrastructure/m4/boxbackup_tests.m4 M lib/bbackupd/BackupClientContext.cpp M lib/bbackupd/BackupClientContext.h M lib/server/TcpNice.cpp M lib/server/TcpNice.h Log Message: ----------- Add support for TcpNice on OSX, remove on Windows I'm not convinced that it does anything on Windows without an RTT estimate, which we can't get without administrator access: https://msdn.microsoft.com/en-us/library/windows/desktop/bb485738(v=vs.85).aspx Compare: https://github.com/boxbackup/boxbackup/compare/e09abc7017dc...b646d22f415b From noreply at github.com Sat Aug 19 22:24:19 2017 From: noreply at github.com (GitHub) Date: Sat, 19 Aug 2017 14:24:19 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] f399fa: Custom build system: fix autogen dependencies Message-ID: <5998ac83188ba_cf73fac1041dc40134382@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support Home: https://github.com/boxbackup/boxbackup Commit: f399fa8b98a6d2f948adba55c1ac9b630c172533 https://github.com/boxbackup/boxbackup/commit/f399fa8b98a6d2f948adba55c1ac9b630c172533 Author: Chris Wilson Date: 2017-08-19 (Sat, 19 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in Log Message: ----------- Custom build system: fix autogen dependencies Remove useless inclusion of Makefile.extras with paths in subdirs that will never match anything, and replace those inclusions with recursive Make calls to build those targets. This allows changes to exception and protocol files to be detected automatically, and the C++ code regenerated. (cherry picked from commit e80848efa754a1d1afcd9039608e98ce2e2143d1) From noreply at github.com Sat Aug 19 22:36:47 2017 From: noreply at github.com (GitHub) Date: Sat, 19 Aug 2017 14:36:47 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 4c3718: Whitespace cleanup Message-ID: <5998af6fb7d2d_6b863fca1394bc3c1619fd@hookshot-fe1-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support Home: https://github.com/boxbackup/boxbackup Commit: 4c37189ffb1abc0aedaa5c13bff3257b15d831d5 https://github.com/boxbackup/boxbackup/commit/4c37189ffb1abc0aedaa5c13bff3257b15d831d5 Author: Chris Wilson Date: 2017-08-19 (Sat, 19 Aug 2017) Changed paths: M infrastructure/makebuildenv.pl.in M lib/backupstore/BackupFileSystem.cpp M lib/common/MainHelper.cpp M test/common/testcommon.cpp M test/httpserver/testhttpserver.cpp Log Message: ----------- Whitespace cleanup From noreply at github.com Sat Aug 26 17:27:51 2017 From: noreply at github.com (GitHub) Date: Sat, 26 Aug 2017 09:27:51 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 78e9f8: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a1a18771667_61873fe6b0a9dc3422996@hookshot-fe1-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 78e9f8b6ebea5cf4a4448587a8619ade9376b5f3 https://github.com/boxbackup/boxbackup/commit/78e9f8b6ebea5cf4a4448587a8619ade9376b5f3 Author: Chris Wilson Date: 2017-08-26 (Sat, 26 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 1 Create a RaidBackupFileSystem locally in BackupStoreCheck and call GetAccountIdentifier() on it. Trivial, but it's a start. From noreply at github.com Sat Aug 26 17:58:57 2017 From: noreply at github.com (GitHub) Date: Sat, 26 Aug 2017 09:58:57 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a1a8d1bd474_1093c3f9847b8dc3c3218d@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170826.78e9f8b Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 27 10:24:15 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 02:24:15 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] ecdf33: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a28fbf3eb49_46c33f97d9629c2868969@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: ecdf33bab3006870702c50ecc42fba4231035685 https://github.com/boxbackup/boxbackup/commit/ecdf33bab3006870702c50ecc42fba4231035685 Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 1 Pass the BackupFileSystem from BackupAccountControl to BackupStoreCheck, and use it for locking the account (actually a no-op because it's already locked). Trivial, but it's a start. From noreply at github.com Sun Aug 27 10:28:21 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 02:28:21 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 90ab66: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a290b578481_2603fa6017a1c3c93399@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 90ab66f13c907dd4126e75fb66296f40b36bc95e https://github.com/boxbackup/boxbackup/commit/90ab66f13c907dd4126e75fb66296f40b36bc95e Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 2 Use BackupFileSystem to access the refcount database. From noreply at github.com Sun Aug 27 10:35:52 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 02:35:52 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 8992d8: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a29278383c6_109803f8a78df1c3017525@hookshot-fe-6dbb0c4.cp1-iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 8992d86a50a2cf75f0ee2bf4b3662020017ebe41 https://github.com/boxbackup/boxbackup/commit/8992d86a50a2cf75f0ee2bf4b3662020017ebe41 Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 1 Pass the BackupFileSystem from BackupAccountControl to BackupStoreCheck, and use it for locking the account (actually a no-op because it's already locked). Trivial, but it's a start. Commit: bebb4ce8024e67434a59f8bdf6ae35bff592437b https://github.com/boxbackup/boxbackup/commit/bebb4ce8024e67434a59f8bdf6ae35bff592437b Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 2 Use BackupFileSystem to access the refcount database. Compare: https://github.com/boxbackup/boxbackup/compare/90ab66f13c90...bebb4ce8024e From noreply at github.com Sun Aug 27 10:55:14 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 02:55:14 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] ecdf33: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a29702bf51a_7bfc3fc9f7ba7c2c3255f@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170827.90ab66f Home: https://github.com/boxbackup/boxbackup Commit: ecdf33bab3006870702c50ecc42fba4231035685 https://github.com/boxbackup/boxbackup/commit/ecdf33bab3006870702c50ecc42fba4231035685 Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 1 Pass the BackupFileSystem from BackupAccountControl to BackupStoreCheck, and use it for locking the account (actually a no-op because it's already locked). Trivial, but it's a start. Commit: 90ab66f13c907dd4126e75fb66296f40b36bc95e https://github.com/boxbackup/boxbackup/commit/90ab66f13c907dd4126e75fb66296f40b36bc95e Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 2 Use BackupFileSystem to access the refcount database. Compare: https://github.com/boxbackup/boxbackup/compare/ecdf33bab300^...90ab66f13c90 From noreply at github.com Sun Aug 27 11:30:33 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 03:30:33 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 1b856f: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a29f494a852_3a803fa8d6fa3c3c19952@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 1b856f21c06393274284a44680c4f3530addb2c5 https://github.com/boxbackup/boxbackup/commit/1b856f21c06393274284a44680c4f3530addb2c5 Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreInfo.cpp M lib/backupstore/BackupStoreInfo.h M test/backupstore/testbackupstore.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 3 Use BackupFileSystem to access the per-account BackupStoreInfo. From noreply at github.com Sun Aug 27 11:32:50 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 03:32:50 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a29fd2f2af5_4d6f3fe43909fc2467546@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170827.bebb4ce Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 27 11:57:51 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 03:57:51 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a2a5af14e41_44fe3f9b952a5c401654e@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170827.1b856f2 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 27 20:34:56 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 12:34:56 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 1cb75f: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a31ee0c1bdd_41963fbd36b8bc34488bc@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 1cb75f28de801bb983dfff51417e168f32db9dcb https://github.com/boxbackup/boxbackup/commit/1cb75f28de801bb983dfff51417e168f32db9dcb Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreCheck.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 4 Use BackupFileSystem to EnsureObjectIsPermanent. From noreply at github.com Sun Aug 27 20:41:19 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 12:41:19 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 243257: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a3205f52465_773c3fd1fb3e7c382556@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 2432570200cef3abd70b321ecbfe48db30b741d5 https://github.com/boxbackup/boxbackup/commit/2432570200cef3abd70b321ecbfe48db30b741d5 Author: Chris Wilson Date: 2017-08-27 (Sun, 27 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 5 Use BackupFileSystem to Get and Put Files and Directories. From noreply at github.com Sun Aug 27 20:58:38 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 12:58:38 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a3246e5b5d2_79593fab885abc347658e@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170827.1cb75f2 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Sun Aug 27 21:23:41 2017 From: noreply at github.com (GitHub) Date: Sun, 27 Aug 2017 13:23:41 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a32a4df130_39e23f91fe72dc2c474f0@hookshot-fe2-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170827.2432570 Home: https://github.com/boxbackup/boxbackup From noreply at github.com Mon Aug 28 18:49:14 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 10:49:14 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 19d2fc: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a4579ae5bae_4b2d3ff2e82f5c2c62761@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 19d2fccd46c54f4d543da482d90d198354230566 https://github.com/boxbackup/boxbackup/commit/19d2fccd46c54f4d543da482d90d198354230566 Author: Chris Wilson Date: 2017-08-28 (Mon, 28 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreCheck.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 6 Use BackupFileSystem to delete corrupt objects. From noreply at github.com Mon Aug 28 19:12:55 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 11:12:55 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a45d27a6317_3e923f9c5637fc4466188@hookshot-fe6-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170828.19d2fcc Home: https://github.com/boxbackup/boxbackup From noreply at github.com Mon Aug 28 19:22:11 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 11:22:11 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 61aa8c: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a45f53ddfa6_79593fab885abc34871ac@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 61aa8cd490ff112a713e64419e2aff7efd16f171 https://github.com/boxbackup/boxbackup/commit/61aa8cd490ff112a713e64419e2aff7efd16f171 Author: Chris Wilson Date: 2017-08-28 (Mon, 28 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupFileSystem.h M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck2.cpp M lib/backupstore/BackupStoreException.txt Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 7 Move most low-level filesystem structure check code into BackupFileSystem, and add specialised implementation for S3. From noreply at github.com Mon Aug 28 19:49:52 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 11:49:52 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] 92c3ef: Refactor BackupStoreCheck to use BackupFileSystem, ... Message-ID: <59a465d08183_172c3fd470611c4445c8@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: 92c3ef0cfbe76a90256ebf106a5d5c6bd9d34499 https://github.com/boxbackup/boxbackup/commit/92c3ef0cfbe76a90256ebf106a5d5c6bd9d34499 Author: Chris Wilson Date: 2017-08-28 (Mon, 28 Aug 2017) Changed paths: M lib/backupstore/BackupAccountControl.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Refactor BackupStoreCheck to use BackupFileSystem, part 8 Remove last vestiges of RaidFile from BackupStoreCheck. From noreply at github.com Mon Aug 28 19:53:44 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 11:53:44 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] dbcb5f: Backport improvements from s3_support_merge branch Message-ID: <59a466b84dd36_4cac3fdb36193c2c263a4@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support Home: https://github.com/boxbackup/boxbackup Commit: dbcb5f0f03a4b6a88ef472aa1cb751978758a2b8 https://github.com/boxbackup/boxbackup/commit/dbcb5f0f03a4b6a88ef472aa1cb751978758a2b8 Author: Chris Wilson Date: 2017-08-28 (Mon, 28 Aug 2017) Changed paths: M lib/backupstore/BackupFileSystem.cpp M lib/backupstore/BackupStoreCheck.cpp M lib/backupstore/BackupStoreCheck.h M lib/backupstore/BackupStoreInfo.h Log Message: ----------- Backport improvements from s3_support_merge branch (cherry picked from commit 92ed15404dbb84186dbf4a7d8d29b585e97e81c4) From noreply at github.com Mon Aug 28 19:56:48 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 11:56:48 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] a86dd7: Rename some local variables in BackupStoreCheck Message-ID: <59a467706c86b_1f9b3fccbd3b7c38908d@hookshot-fe3-cp1-prd.iad.github.net.mail> Branch: refs/heads/s3_support_merge Home: https://github.com/boxbackup/boxbackup Commit: a86dd72584a927627984070a9ac8ef51b7cd122b https://github.com/boxbackup/boxbackup/commit/a86dd72584a927627984070a9ac8ef51b7cd122b Author: Chris Wilson Date: 2017-08-28 (Mon, 28 Aug 2017) Changed paths: M lib/backupstore/BackupStoreCheck2.cpp Log Message: ----------- Rename some local variables in BackupStoreCheck From noreply at github.com Mon Aug 28 20:04:13 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 12:04:13 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a4692d566cc_12ca3f9275aebc3475761@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170828.61aa8cd Home: https://github.com/boxbackup/boxbackup From noreply at github.com Mon Aug 28 22:01:47 2017 From: noreply at github.com (GitHub) Date: Mon, 28 Aug 2017 14:01:47 -0700 Subject: [Box Backup-commit] [boxbackup/boxbackup] Message-ID: <59a484bb4f284_af33fb172f35c384656e@hookshot-fe5-cp1-prd.iad.github.net.mail> Branch: refs/tags/BoxBackup-0.12.s3_support_merge.170828.a86dd72 Home: https://github.com/boxbackup/boxbackup