From heartswild83 at gmail.com Mon Dec 10 03:54:05 2018 From: heartswild83 at gmail.com (Debra Heart) Date: Sun, 9 Dec 2018 19:54:05 -0800 Subject: [Box Backup] Thanks In-Reply-To: References: Message-ID: Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From heartswild83 at gmail.com Thu Dec 13 03:25:46 2018 From: heartswild83 at gmail.com (Debra Heart) Date: Wed, 12 Dec 2018 19:25:46 -0800 Subject: [Box Backup] (no subject) Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Attachment001.png Type: image/png Size: 19678 bytes Desc: not available URL: From hjb at pro-linux.de Sun Dec 30 17:07:47 2018 From: hjb at pro-linux.de (Hans-Joachim Baader) Date: Sun, 30 Dec 2018 18:07:47 +0100 Subject: [Box Backup] Object too big error on 32 bit client Message-ID: <20181230170746.qt463sugbvl2brqn@toba.hjbaader.home> Hi, today I verified my backups by downloading them on a separate machine. This machine is 32 bit and very ancient - Debian 8 is the system currently running on it and there will be no update. The verification works well, except when it encounters a large file (about 280 MB in this case). Then it aborts with THROW_EXCEPTION(ConnectionException, Protocol_ObjTooBig) This comes from lib/server/Protocol.cpp line 195. I tried the current git version and an older version on both server and client. When I restore this file on the original machine it works. This is a 64 bit machine. So it appears there is some difference in the 32 bit and 64 bit versions of boxbackup. The 64 bit client works, the 32 bit client doesn't. I don't know if the server makes a difference. This is a 32 bit machine, too. It recently had its 20. anniversary runing almost without interruptions. Since it just doesn't want do die, I didn't move boxbackup to another machine yet. Don't hurt me :) Regards, Hans-Joachim Hans-Joachim Baader - Pro-Linux.de Poppelfeld 9, D-76646 Bruchsal Gesch?ftsf?hrer/Herausgeber * CEO/Editor Fon +49 07257-930142 * Mobil 0170-6500228 -- Pro-Linux - Germany's largest volunteer Linux support site Support Pro-Linux: https://www.pro-linux.de/user/index50.html https://www.pro-linux.de/ Public Key ID 2DBC923E3DDBDDEA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available URL: From chris+google at qwirx.com Sun Dec 30 19:55:16 2018 From: chris+google at qwirx.com (Chris Wilson) Date: Sun, 30 Dec 2018 19:55:16 +0000 Subject: [Box Backup] Object too big error on 32 bit client In-Reply-To: <20181230170746.qt463sugbvl2brqn@toba.hjbaader.home> References: <20181230170746.qt463sugbvl2brqn@toba.hjbaader.home> Message-ID: Hi Hans-Joachim, Very sorry to hear that you're having problems. I'm not aware of any difference or incompatibility between 32-bit and 64-bit clients and servers. I want to fix this bug, but will need your help to reproduce it. This particular error can happen when the client encounters an error in a previous operation, and tries to move onto the next file, but the protocol is out of sync (the server is still sending file data, but the client tries to interpret it as a reply to its next request, and aborts). Therefore it would be useful to see the last few lines of the output, including any error that occurred immediately before, or while restoring the previous file. The server logs may also have some useful information, such as warnings. How large is the file that failed on the server, in blocks? Can you restore it on its own (without the other files)? Can you restore the file immediately before, and then perform another protocol operation such as listing files in a directory? Thanks, Chris. On Sun, 30 Dec 2018 at 17:18, Hans-Joachim Baader wrote: > Hi, > > today I verified my backups by downloading them on a separate machine. > This machine is 32 bit and very ancient - Debian 8 is the system > currently running on it and there will be no update. > > The verification works well, except when it encounters a large > file (about 280 MB in this case). Then it aborts with > > THROW_EXCEPTION(ConnectionException, Protocol_ObjTooBig) > > This comes from lib/server/Protocol.cpp line 195. I tried the > current git version and an older version on both server and > client. > > When I restore this file on the original machine it works. This is > a 64 bit machine. So it appears there is some difference in > the 32 bit and 64 bit versions of boxbackup. The 64 bit client > works, the 32 bit client doesn't. > > I don't know if the server makes a difference. This is a 32 bit machine, > too. It recently had its 20. anniversary runing almost without > interruptions. Since it just doesn't want do die, I didn't move > boxbackup to another machine yet. Don't hurt me :) > > Regards, > Hans-Joachim > > Hans-Joachim Baader - Pro-Linux.de > Poppelfeld 9, D-76646 Bruchsal > Gesch?ftsf?hrer/Herausgeber * CEO/Editor > Fon +49 07257-930142 * Mobil 0170-6500228 > > -- > Pro-Linux - Germany's largest volunteer Linux support site > Support Pro-Linux: https://www.pro-linux.de/user/index50.html > https://www.pro-linux.de/ Public Key ID 2DBC923E3DDBDDEA > _______________________________________________ > Boxbackup mailing list > Boxbackup at boxbackup.org > http://lists.boxbackup.org/mailman/listinfo/boxbackup > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hjb at pro-linux.de Sun Dec 30 23:09:00 2018 From: hjb at pro-linux.de (Hans-Joachim Baader) Date: Mon, 31 Dec 2018 00:09:00 +0100 Subject: [Box Backup] Object too big error on 32 bit client In-Reply-To: References: <20181230170746.qt463sugbvl2brqn@toba.hjbaader.home> Message-ID: <20181230230859.ysb65g7xobb6hlr2@toba.hjbaader.home> Hi Chris, > Very sorry to hear that you're having problems. I'm not aware of any > difference or incompatibility between 32-bit and 64-bit clients and > servers. I want to fix this bug, but will need your help to reproduce it. thank you. > This particular error can happen when the client encounters an error in a > previous operation, and tries to move onto the next file, but the protocol > is out of sync (the server is still sending file data, but the client tries > to interpret it as a reply to its next request, and aborts). Therefore it > would be useful to see the last few lines of the output, including any > error that occurred immediately before, or while restoring the previous > file. The server logs may also have some useful information, such as > warnings. > > How large is the file that failed on the server, in blocks? Can you restore > it on its own (without the other files)? Can you restore the file > immediately before, and then perform another protocol operation such as > listing files in a directory? After some tests, I must correct some of my statements. I can reproduce the problem with a single file, 26 MB long (maybe the size doesn't have anything to do with the problem). Contrary to what I said the behavior is the same on all 32 and 64 bit clients. A simple get command is sufficient to reproduce the error. It also happens when the restore command is used. On the server side I see the following in the log, and nothing else of interest: Dec 30 23:38:54 server bbstored client=0x00000016[28613]: WARNING: Exception thrown: ConnectionException(TLSWriteFailed) at lib/server/SocketStreamTLS.cpp(372) Dec 30 23:38:54 server bbstored client=0x00000016[28613]: NOTICE: Connection statistics for 0x00000016 (name=): IN=189 OUT=628976 NET_IN=-628787 TOTAL=629165 Dec 30 23:38:54 server bbstored[28613]: ERROR: Error in child process, terminating connection: TLSWriteFailed (7/33) These entries only appear in the log after the client disconnects, this may be long after the error happens on the client. I straced the server and didn't see any error there. So the error seems to be produced on the client. The client says: query > get item.MYI WARNING: zlib error code = -3 WARNING: Exception thrown: CompressException(TransformFailed) at ../../lib/compress/Compress.h(154) ERROR: Failed to fetch file: Compress TransformFailed And after additional commands: query > ls WARNING: Exception thrown: ConnectionException(Conn_Protocol_ObjTooBig) at Protocol.cpp(219) ERROR: Failed to list directory: Connection Protocol_ObjTooBig query > exit WARNING: Exception thrown: ConnectionException(Conn_Protocol_ObjTooBig) at Protocol.cpp(219) Exception: Connection Protocol_ObjTooBig (7/42) The same messages are in the syslog. I did a ltrace on the client and got: 19104 SSL_read(0x55a6a0643d40, 0x55a6a067079f, 2129, 0xdbba0) = 2129 19104 SSL_get_error(0x55a6a0643d40, 2129, 0x55a6a0644410, 0) = 0 19104 EVP_CIPHER_CTX_iv_length(0x55a69f0dcba0, 2129, 0x55a69f0dcd60, 8608) = 16 19104 EVP_CipherInit(0x55a69f0dcba0, 0, 0, 0x55a6a06707a0) = 1 19104 EVP_CipherInit(0x55a69f0dcba0, 0, 0, 0) = 1 19104 EVP_CIPHER_CTX_block_size(0x55a69f0dcba0, 2048, 16, 0) = 16 19104 inflateInit_(0x7fffa6d0ba60, 0x55a69eeb90be, 112, 0) = 0 19104 EVP_CIPHER_CTX_block_size(0x55a69f0dcba0, 0x7fffa6d0bf60, 2048, 0x55a6a06707b0) = 16 19104 EVP_CipherUpdate(0x55a69f0dcba0, 0x7fffa6d0bf60, 0x7fffa6d0b55c, 0x55a6a06707b0) = 1 19104 inflate(0x7fffa6d0ba60, 0, 0, 0) = 0xfffffffd So, zlib has indeed a problem. What next? The data store on the server seems Ok, I checked that. I could test again with more verbose output. Or delete the file in the store to have it backed up again. Zlib is version 1.2.8 or 1.2.6. Regards, Hans-Joachim Hans-Joachim Baader - Pro-Linux.de Poppelfeld 9, D-76646 Bruchsal Gesch?ftsf?hrer/Herausgeber * CEO/Editor Fon +49 07257-930142 * Mobil 0170-6500228 -- Pro-Linux - Germany's largest volunteer Linux support site Support Pro-Linux: https://www.pro-linux.de/user/index50.html https://www.pro-linux.de/ Public Key ID 2DBC923E3DDBDDEA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available URL: From chris+google at qwirx.com Mon Dec 31 14:57:25 2018 From: chris+google at qwirx.com (Chris Wilson) Date: Mon, 31 Dec 2018 14:57:25 +0000 Subject: [Box Backup] Object too big error on 32 bit client In-Reply-To: <20181230230859.ysb65g7xobb6hlr2@toba.hjbaader.home> References: <20181230170746.qt463sugbvl2brqn@toba.hjbaader.home> <20181230230859.ysb65g7xobb6hlr2@toba.hjbaader.home> Message-ID: Hi Hans-Joachim, If the file cannot be restored correctly on any client, then it seems that the file data is corrupt. Since individual blocks are encrypted separately, it seems that one block was probably corrupted before being encrypted, perhaps due to bad memory or a cosmic ray bit flip on the client being backed up. (It should not be possible that it was damaged after encryption, including on the server, as a different error would occur on decryption). Do you have any old versions of this file on the store, and can they be restored successfully? This should be detected by verifying (comparing) the backup on the original client, which is something that you should really be doing regularly to check the integrity of your backups. It would probably be useful to add a command to bbackupquery that decrypts the blocks and checks that they can be decompressed, without comparing to the current file data, as this would be easier on the box (less file I/O) and could be done on a different box (with the same key). The Zlib manual says that it's possible to resync the compression and partially restore the file in such cases, or at least we could drop the encrypted block. The file would have errors, but in some cases some content could be recovered from it. In this case, if this is really a MySQL ISAM database file as it appears, it could probably be repaired by myisamchk with a few lost records. You can delete the file from the server with the bbackupquery delete command, and try the restore again, and it should succeed this time (hopefully!). Thanks, Chris. On Sun, 30 Dec 2018 at 23:09, Hans-Joachim Baader wrote: > Hi Chris, > > > Very sorry to hear that you're having problems. I'm not aware of any > > difference or incompatibility between 32-bit and 64-bit clients and > > servers. I want to fix this bug, but will need your help to reproduce it. > > thank you. > > > This particular error can happen when the client encounters an error in a > > previous operation, and tries to move onto the next file, but the > protocol > > is out of sync (the server is still sending file data, but the client > tries > > to interpret it as a reply to its next request, and aborts). Therefore it > > would be useful to see the last few lines of the output, including any > > error that occurred immediately before, or while restoring the previous > > file. The server logs may also have some useful information, such as > > warnings. > > > > How large is the file that failed on the server, in blocks? Can you > restore > > it on its own (without the other files)? Can you restore the file > > immediately before, and then perform another protocol operation such as > > listing files in a directory? > > After some tests, I must correct some of my statements. I can reproduce > the problem with a single file, 26 MB long (maybe the size doesn't have > anything to do with the problem). Contrary to what I said the behavior > is the same on all 32 and 64 bit clients. > > A simple get command is sufficient to reproduce the error. It also happens > when the restore command is used. On the server side I see the following > in the log, and nothing else of interest: > > Dec 30 23:38:54 server bbstored client=0x00000016[28613]: WARNING: > Exception thrown: ConnectionException(TLSWriteFailed) at > lib/server/SocketStreamTLS.cpp(372) > Dec 30 23:38:54 server bbstored client=0x00000016[28613]: NOTICE: > Connection statistics for 0x00000016 (name=): IN=189 OUT=628976 > NET_IN=-628787 TOTAL=629165 > Dec 30 23:38:54 server bbstored[28613]: ERROR: Error in child process, > terminating connection: TLSWriteFailed (7/33) > > These entries only appear in the log after the client disconnects, > this may be long after the error happens on the client. > > I straced the server and didn't see any error there. So the error > seems to be produced on the client. > > The client says: > > query > get item.MYI > WARNING: zlib error code = -3 > WARNING: Exception thrown: CompressException(TransformFailed) at > ../../lib/compress/Compress.h(154) > ERROR: Failed to fetch file: Compress TransformFailed > > And after additional commands: > > query > ls > WARNING: Exception thrown: ConnectionException(Conn_Protocol_ObjTooBig) at > Protocol.cpp(219) > ERROR: Failed to list directory: Connection Protocol_ObjTooBig > query > exit > WARNING: Exception thrown: ConnectionException(Conn_Protocol_ObjTooBig) at > Protocol.cpp(219) > Exception: Connection Protocol_ObjTooBig (7/42) > > The same messages are in the syslog. I did a ltrace on the client and got: > 19104 SSL_read(0x55a6a0643d40, 0x55a6a067079f, 2129, 0xdbba0) > = 2129 > 19104 SSL_get_error(0x55a6a0643d40, 2129, 0x55a6a0644410, 0) > = 0 > 19104 EVP_CIPHER_CTX_iv_length(0x55a69f0dcba0, 2129, 0x55a69f0dcd60, > 8608) = 16 > 19104 EVP_CipherInit(0x55a69f0dcba0, 0, 0, 0x55a6a06707a0) > = 1 > 19104 EVP_CipherInit(0x55a69f0dcba0, 0, 0, 0) > = 1 > 19104 EVP_CIPHER_CTX_block_size(0x55a69f0dcba0, 2048, 16, 0) > = 16 > 19104 inflateInit_(0x7fffa6d0ba60, 0x55a69eeb90be, 112, 0) > = 0 > 19104 EVP_CIPHER_CTX_block_size(0x55a69f0dcba0, 0x7fffa6d0bf60, 2048, > 0x55a6a06707b0) = 16 > 19104 EVP_CipherUpdate(0x55a69f0dcba0, 0x7fffa6d0bf60, 0x7fffa6d0b55c, > 0x55a6a06707b0) = 1 > 19104 inflate(0x7fffa6d0ba60, 0, 0, 0) > = 0xfffffffd > > So, zlib has indeed a problem. What next? The data store on the server > seems Ok, I checked that. I could test again with more verbose output. > Or delete the file in the store to have it backed up again. > > Zlib is version 1.2.8 or 1.2.6. > > Regards, > Hans-Joachim > > Hans-Joachim Baader - Pro-Linux.de > Poppelfeld 9, D-76646 Bruchsal > Gesch?ftsf?hrer/Herausgeber * CEO/Editor > Fon +49 07257-930142 * Mobil 0170-6500228 > -- > Pro-Linux - Germany's largest volunteer Linux support site > Support Pro-Linux: https://www.pro-linux.de/user/index50.html > https://www.pro-linux.de/ Public Key ID 2DBC923E3DDBDDEA > -------------- next part -------------- An HTML attachment was scrubbed... URL: