[Box Backup] Problem with 'get' command on large files

[Ben Summers]

On 9 Aug 2006, at 10:56, Per Thomsen wrote:

>
> On the server, I get the following in the (extended logging;  
> release-build) logs:
> -------------------------------
> Aug  9 01:47:01 backup01 bbstored[19717]: Sending stream, size 256
> Aug  9 01:47:01 backup01 bbstored[19717]: Receive GetFile 
> (0x31e85,0x37666)
> Aug  9 01:47:01 backup01 bbstored[19717]: Receive GetFile 
> (0x31e85,0x37666)
> Aug  9 01:47:01 backup01 bbstored[19717]: Send Success(0x37666)
> Aug  9 01:47:01 backup01 bbstored[19717]: Send Success(0x37666)
> Aug  9 01:47:01 backup01 bbstored[19717]: Sending stream, size  
> -1141640418
> Aug  9 01:47:01 backup01 bbstored[19717]: Connection statistics for  
> BACKUP-0005: IN=210 OUT=75545 TOTAL=75755
> Aug  9 01:47:01 backup01 bbstored[19717]: in server child,  
> exception RaidFile OSError (Error when accessing an underlying  
> file. Check file permissions allow files to be read and written in  
> the configured raid directories.) (2/8) -- terminating child  
> -------------------------------
>
> I noticed the negative number for stream size, and worry that it is  
> there because something in my builds is not using 64-bit numbers  
> for file size. The file I'm trying to restore (id 00032547) is  
> 3,153,326,894 bytes in the store, which (when 16 is subtracted from  
> it) is -1,141,640,418 as a signed 32-bit number.


Oooops, this is a very silly bug. To write a test to catch the bug...

test/common/testcommon.cpp ... write stream class which gives >  
0x7fffffff bytes of zeros. Then create a PartialReadStream from it,  
asking for > 0x7fffffff, and check to see if the results are as  
expected. With the current code, this should fail.

So on to the fix:

lib/common/PartialReadStream.*

Change

	int mBytesLeft;

to
	
	pos_type mBytesLeft;

then all vars which are related (eg BytesToRead in constructor) to  
pos_type too. (Don't change the args to Read otherwise it won't  
implement the IOStream interface.)

What a stupid mistake to make. Who wrote this code? :-(

Ben


PS: Sorry.