Ticket #2 (assigned defect)

Opened 2 years ago

Last modified 7 months ago

When downloading a large file (>0x7fffffff) from server, connection is aborted

Reported by: ben Assigned to: chris (accepted)
Priority: critical Milestone: 0.11
Component: box libraries Version: 0.10
Keywords: windows unix large file BackupStore BadBackupStoreFile 4/11 Cc: dave@logical-progress.com

Description (Last modified by ben)

PartialReadStream? is used to grab a portion of the incoming data. However, it uses signed ints where it shouldn't, and it all goes horribly wrong.

To fix:

test/common/testcommon.cpp ... write stream class which gives > 0x7fffffff bytes of zeros. Then create a PartialReadStream? from it, asking for > 0x7fffffff, and check to see if the results are as expected. With the current code, this should fail.

lib/common/PartialReadStream.*

Change

int mBytesLeft;

to

pos_type mBytesLeft;

then all vars which are related (eg BytesToRead? in constructor) to pos_type too. (Don't change the args to Read otherwise it won't implement the IOStream interface.)

Change History

11/08/2006 18:53:43 changed by ben

  • owner deleted.
  • priority changed from major to critical.
  • component changed from bbackupd to box libraries.
  • description changed.
  • milestone set to 0.11.

28/04/2007 18:18:08 changed by chris

  • owner set to chris.
  • status changed from new to assigned.

Should be fixed by [1585], [1586], [1587] in chris/merge, needs to be merged to trunk.

28/04/2007 20:57:25 changed by chris

(In [1598]) Fix getting files with uncertain size (over 2GB) from the store. Failure to drain the stream will leave the EOF byte in it, which breaks further communications with the store over the same connection. (refs #2, refs #3)

28/04/2007 21:42:45 changed by chris

(In [1599]) Fix bbackupd choosing an invalid (too large) block size for large files (over 2GB) which will cause compare to fail: when rBlockSizeOut == BACKUP_FILE_MAX_BLOCK_SIZE we would have proceeded around the loop one more time and doubled the block size again. (refs #2, refs #3)

29/04/2007 22:31:15 changed by chris

(In [1623]) Read any remaining data from the encoded stream (such as EOF marker) before discarding it, to ensure that we don't break the protocol. (refs #2, refs #3)

25/08/2007 17:26:23 changed by chris

  • keywords set to win32 unix large file BackupStore BadBackupStoreFile 4/11.

25/08/2007 17:27:01 changed by chris

  • keywords changed from win32 unix large file BackupStore BadBackupStoreFile 4/11 to windows unix large file BackupStore BadBackupStoreFile 4/11.

25/09/2007 00:24:35 changed by chris

  • cc set to Dave, Bamford, <dave@logical-progress.com>.

Dave Bamford reports that this is still not fixed on Windows:

Date: Mon, 24 Sep 2007 22:23:19 +0100

Is the backup/restore of large files over 2G fixed? from http://bbdev.fluffy.co.uk/trac/wiki/WindowsClientReleases Build 1662 says its fixed. But its still listed under known bugs for the windows client.

I am having trouble backing up a 2.5G file. Its reporting 00001 blocks on the server and a "get" restores a file of 0 bytes. However the event log reports it being backed up (no issues).

I am using release 1857 in http://bbdev.fluffy.co.uk/svn/box/chris/win32/releases/ as the client on a windows 2003 server to a Debian etch server running the vanilla 0.10 release.

25/09/2007 07:13:03 changed by jamesog

  • cc changed from Dave, Bamford, <dave@logical-progress.com> to dave@logical-progress.com.

Correcting CC: field (email addresses only).

12/01/2008 18:39:35 changed by chris

Hi Dave,

You need to upgrade the server as well to fix this problem. There are bugs on both sides. You also need to delete any old copies of the file on the server.

Cheers, Chris.