s3sync performance testing: Slow Downloads

[lowflyinghawk]

of course total time *would* be of interest if it didn't match up pretty well with available bandwidth ;-).

[ferrix]

Only if there was a noticeable difference from wget (which is accepted as the control in our tests)

[bdixon]

however brad didn't say what platform he's running on.

I was running this on the Amazon provided Fedora Core 4 image. I hacked it up a little but not tons.


# uname -a
Linux domU-12-31-34-00-02-27 2.6.16-xenU #1 SMP Thu Oct 26 08:41:26 SAST 2006 i686 athlon i386 GNU/Linux
# ruby -v
ruby 1.8.4 (2005-12-24) [i386-linux]

[ferrix]

I think all my ruby's are 1.8.5.  It's possible that they sped it up?

[lowflyinghawk]

mine are 1.8.5 also.  it's possible there were fixes in the http lib, although it is hard to imagine what would affect what is basically a loop writing into a socket.

[tmoney]

Has the download speed issue been solved? I got the following results using s3sync from an EC2 instance. I am using FC6 with ruby version 1.8.5.

22MB file using s3cmd
Upload - EC2 => S3  = 6 MB/s
Download - S3 => EC2  = 1.4 MB/s

22MB file using WGET
Download S3 => EC2 = 11MB/s

I can live with the upload speed of 6MB/s. However, the download speed is pretty slow. Has anyone found a better way to go about this?

[ferrix]

We haven't been able to repro the slowness, so there's nothing "solved" per se.