I ran into this too this morning and quickly fixed it.  Try getting the tarball again and see if it's better for you as well.  I didn't even up the version for this, which is naughty I know.  But I'm supa busy today.

subject says it all

I split your reply and put it in feature requests, because I like your suggestions and want to make sure my stuff follows those recommendations in the future.

This topic has been moved to Feature Requests.

http://s3sync.net/forum/index.php?topic=31.0

OK so.. for the sake of argument, what's the "correct" way to handle this then?

Sure but your scenario still works as long as one of the systems is S3.

Why would you move [platform A] => S3 => [platform B]
why not just rsync between the two platforms?

The point of s3sync is because one of your endpoints is S3... not because you are using S3 as a cache between the two endpoints 

I don't much care about crossing platforms with a particular set of data.  The important thing is that whatever platform you use, the data round trip works as expected.  Once you start juggling the same data from linux, to s3, and then to windows, that's not my problem. 

Hints:
- Learn how S3 REST interface works by reading the amazon documentation on it.
- Get a working knowledge of Ruby with http://www.rubycentral.com/book/index.html
- Look at s3sync.rb and s3cmd.rb code and comments

Your bucket is probably not named "bucket".

I think all my ruby's are 1.8.5.  It's possible that they sped it up?

They're not especially "compatible", as you can tell from the following caveats.  s3cmd can't create the "folder nodes" that s3sync expects, and there's no facility for automatically adding s3sync-like meta-data for permissions and ownership.  It's not a defect, per se.. s3cmd is not intended as a "light weight s3sync" or anything of the kind.  It is strictly meant to cover the low level s3 operations like other s3 "shells".  Things such as listing and creating buckets, poking at individual files, etc.

Let me say that again and clarify: s3cmd is LOW LEVEL.. direct access to the keys and objects on s3.  You can't be thinking about them like directories and files, because they're not, and they don't behave as such.  You can't use s3cmd to "get a sub directory" because there's no such thing.

Having said that, s3cmd should be able to list the keys in your bucket (added by s3sync or anything else) and then you can do a "get" for whatever key you want. 

I would say in general you shouldn't be trying to poke at s3sync'd stuff with s3cmd.. it's just not intended.  It would be more likely, if anything, for me to enhance s3sync to handle single files some day.  The code for that is mostly there already, but the initial conditions setup is brittle, and I am disinclined to kick it around any more than necessary.

Oops I missed that.  I'll answer it over there.

Only if there was a noticeable difference from wget (which is accepted as the control in our tests)

Yes I was testing with your file lfh.

The total time to get it is not of interest.. It just reflects my maximum available bandwidth to my home windows machine here.  In addition to having a cheap connection, I am also running freenet, which is a nearly constant drain on part of my available width.

wget times are mostly within 43-45s
s3cmd are mostly within 46-48s

This is on Windows.  When I profile ruby here, it REALLY sucks the performance down though.. So instead I just clock timed it.