Excellent work, it works for me.
But there seems to be a small typo in README.txt: The environment variable to choose the calling format -needed for european buckets- seems to be AWS_CALLING_FORMAT instead of S3SYNC_CALLING_FORMAT.

Greets,
Clou

@Clou: You're mentioning an interesting point: There are US and EU Buckets. But for now (as far as I could see) there are only EC2-Instances for US I guess. So when combining EC2/S3 for now it would only make sense to have US-Only EC2/S3-Combinations unless one want to give EU-Customers direct access to S3-Buckets via Link, right?

As I understand it that's correct. It also has to be considered that traffic is charged between EC2/S3-EU but not between EC2/S3-US, but we're getting off-topic ;-)

Just 5 minutes into trying s3sync I stumbled at exactly this problem and as far as I understand the fix is quite simple.
The only thing you need to do is change your URL scheme

from http://s3.amazonaws.com/<bucketname>
to http://<bucketname>.s3.amazonaws.com/

and DNS will do the rest... it's transparent, it's easy. As far as I understand.

I would change it myself if I knew ruby and where you construct your URL in the code.

Unfortunately it's not that simple: You could use the changed URL scheme, and it would work for all EU buckets, and for most US buckets. But in US buckets uppercase letters are allowed, and those buckets can not be addressed as virtual hosts... I dont' know if there are other characters that can be used in US buckets which cannot be used in EU buckets. But that will have to be considered.


I'm also not very firm in ruby and don't know how to code it, so i'm looking forward to see when ferrix has had time to look into it.

The Link is
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/

Don't mind the date in the URL, which means that the API version has not changed. Obviously when the new functionality was implemented it was not neccessary to change the API itself...
Have a look at the "What's new" section...

The API Version is still from 2006, but the developer guide was last updated Nov-5. There you can find information on the location constraints.

HTH
Clou

No fix is available yet. AFAIK.

Greets,
Clou

The API Docs were last updated Nov-5. The relevant section seems to be "Location Selection" where it is mentioned:

====
To access Amazon S3 buckets and objects that were created using CreateBucketConfiguration, you must use the virtual hosted-style request. For example:
http://yourbucket.s3.amazonaws.com/yourobject
You cannot use the path-style request:
http://s3.amazonaws.com/yourbucket/yourobject
If you use the path-style request, you receive a permanent redirect.
====

I think I got behind what might be the problem: The permanent redirect is to <bucketname>.s3.amazonaws.com, and so the bucketname is encoded into the hostname.
So the request is made to
<bucketname>.s3.amazonaws.com/<bucketname>
which is wrong and should be
<bucketname>.s3.amazonaws.com/

(just a wild guess...)

I don't have one of those buckets but I'm guessing that the server name might be different?  You can set that via the env variable AWS_S3_HOST (defaults to s3.amazonaws.com)

I can confirm that. Debug output shows that a request is expected to <bucketname>.s3.amazonaws.com. Setting the env-variable to that hostname leads to another error:

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>...

So there might be something wrong with request signing.

@maelcum: The buckets don't look different when viewed via listbuckets...

I created a "european" bucket with S3Fox, then tried to fill it with s3sync.rb.
Filling failed with:



./s3sync.rb --verbose /var/archives/ testeu:test
S3 command failed:
list_bucket testeu max-keys 200 prefix test delimiter /
With result 301 Moved Permanently
S3 ERROR: #<Net::HTTPMovedPermanently:0xb7ad27a0>
./s3sync.rb:292:in `+': can't convert nil into Array (TypeError)
        from ./s3sync.rb:292:in `s3TreeRecurse'
        from ./s3sync.rb:348:in `main'
        from ./thread_generator.rb:79:in `call'
        from ./thread_generator.rb:79:in `initialize'
        from ./thread_generator.rb:76:in `new'
        from ./thread_generator.rb:76:in `initialize'
        from ./s3sync.rb:269:in `new'
        from ./s3sync.rb:269:in `main'
        from ./s3sync.rb:708