S3Sync.net

General Category => Questions => Topic started by: nzimas on November 22, 2007, 07:01:45 PM



Title: how to retrieve an entire bucket from another instance?
Post by: nzimas on November 22, 2007, 07:01:45 PM
I have managed to sync a local dir with an s3 bucket on an instance.
Now i want to retrieve the entire bucket from another instance.

Have tried s3sync.rb but, as anticipated, it did not fetch anything.

Is this the right set of tools for such task?

Nuno.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: ferrix on November 24, 2007, 06:13:06 PM
From your post I am confused about "s3 bucket on an instance".  If you tell me the commands you used and what failed I might be able to offer advice.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: nzimas on November 24, 2007, 10:13:48 PM
From an instance is the proper phrasing :)

The scenario is quite simple.
From instance 1 i have synced a dir into a bucket.
Is there any tool i can use to retieve that contents of that bucket from another instance?

Nuno.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: ferrix on November 25, 2007, 08:15:33 AM
One instance of what??


Title: Re: how to retrieve an entire bucket from another instance?
Post by: nzimas on November 25, 2007, 08:20:20 AM
instance = ec2 instance :)

Nuno.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: ferrix on November 26, 2007, 03:51:14 AM
Oh.  Well s3sync doesn't need to know anything about where you are running it.  Just tell it the s3 source path and the target local path like in the README examples.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: nzimas on November 26, 2007, 09:55:22 AM
I did so, but it did not retrieve the contents of the bucket :)


Title: Re: how to retrieve an entire bucket from another instance?
Post by: ferrix on November 26, 2007, 10:17:00 AM
Then go back to what I said above.  Post the commands you are using for both operations.  Also use the -d option so I can see what it is thinking.

You are essentially saying "hi it does not work"; how can we help you without even seeing what you might be doing wrong?


Title: Re: how to retrieve an entire bucket from another instance?
Post by: nzimas on November 26, 2007, 11:03:06 AM
Quote
root@domU-12-31-38-00-6D-94:~/s3sync# ./s3sync.rb -r -d c24h-vBkp20071122:/ /root/data/
s3Prefix /
localPrefix /root/data
s3TreeRecurse c24h-vBkp20071122 /
Creating new connection
Trying command list_bucket c24h-vBkp20071122 max-keys 200 prefix / delimiter / with 100 retries left
Response code: 200
local node object init. Name: Path:/root/data/ Size:38 Tag:d66759af42f282e1ba19144df2d405d0
dest:
Source does not have
localTreeRecurse /root/data
Test /root/data/s3cmd.rb
local item /root/data/s3cmd.rb
local node object init. Name:s3cmd.rb Path:/root/data/s3cmd.rb Size:8829 Tag:ab5eb228472efad865a88b09897864d7
dest: s3cmd.rb
Source does not have s3cmd.rb

This verbose is quite obvious. I am trying to FETCH the contents of a bucket that was synced from another ec2 instance into a new one.
The local folder is, therefore, empty. All i want is to retrieve what's inside that buckent to this new ec2 instance. This simple :)

The question remains. Is s3sync the right tool for the job?

Nuno.


Title: Re: how to retrieve an entire bucket from another instance?
Post by: ferrix on November 26, 2007, 12:32:18 PM
Sure s3sync should work fine.

Can you show me the results of s3cmd.rb list c24h-vBkp20071122
(if it's long then the first page is enough)