S3Sync.net
February 02, 2014, 01:19:57 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: failure to create necessary directories on localhost?  (Read 5974 times)
Luis
Newbie
*
Posts: 2


View Profile
« on: February 17, 2007, 11:35:12 AM »

(I think this is a bug, but not completely sure.)

So I'm trying to get data off s3. The data looks something like:
/data/foo/foo/bar.txt
/data/foo/foo/baz.txt

the directory:
/data/foo/

contains only subdirectories; no files.

when I try to do:
ruby s3sync.rb -r --delete mybucket:/data/ /home/me/data-test

I get:
Could not mkdir /home/me/data-test/foo/foo: No such file or directory - /home/me/data-test/foo/foo

/home/me/data-test/ exists; I think it is failing because data-test/foo/ doesn't exist, and it is trying to create data-test/foo/foo/ without creating data-test/foo/ first.

Is this a bug? Silliness on my end? I'm pretty sure bug, but maybe there is a subtlety/design decision I'm missing here.
« Last Edit: February 19, 2007, 03:58:34 AM by ferrix » Logged
Luis
Newbie
*
Posts: 2


View Profile
« Reply #1 on: February 17, 2007, 11:37:28 AM »

Forgot to add that if I create the local directory /home/me/data-test/foo/  that it works fine. But if I have to manually create every directory which has no files in it, that isn't going to scale very well :/
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #2 on: February 17, 2007, 10:04:28 PM »

Did you *upload* the files using s3sync.rb?  If so, then directory nodes should have been placed which would, upon retreival, do all the correct local dir creation.
Logged
ulbrich
Newbie
*
Posts: 1


View Profile
« Reply #3 on: July 26, 2007, 06:49:36 AM »

I had the same problem of syncing data from s3 to a local filesystem where the data *never* was before: The missing directories are simply not created on the fly.

To fix this insert the following in the require section:

require 'ftools'

And add File.makedirs() as follows:

def updateFrom(fromNode)
    if fromNode.respond_to?(:to_stream)
        fName = @path + '.s3syncTemp'
        File.makedirs(File.dirname(@path)) # This is the important part!

    [...]
end

Perhapes, this could make it to the next release, please?

Best regards, Jan
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #4 on: November 17, 2007, 08:46:23 PM »

Not promiscuously creating directories locally when there are no dir nodes stored @S3 for them is a design decision.  Dow would it know what permissions and owner to use, etc. 
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!