General Category => Questions => Topic started by: triggsley on October 19, 2007, 05:42:40 AM

Title: Error while syncing back
Post by: triggsley on October 19, 2007, 05:42:40 AM
Hi all

Firstly a bit of brown-nosing is in order, what a fantastic little app, without this my life would've been made very difficult.

Ok now thats over, I've a strange bug.

On S3 i've a bucket HDLCode: with several folders nested inside.

  • analyzer
  • analyzer_alliance
  • analyzer_dev
  • ..more folders..
  • prism2
  • prism2_demo
  • prism2_dev
  • ..more folders..

If i run the command :-
./s3sync.rb -r HDLCode:analyzer /mnt/www/html/

I get the following error:-
./s3sync.rb:614:in `initialize': No such file or directory - /mnt/www/html/analyzer/_alliance/LIST.BAT.s3syncTemp (Errno::ENOENT)
        from ./s3sync.rb:614:in `open'
        from ./s3sync.rb:614:in `updateFrom'
        from ./s3sync.rb:378:in `main'
        from ./s3sync.rb:708

Notice the _alliance, there is no folder in this structure see list above, it looks like s3sync is using the singular folder name and then the next folder in the tree list.

This also happens with prism2.

Any ideas on whats going wrong.

Many thanks


Title: Re: Error while syncing back
Post by: ferrix on October 20, 2007, 06:30:22 PM
I'm guessing one of two Classic mistakes.  Just because you have stuff stored/with/slashes/in/key doesn't mean that upon restore all those levels will be created as folders.  s3 has no concept of folders and files, only nodes.  So s3sync saves special nodes as placeholders for folders. 

So if you store in some tool other than s3sync, restore will fail.  Or if you use s3sync to store a bunch of leaf folders but not the intermediate folder nodes, then when you try to restore them all in one go it will fail.

This is because on the local file system those intermediate folders aren't there, and s3sync sees none of its "folder nodes", so folders won't be created during the restore operation.