Restore failing - how to skip errors?

[Rinchen]

Hi,

My NAS died.  Luckily I had the content backed up to S3 using s3sync.

I'm now trying to restore but I'm having lots of problems.  After several hours of running, I end up getting odd errors (can't create directory, bad characters in file names, etc) and then s3sync exists.  The behaviour I want is for s3sync to continue it's mission even when presented with an error (because right now my data is stuck in S3 and I can't get it out!).   Some of the errors might be introduced because I have to use CIFS to mount the NAS but frankly I don't care.  I need as much data as possible restored and if I can't get some dot file 10 levels deep it doesn't matter at this point. 

I'm not a ruby expert.  I'm hoping someone here can tell me how to modify the code so when there is an error, it prints it out and keeps going.

Thanks,

Rin

[ferrix]

Most of the existing error handling is in s3try.rb (not that it's necessarily where you need error handling added).  Look for blocks like

Code:

begin
   ....
rescue Some::ErrorType => e
   ...
end

If you find where you are erroring out, and what error type(s) you have, you can wrap it in begin/rescue/end

What happened to your NAS?  HD bork?  I recommend spinrite for drive health (not affiliated, just a satisfied customer)
http://www.grc.com/sr/spinrite.htm
Depending on the problem, you might even get to recover the old drive.

[Rinchen]

Most of the existing error handling is in s3try.rb (not that it's necessarily where you need error handling added). 

If you find where you are erroring out, and what error type(s) you have, you can wrap it in begin/rescue/end

Ok, I'll give that a try. I have a feeling this won't be easy.  :-(

What happened to your NAS?  HD bork?

I'm not exactly sure and Lacie didn't tell me.  I think the power supply failed, spiked, and as a result I had a big, smokey, headcrash on every bloody disk in the raid. Complete loss.  Most of the data was backed up elsewhere but there are a few things which only S3 has at the moment which I was going to backup elsewhere the exact DAY the darn thing died (well, I was traveling and the day I got back is the day this all happened). The NAS was still under warranty so I didn't think something like this would happen so soon and thus didn't make the data redundancy exercise a super high priority.

[ferrix]

If you paste in the error out messages I can advise you about changes.

[Rinchen]

Code:

root@NOAA:~# ./s3restore.sh
EOF error: end of file reached
99 retries left, sleeping for 30 seconds
s3sync.rb:638:in `initialize': No such file or directory - /media/amphora-music/amphora/amphora-joey-work/joey/.googleearth/Registry/google/gecommonsettings/User/layers/r??sultats.s3syncTemp (Errno::ENOENT)
        from s3sync.rb:638:in `open'
        from s3sync.rb:638:in `updateFrom'
        from s3sync.rb:376:in `main'
        from s3sync.rb:735
root@NOAA:~#

Issue with my backup script not using different buckets causes a long url which I'll sort out later.  On a positive note, my NAS has been up for 2 days and it's now dead again. :-(  I think I need to upgrade. :-)

[ferrix]

OK so try near line 376 of s3sync.rb

Code:

begin
   dNode.updateFrom(sourceNode) unless $S3syncOptions['--dryrun']
rescue Errno::ENOENT => e
   $stderr.puts "Skipped item: #{e}"
end

As you can see it's just wrapping the existing line with a begin/rescue/end that matches the error type thrown.  The reason I put it out at the comparator instead of *right* where the error occurs at like 638 is that we need to catch at a place where we're able to easily move on to the next item.

[Rinchen]

Awesome, thanks. I'll give this a try when the RMA get's back.

[Rinchen]

so far, so good.  It's been running for a day now with that patch and is working as expected.

[Rinchen]

So, cleaning out my bucket a little to make the restore complete... I ran into another error...


joey@NOAA:~$ s3sync/s3cmd.rb deleteall s3syncbucket:amphora/amphora-joey

...skip a bazillion lines...
Ran out of retries; operations did not complete!
Ran out of retries; operations did not complete!
Ran out of retries; operations did not complete!
s3sync/s3cmd.rb:110:in `s3cmdMain': undefined method `entries' for nil:NilClass (NoMethodError)
        from s3sync/s3cmd.rb:245

[Rinchen]

I should state that doing a dryrun produces the correct output but when I run it without that, it goes into the retry cycle for some reason.  The box is on a static ip so.....

[Rinchen]

forgot -s ....