S3Sync.net
February 02, 2014, 01:20:24 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: Restore failing - how to skip errors?  (Read 9587 times)
Rinchen
Newbie
*
Posts: 8


View Profile
« on: August 10, 2008, 11:41:59 PM »

Hi,

My NAS died.  Luckily I had the content backed up to S3 using s3sync.

I'm now trying to restore but I'm having lots of problems.  After several hours of running, I end up getting odd errors (can't create directory, bad characters in file names, etc) and then s3sync exists.  The behaviour I want is for s3sync to continue it's mission even when presented with an error (because right now my data is stuck in S3 and I can't get it out!).   Some of the errors might be introduced because I have to use CIFS to mount the NAS but frankly I don't care.  I need as much data as possible restored and if I can't get some dot file 10 levels deep it doesn't matter at this point. 

I'm not a ruby expert.  I'm hoping someone here can tell me how to modify the code so when there is an error, it prints it out and keeps going.

Thanks,

Rin
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #1 on: August 11, 2008, 03:24:33 PM »

Most of the existing error handling is in s3try.rb (not that it's necessarily where you need error handling added).  Look for blocks like

Code:
begin
   ....
rescue Some::ErrorType => e
   ...
end

If you find where you are erroring out, and what error type(s) you have, you can wrap it in begin/rescue/end

What happened to your NAS?  HD bork?  I recommend spinrite for drive health (not affiliated, just a satisfied customer)
http://www.grc.com/sr/spinrite.htm
Depending on the problem, you might even get to recover the old drive.
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #2 on: August 11, 2008, 04:25:08 PM »

Most of the existing error handling is in s3try.rb (not that it's necessarily where you need error handling added). 

If you find where you are erroring out, and what error type(s) you have, you can wrap it in begin/rescue/end

Ok, I'll give that a try. I have a feeling this won't be easy.  :-(

What happened to your NAS?  HD bork?

I'm not exactly sure and Lacie didn't tell me.  I think the power supply failed, spiked, and as a result I had a big, smokey, headcrash on every bloody disk in the raid. Complete loss.  Most of the data was backed up elsewhere but there are a few things which only S3 has at the moment which I was going to backup elsewhere the exact DAY the darn thing died (well, I was traveling and the day I got back is the day this all happened). The NAS was still under warranty so I didn't think something like this would happen so soon and thus didn't make the data redundancy exercise a super high priority.
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #3 on: August 11, 2008, 04:56:57 PM »

If you paste in the error out messages I can advise you about changes.
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #4 on: August 12, 2008, 11:04:30 AM »

Code:
root@NOAA:~# ./s3restore.sh
EOF error: end of file reached
99 retries left, sleeping for 30 seconds
s3sync.rb:638:in `initialize': No such file or directory - /media/amphora-music/amphora/amphora-joey-work/joey/.googleearth/Registry/google/gecommonsettings/User/layers/r??sultats.s3syncTemp (Errno::ENOENT)
        from s3sync.rb:638:in `open'
        from s3sync.rb:638:in `updateFrom'
        from s3sync.rb:376:in `main'
        from s3sync.rb:735
root@NOAA:~#

Issue with my backup script not using different buckets causes a long url which I'll sort out later.  On a positive note, my NAS has been up for 2 days and it's now dead again. :-(  I think I need to upgrade. :-)
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #5 on: August 13, 2008, 08:27:09 PM »

OK so try near line 376 of s3sync.rb
Code:
begin
   dNode.updateFrom(sourceNode) unless $S3syncOptions['--dryrun']
rescue Errno::ENOENT => e
   $stderr.puts "Skipped item: #{e}"
end

As you can see it's just wrapping the existing line with a begin/rescue/end that matches the error type thrown.  The reason I put it out at the comparator instead of *right* where the error occurs at like 638 is that we need to catch at a place where we're able to easily move on to the next item.
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #6 on: August 14, 2008, 06:37:52 PM »

Awesome, thanks. I'll give this a try when the RMA get's back.
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #7 on: September 23, 2008, 05:32:32 PM »

so far, so good.  It's been running for a day now with that patch and is working as expected.
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #8 on: September 25, 2008, 04:55:48 PM »

So, cleaning out my bucket a little to make the restore complete... I ran into another error...


joey@NOAA:~$ s3sync/s3cmd.rb deleteall s3syncbucket:amphora/amphora-joey

...skip a bazillion lines...
Ran out of retries; operations did not complete!
Ran out of retries; operations did not complete!
Ran out of retries; operations did not complete!
s3sync/s3cmd.rb:110:in `s3cmdMain': undefined method `entries' for nil:NilClass (NoMethodError)
        from s3sync/s3cmd.rb:245

Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #9 on: September 25, 2008, 05:04:49 PM »

I should state that doing a dryrun produces the correct output but when I run it without that, it goes into the retry cycle for some reason.  The box is on a static ip so..... Huh
Logged
Rinchen
Newbie
*
Posts: 8


View Profile
« Reply #10 on: September 25, 2008, 05:16:47 PM »

forgot -s ....
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!