S3Sync.net
February 02, 2014, 01:19:46 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: Transfers randomly dying  (Read 4432 times)
ankaerith
Newbie
*
Posts: 3


View Profile
« on: May 19, 2008, 05:18:04 PM »

I've had a consistent issue with transfers causing s3sync to spit the follow error:

Code:
./s3sync.rb -r --progress --ssl --verbose /whatever/Pictures/ Stuff:/backup/Pictures

[sniped lots of create nodes]

Create node Canon/MVI_0114.AVI
Progress: 5635072b  60964b/s  4%       ./HTTPStreaming.rb:71:in `floor': NaN (FloatDomainError)
        from ./HTTPStreaming.rb:71:in `read'
        from /usr/lib/ruby/1.8/net/http.rb:1556:in `send_request_with_body_stream'
        from /usr/lib/ruby/1.8/net/http.rb:1527:in `exec'
        from /usr/lib/ruby/1.8/net/http.rb:1048:in `_HTTPStreaming_request'
        from ./HTTPStreaming.rb:43:in `request'
        from ./S3_s3sync_mod.rb:88:in `make_request'
        from ./S3.rb:172:in `put'
        from ./s3try.rb:81:in `send'
        from ./s3try.rb:81:in `S3try'
        from ./s3sync.rb:510:in `updateFrom'
        from ./s3sync.rb:375:in `main'
        from ./s3sync.rb:724

It seems to happen with no rhyme or reason.   

A friend provided a quick workaround for me with a patched HTTPStreaming.rb:

Code:
module S3sync
        class ProgressStream < SimpleDelegator
                def initialize(s, size=0)
                        @start = @last = Time.new
                        @total = size
                        @transferred = 0
                        @closed = false
                        @printed = false
                        @innerStream = s
                        super(@innerStream)
                        __setobj__(@innerStream)
                end
                # need to catch reads and writes so we can count what's being transferred
                def read(i)
                        res = @innerStream.read(i)
                        @transferred += res.respond_to?(:length) ? res.length : 0
                        now = Time.new
                        if(now - @last > 1) # don't do this oftener than once per second
                                @printed = true
                begin
                                $stdout.printf("\rProgress: %db  %db/s  %s       ", @transferred, (@transferred/(now - @start)).floor,
                                        @total > 0? (100 * @transferred/@total).floor.to_s + "%" : ""
                                )
                rescue FloatDomainError
                    print "Caught a FloatDomainError -- it's purely cosmetic";
                end
                                $stdout.flush
                                @last = now
                        end
                        res
                end
                def write(s)
                        @transferred += s.length
                        res = @innerStream.write(s)
                        now = Time.new
                        if(now -@last > 1) # don't do this oftener than once per second
                                @printed = true
                                $stdout.printf("\rProgress: %db  %db/s  %s       ", @transferred, (@transferred/(now - @start)).floor,
                                        @total > 0? (100 * @transferred/@total).floor.to_s + "%" : ""
                                )
                                $stdout.flush
                                @last = now
                        end
                        res
                end
                def rewind()
                        @transferred = 0
                        @innerStream.rewind if @innerStream.respond_to?(:rewind)
                end
                def close()
                        $stdout.printf("\n") if @printed and not @closed
                        @closed = true
                        @innerStream.close
                end
        end
end #module
Logged
ankaerith
Newbie
*
Posts: 3


View Profile
« Reply #1 on: May 19, 2008, 10:26:01 PM »

Erg, the above print command causes an error; switched to this:

$stdout.printf "Caught a FloatDomainError -- it's purely cosmetic";
Logged
ferrix
Sr. Member
****
Posts: 363


(I am greg13070 on AWS forum)


View Profile
« Reply #2 on: May 21, 2008, 06:01:36 PM »

I don't get what is causing that error here.. But I'll catch it starting in the next ver.
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!