Title: Transfers randomly dying Post by: ankaerith on May 19, 2008, 05:18:04 PM I've had a consistent issue with transfers causing s3sync to spit the follow error:
Code: ./s3sync.rb -r --progress --ssl --verbose /whatever/Pictures/ Stuff:/backup/Pictures [sniped lots of create nodes] Create node Canon/MVI_0114.AVI Progress: 5635072b 60964b/s 4% ./HTTPStreaming.rb:71:in `floor': NaN (FloatDomainError) from ./HTTPStreaming.rb:71:in `read' from /usr/lib/ruby/1.8/net/http.rb:1556:in `send_request_with_body_stream' from /usr/lib/ruby/1.8/net/http.rb:1527:in `exec' from /usr/lib/ruby/1.8/net/http.rb:1048:in `_HTTPStreaming_request' from ./HTTPStreaming.rb:43:in `request' from ./S3_s3sync_mod.rb:88:in `make_request' from ./S3.rb:172:in `put' from ./s3try.rb:81:in `send' from ./s3try.rb:81:in `S3try' from ./s3sync.rb:510:in `updateFrom' from ./s3sync.rb:375:in `main' from ./s3sync.rb:724 It seems to happen with no rhyme or reason. A friend provided a quick workaround for me with a patched HTTPStreaming.rb: Code: module S3sync class ProgressStream < SimpleDelegator def initialize(s, size=0) @start = @last = Time.new @total = size @transferred = 0 @closed = false @printed = false @innerStream = s super(@innerStream) __setobj__(@innerStream) end # need to catch reads and writes so we can count what's being transferred def read(i) res = @innerStream.read(i) @transferred += res.respond_to?(:length) ? res.length : 0 now = Time.new if(now - @last > 1) # don't do this oftener than once per second @printed = true begin $stdout.printf("\rProgress: %db %db/s %s ", @transferred, (@transferred/(now - @start)).floor, @total > 0? (100 * @transferred/@total).floor.to_s + "%" : "" ) rescue FloatDomainError print "Caught a FloatDomainError -- it's purely cosmetic"; end $stdout.flush @last = now end res end def write(s) @transferred += s.length res = @innerStream.write(s) now = Time.new if(now -@last > 1) # don't do this oftener than once per second @printed = true $stdout.printf("\rProgress: %db %db/s %s ", @transferred, (@transferred/(now - @start)).floor, @total > 0? (100 * @transferred/@total).floor.to_s + "%" : "" ) $stdout.flush @last = now end res end def rewind() @transferred = 0 @innerStream.rewind if @innerStream.respond_to?(:rewind) end def close() $stdout.printf("\n") if @printed and not @closed @closed = true @innerStream.close end end end #module Title: Re: Transfers randomly dying Post by: ankaerith on May 19, 2008, 10:26:01 PM Erg, the above print command causes an error; switched to this:
$stdout.printf "Caught a FloatDomainError -- it's purely cosmetic"; Title: Re: Transfers randomly dying Post by: ferrix on May 21, 2008, 06:01:36 PM I don't get what is causing that error here.. But I'll catch it starting in the next ver.
|