Uploads fail - s3sync.rb:556:in `<<': can't convert nil into String (TypeError)

[aupton]

Hello All,

I'm running into a strange problem, When I try to upload any directory on my system I only get a few to partially upload and then it dies out. Listing works and pulling single small test files appear to work.

Any ideas?

Bucket name = test-bucket

[root@NAS scripts]# ./s3sync.rb -r -v --ssl /etc test-bucket:
Create node
Create node RAID-message-sender-en.sh
Create node TZ
Create node busybox.conf
Create node crontabs
Create node crontabs/root
Create node default
Create node default-settings
Create node efstab
Create node exports
./s3sync.rb:556:in `<<': can't convert nil into String (TypeError)
        from ./s3sync.rb:556:in `initialize'
        from ./s3sync.rb:244:in `new'
        from ./s3sync.rb:244:in `localTreeRecurse'
        from ./s3sync.rb:223:in `each'
        from ./s3sync.rb:223:in `localTreeRecurse'
        from ./s3sync.rb:259:in `main'
        from ./thread_generator.rb:79:in `call'
        from ./thread_generator.rb:79:in `initialize'
        from ./thread_generator.rb:76:in `new'
        from ./thread_generator.rb:76:in `initialize'
        from ./s3sync.rb:173:in `new'
        from ./s3sync.rb:173:in `main'
        from ./s3sync.rb:708

[aupton]

This issue seems very similar to what i'm experiencing which is detailed in this post:
http://s3sync.net/forum/index.php?topic=102.0

I'm new to s3sync, but believe i have it setup properly as it can communicate with my s3 account and upload/download files, but with random failures.

I've created a bucket through s3cmd and still have the same problem. To the best of my knowledge and according to the docs the default behavior is to create a bucket in the united states, so when I createbucket and then try to upload to that bucket it should work, instead only a few files make it up and then it dies out.

Any ideas or suggestions would be greatly appreciated.

Thanks!

[ferrix]

This happens when calculating the md5 (but the same kind of failure could happen during a transfer)
The stream says it has data left, but then when we try to read it.. failure.

Best guess is the item it's failing on is either a non-regular file (a device, socket, term, pipe, etc.) or it is being modified by some external program while the sync is trying to read it.

Via excludes and other precautions you need to be responsible for making sure the things you ask it to do are well-defined and possible.

It is certainly the case that the software could have some softer padding around these problems.  I will try to improve the error message for this case.