S3Sync.net

General Category => Report Bugs => Topic started by: aupton on November 10, 2007, 01:29:18 AM



Title: Uploads fail - s3sync.rb:556:in `<<': can't convert nil into String (TypeError)
Post by: aupton on November 10, 2007, 01:29:18 AM
Hello All,

I'm running into a strange problem, When I try to upload any directory on my system I only get a few to partially upload and then it dies out. Listing works and pulling single small test files appear to work.

Any ideas?

Bucket name = test-bucket

[root@NAS scripts]# ./s3sync.rb -r -v --ssl /etc test-bucket:
Create node
Create node RAID-message-sender-en.sh
Create node TZ
Create node busybox.conf
Create node crontabs
Create node crontabs/root
Create node default
Create node default-settings
Create node efstab
Create node exports
./s3sync.rb:556:in `<<': can't convert nil into String (TypeError)
        from ./s3sync.rb:556:in `initialize'
        from ./s3sync.rb:244:in `new'
        from ./s3sync.rb:244:in `localTreeRecurse'
        from ./s3sync.rb:223:in `each'
        from ./s3sync.rb:223:in `localTreeRecurse'
        from ./s3sync.rb:259:in `main'
        from ./thread_generator.rb:79:in `call'
        from ./thread_generator.rb:79:in `initialize'
        from ./thread_generator.rb:76:in `new'
        from ./thread_generator.rb:76:in `initialize'
        from ./s3sync.rb:173:in `new'
        from ./s3sync.rb:173:in `main'
        from ./s3sync.rb:708


Title: Re: How to choose storage location in Amazon S3
Post by: aupton on November 12, 2007, 11:56:34 AM
This issue seems very similar to what i'm experiencing which is detailed in this post:
http://s3sync.net/forum/index.php?topic=102.0

I'm new to s3sync, but believe i have it setup properly as it can communicate with my s3 account and upload/download files, but with random failures.

I've created a bucket through s3cmd and still have the same problem. To the best of my knowledge and according to the docs the default behavior is to create a bucket in the united states, so when I createbucket and then try to upload to that bucket it should work, instead only a few files make it up and then it dies out.

Any ideas or suggestions would be greatly appreciated.

Thanks!



Title: Re: Uploads fail - s3sync.rb:556:in `<<': can't convert nil into String (TypeErr
Post by: ferrix on November 17, 2007, 08:56:28 PM
This happens when calculating the md5 (but the same kind of failure could happen during a transfer)
The stream says it has data left, but then when we try to read it.. failure.

Best guess is the item it's failing on is either a non-regular file (a device, socket, term, pipe, etc.) or it is being modified by some external program while the sync is trying to read it.

Via excludes and other precautions you need to be responsible for making sure the things you ask it to do are well-defined and possible.

It is certainly the case that the software could have some softer padding around these problems.  I will try to improve the error message for this case.