S3Sync.net

General Category => Questions => Topic started by: redox on June 15, 2007, 04:25:19 AM



Title: Problems using SSL
Post by: redox on June 15, 2007, 04:25:19 AM
Hi,
I am having some troubles communicating with aws s3 over SSL. Both s3sync and s3cmd works fine, but when i try to use ssl (-s option) they crash with different errors.
I have downloaded the root certificate shar example from the manual and placed them in the /usr/local/certs

Here are two test runs:


$ export AWS_ACCESS_KEY_ID=XXX
$ export AWS_SECRET_ACCESS_KEY=XXX
$ export SSL_CERT_DIR=/usr/local/certs

$ ruby s3cmd.rb -n -v -d list mybucket:backup
list mybucket:backup   {}
--------------------
nil
Trying command list_bucket mybucket max-keys 100 prefix backup  with 100 retries left
Response code: 200
backup/var/www/dev/dev
backup/var/www/dev/dev/rails.pdf

$ ruby s3cmd.rb -s -n -v -d list mybucket:backup
list mybucket:backup   {}
--------------------
nil
Trying command list_bucket mybucket max-keys 100 prefix backup  with 100 retries left
Null stream error:
s3cmd.rb:119:in `s3cmdMain': undefined method `entries' for nil:NilClass (NoMethodError)
        from s3cmd.rb:207

As you can see, the first run without ssl is fine, the second one crashes. Have you any idea what might be the problem?

Best regards,
Redox


Title: Re: Problems using SSL
Post by: ferrix on June 16, 2007, 06:44:48 AM
Your ruby version is too old; see README


Title: Re: Problems using SSL
Post by: redox on June 19, 2007, 04:56:54 AM
Your ruby version is too old; see README
Thanks for the tip, but it doesn't seem like the ruby version is too old, it is 1.8.4
which is the requirement in the README for s3sync.rb

$ cat README.txt
--- SNIP ---
Prerequisites:
--------------
You need a functioning Ruby (>=1.8.4) installation, as well as the OpenSSL ruby
library (which may or may not come with your ruby).
--- END SNIP ---

$ ruby --version
ruby 1.8.4 (2005-12-24) [i486-linux]

It seems more like i could need som help on how to get openssl support compiled into a fresh ruby installation
on a Debian system as the debian package openssl-ruby doesn't help (maybe too old, who knows)

Best,
Redox







Title: Re: Problems using SSL
Post by: ferrix on June 19, 2007, 07:22:09 AM
All my boxes are debians and it seems fine.  Though you need something newer than sarge.


Title: Re: Problems using SSL
Post by: redox on June 20, 2007, 08:49:34 AM
Yeah, i had sarge and upgraded to etch...
It's all good now.

Thanks.


Title: Re: Problems using SSL
Post by: seengee on November 19, 2007, 11:03:06 AM
i am also having problems running s3sync on SSL (ruby 1.8.6 (2007-11-18 patchlevel 5000) [i686-linux] on CentOS), i get this error:

./S3_s3sync_mod.rb:109:in `verify_mode=': undefined method `verify_mode=' for nil:NilClass (NoMethodError)
        from ./s3try.rb:46:in `s3trySetup'
        from s3sync.rb:108:in `main'
        from s3sync.rb:711


when running this code:

#!/bin/bash
# script to upload local directory upto s3
cd /path/s3sync/s3sync/
export AWS_ACCESS_KEY_ID=111111111111111111111
export AWS_SECRET_ACCESS_KEY=KEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEY
#export SSL_CERT_DIR=/path/s3sync/certs
export SSL_CERT_FILE=/path/s3sync/certs/certfile
ruby s3sync.rb -r --ssl  /path/s3sync/localuploadfolder/ Jasons:pre/etcbackup
# copy and modify line above for each additional folder to be synced


it works when i remove the --SSL option though ?


Title: Re: Problems using SSL
Post by: maelcum on November 19, 2007, 01:19:29 PM
#export SSL_CERT_DIR=/path/s3sync/certs
export SSL_CERT_FILE=/path/s3sync/certs/certfile

Have you tried exporting the path only (not the certificate)? Just the path pointing to it.
That works for me. Exporting the certificate does not (though I have not checked why, yet. Certainly my fault.)

Addition on 24. Nov.:
It doesn't work for me either since I've switched from 1.1.4 to 1.2.2...
 :'(


Title: Re: Problems using SSL
Post by: seengee on November 19, 2007, 05:16:56 PM
yeah i tried that, when that didnt work i then moved onto using the SSL_CERT_FILE option, can you see any reason why the SSL_CERT_DIR option wouldnt work compared to your own usage ?


Title: Re: Problems using SSL
Post by: maelcum on November 20, 2007, 03:14:40 AM
No, not really. I'm using a lower version of ruby (1.8.4) but I'll try your version tonight (European Time, that is).

Could it be your certificate? Did you use your own, or followed the procedures floating around here?


Title: Re: Problems using SSL
Post by: seengee on November 20, 2007, 04:37:13 AM
i tried using the certificate code from the readme.txt and named that certfile, i also exported all the certificates from the ssl.certs.shar file into the certs folder although admittedly i'm not 100% sure i did it right in either case, not sure how to check though tbh.


Title: Re: Problems using SSL
Post by: maelcum on November 20, 2007, 03:32:56 PM
Did you run the shar-file, or just downloaded it?

You'd need to:
> wget http://mirbsd.mirsolutions.de/cvs.cgi/~checkout~/src/etc/ssl.certs.shar
and then:
> sh ssl.certs.shar

If you are running unter any Un*x-OS, that would be.



Title: Re: Problems using SSL
Post by: seengee on November 21, 2007, 06:51:14 AM
yep, i did that.

have tried pointing to both the folder and to individual files within and still get exactly the same error.

can anyone shed any light on what this error message actually means ?


Title: Re: Problems using SSL
Post by: seengee on November 21, 2007, 07:58:32 AM
i have now checked the validity of the certificate using http://prefetch.net/articles/checkcertificate.html and the certificate is valid. Is this a case of ruby not trusting the certificate or something like that ?


Title: Re: Problems using SSL
Post by: berlin on November 22, 2007, 10:54:33 PM
I'm seeing the same (?) issue on Centos5, ruby 1.8.5 (2006-08-25) [x86_64-linux].

./S3_s3sync_mod.rb:119:in `verify_mode=': undefined method `verify_mode=' for nil:NilClass (NoMethodError)
        from ./s3try.rb:49:in `s3trySetup'
        from s3sync.rb:102:in `main'
        from s3sync.rb:721


I fiddled with SSL_CERT_DIR and SSL_CERT_FILE to no avail.


Title: Re: Problems using SSL
Post by: maelcum on November 24, 2007, 03:49:41 PM
And now it hit me, too.
Switching from 1.1.4 to 1.2.2 broke the scripts I've been using without chance for month'.

Just for sakes:
I have done nothing but changed the .rb-files in the directories that once held the 1.1.4-version. Everything else was kept as it worked before (scripts that call s3sync.rb, certificates, everything).

Because it's no big deal, I'll revert back to 1.1.4 for the moment; there's a lot to sync tonight, even on the backup machine I'm testing this on. But I'll be happy to try out any idea. Anybody?

Cheers

maelcum

PS: berlin, seengee: You might want to check out the older version (available on the front page of www.s3sync.net) - it works for me - again, now that I've reverted.


Title: Re: Problems using SSL
Post by: ferrix on November 24, 2007, 06:13:46 PM
I'll check this out.


Title: Re: Problems using SSL
Post by: ferrix on November 24, 2007, 06:35:07 PM
Try new ver now.
http://s3sync.net/forum/index.php?topic=117.msg506#msg506


Title: Re: No Problems using SSL anymore with 1.2.3
Post by: maelcum on November 25, 2007, 04:53:16 AM
Per.fekt!

Just synced about 2.500 objects, old and new ones, without a hitch. In a US bucket, that is; I'm not yet using the European ones (and probably will only for sensitive data, not the bulk of things - because of the pricing).

Thank you, ferrix. Thank you very much.

Cheers

maelcum