s3sync works from shell but not cron job

[svittal]

Hi,

My s3sync script to back up a folder to S3 works fine when I run it from command prompt, But when I use the same script in the cron job, it fails

Script:

#!/bin/bash
 # script to upload local directory upto s3
 export AWS_ACCESS_KEY_ID=vvv
 export AWS_SECRET_ACCESS_KEY=vvv
 export AWS_CALLING_FORMAT=REGULAR
 export SSL_CERT_DIR=vvv

 date_time=$(date)
 check="1"
 NUM_BUCKETS=$(/usr/bin/ruby s3cmd.rb list hellobucket:$hostname | wc -l)
 NAME=$(hostname)
 if [ $NUM_BUCKETS != $check ]
 then
 echo "$date_time:Bucket exists. Syncing the files!"
 /usr/bin/ruby /root/s3sync/s3sync.rb -r --ssl --delete --debug /var/log/ hellobucket:$NAME
 else
 echo "$date_time : Creating new bucket"
 /usr/bin/ruby s3cmd.rb createbucket hellobucket:$NAME
 echo "$date_time : Syncing the files!"
 /usr/bin/ruby /root/s3sync/s3sync.rb -r --ssl --delete --debug /var/log/ hellobucket:$NAME
 fi
------------

s3sync cron:
*/5 * * * * /root/s3sync/./updatesync.sh >> /root/s3sync.log

[TeckniX]

This unfortunately will not work, simply because when you export your key, you are setting them for a shell environment.
Cronjobs run without any shell environment per-say, which is also why you need to fully qualify the path of your ruby installation.

s3sync will read the the s3config.yml from a few different location:
   $S3CONF/s3config.yml
   $HOME/.s3conf/s3config.yml
   /etc/s3conf/s3config.yml

So created it in one of those locations all readable only by root and the cronjob will work perfectly.


post back if that doesn't solve your problem!

[maelcum]

> This unfortunately will not work, simply because when you export your key, you are setting them for a shell environment.
> Cronjobs run without any shell environment per-say, which is also why you need to fully qualify the path of your ruby installation.
I beg to differ. I am running s3sync-scripts since years directly from crontab in exactly the way svittal suggests. I wouldn't go so far to say that cronjobs have exactly the same environment than normal users, but it is certainly sufficient to set the appropriate environment variables to get s3sync to work. There is no need for any .yml-file (which has been introduced to s3sync at a later development stage, when people already were happily syncing with crontab). And you don't have to use the fully qualified path to the ruby installation either, if the path variable is properly set as well.

That brings us back to svittals problem. Which might be solved by setting the path the way it's done for the user.
Thats how my crontab-environment looks without doing anything to it:
MAILTO=logs@br...com
SHELL=/bin/sh
USER=maelcum
PATH=/usr/bin:/bin
_=/usr/bin/printenv
PWD=/Users/maelcum
HOME=/Users/maelcum
SHLVL=2
LOGNAME=maelcum

This is what it looks like in my scripts. I am using an ruby- and s3sync-installation in the /opt-directory, so have added that to the PATH-variable. Done.

Code:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export SSL_CERT_FILE=/opt/bin/s3sync/ca-certificates.crt
export S3SYNC_NATIVE_CHARSET=UTF-8
export PATH=/opt/bin:/opt/bin/s3sync:/opt/sbin:$PATH

Probably the easiest way to find out whats missing is making a cronjob that does nothing but "printenv >some_filename_where_the_output_should_go". By comparing this output and the results that printenv gives as a logged in user, you would know what to add.

[maelcum]

>  Is there any way to find out where you got the quest from?

Beg you pardon? Which quest?