Only syncing first sub-directory?

[yanik]

Hi all,

I'm having this problem where only the first sub-directory of my source is getting synced.

I have a backup server (where my other servers backup) which I want to backup to S3 (so a backup of a backup). The structure is as follows:

Code:

/backups/server1
/backups/server2
/backups/server3

When I run the following:

Code:

ruby s3sync.rb -r -s -v /backups $BUCKET:$PREFIX

Only server1 gest synced. The sync works, ends normally (no error), but all other sub-directory (server2, server3, etc) never get touched. The target folders don't get created, the source folders don't get synced. The sync just stops after the first sub-directory.

Anything I'm missing?

Thanks

[ferrix]

Looks right to me.  Can you create a miniature test case of directories that fail?  If so then tar them up and email to me (contact info in README)

[yanik]

A bit more info...

I played a bit more with S3Sync and I don't think it necessarily stops on the first sub-directory. It'll sync one directory after the other until it hits one it can't sync (for whatever reason) and then it stops. It just happens that on my first try it stopped after the first directory. I pipe the S3Sync output to a log file and I see nothing special. No error message. It just stops.

One thing of interest (that may or may not explain my problem). All my servers backup to a backup server in a directory named backups (how many time can you say backup in one sentence?). Every night I take a snapshot of this directory (cp -al ...) and I (try to) S3Sync that snapshot. Can this be my problem? Would S3Sync stop on some hard links?

Here's the snapshot script I use:

Code:

#!/bin/bash
#Keep a snapshot of the last 30 days

MAX_DAYS=30
BACKUP_PATH="/backups"

rm -rf $BACKUP_PATH.30

for ((X=$MAX_DAYS; X > 1; X--))
do
        mv $BACKUP_PATH.$(($X-1)) $BACKUP_PATH.$X
done

cp -al $BACKUP_PATH/. $BACKUP_PATH.1

[ferrix]

None of that seems like it should be a problem.  There's no way for the code to just exit without some kind of error.. try with the -d option and maybe get some idea of what operation was pending when it "stops"?