when syncing an up to date mirror, s3sync is taking about 10 minutes to do what psync/rsync did in < 5 minutes, and uses alot more cpu over the period (somewhat subjective, but it seems to be sitting at 30% the whole time whereas rsync just burst for a short period at the beginning while it built the file list).

i am assuming this is because s3sync can't pull a list of files at the beginning so spends alot more time traversing the directories (in XML rather than ssh no less). is this the case? has anyone else noticed comparably poor results when dealing with alot of files?

(i used to have my whole user directory (~120k files .. crazy, but there we go..) syncing every few hours to strongspace with rsync with no issue, but if its going to take half an hour each time and use 30% cpu while it works then its not really viable (not to mention my cost may be in the get/put/list requests - will see once it balances out).


amendment: rsync does seem to be taking a comparable amount of time, just not using any cpu, and its only about 23k files (not sure where i got the 120k, maybe that was the whole drive)