S3Sync Creating Extraneous 38 Byte File

[jeremylemaire]

I am using the following script to sync a local Linux ext3 directory with S3:

s3sync.rb -r --progress --ssl --delete /home/lucid/luciddb/archive mybucket:lucid-backup

For each directory in the structure I am seeing a 38 byte file with the same name as the directory name.  This is taking up a substantial amount of S3 space and more importantly is causing problems when trying to use EC2 MapReduce Hive scripts to create external tables.

• Why are these extraneous files being uploaded?
• Is there a way to prevent S3Sync from uploading these extraneous files?

[jeremylemaire]

After reading this post http://s3sync.net/forum/index.php?topic=39.msg143#msg143 and tinkering a bit I have come to the conclusion that this 38 byte file is meta data used by s3sync to mimic the hierarchal directory structure being uploaded to S3.

Unless I missed it, I think this should be added to the README.txt in the s3sync install.


 

[Admin]

To just upload items with no extra data needed for syncing, you could use the s3cmd.rb in an external script I guess.

It's not a good solution, but then again s3sync is really outdated too.  I have not had time to rewrite it yet, and no one else volunteers