S3Sync.net

General Category => Announcements => Topic started by: ferrix on June 23, 2008, 09:28:00 PM



Title: Consider how this AWS error may affect your data
Post by: ferrix on June 23, 2008, 09:28:00 PM
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=93408

Due to the fact that s3sync does not (ever, yet) employ the "Content-MD5" header, anyone using s3sync may have stored corrupted data to S3 during the interval discussed above.

Additionally, if you are in the habit of using "--no-md5", as I must do on some of my servers, such corruption will continue to escape notice on future syncs.  This is because there's no other way to check actual node contents vs local.

If possible, the optimal solution would be to run all your local->s3 syncs allowing for md5 comparison (which is the default).  Any corrupt nodes will be detected as changed, and re-synced.