|
Title: Version 1.2.5 is released Post by: ferrix on May 11, 2008, 02:38:40 AM Added option --no-md5
About MD5 hashes ---------------- s3sync's normal operation is to compare the file size and MD5 hash of each item to decide whether it needs syncing. On the S3 side, these hashes are stored and returned to us as the "ETag" of each item when the bucket is listed, so it's very easy. On the local side, the MD5 must be calculated by pushing every byte in the file through the MD5 algorithm. This is CPU and IO intensive! Thus you can specify the option --no-md5. This will compare the upload time on S3 to the "last modified" time on the local item, and not do md5 calculations locally at all. This might cause more transfers than are absolutely necessary. For example if the file is "touched" to a newer modified date, but its contents didn't change. Conversely if a file's contents are modified but the date is not updated, then the sync will pass over it. Lastly, if your clock is very different from the one on the S3 servers, then you may see unanticipated behavior. |