I had a similar need. I run s3sync over my DSL connection, but I could only leave it running at certain times because if I max out my upstream, everything comes to a crawl.

I've added support for throttling upstream and downstream.

--ulimit=<bytes/s>
--dlimit=<bytes/s>

I primarily use it for upload throttling and it seems to work well enough. I did a quick test for download throttling and it seems to work also.


How it works:
It depends on progress stream (since it need to calculate the rate), so we always use progress stream now. To toggle displaying of progress, I changed that to be a parameter in the constructor.

Rate is calculated whenever read() or write() is called. If the rate is greater than the limit, it calculates how long it would need to sleep for to bring it down to the limit, and then it sleeps that amount.


(Un)fortunately the patch is intermingled with my Content-MD5 patch.


Tienshiao

Oops, sorry about the backwards patch.

hexdigest returns the MD5 in hexidecimal representation [0-9A-F].

Amazon's docs specify Content-MD5 should be in base64 (http://en.wikipedia.org/wiki/Base64) .



In hindsight, there probably isn't a need for the commandline switch (--content-md5). There isn't a good reason not to use Content-MD5 if you're already calculating the MD5.

I've added support for the Content-MD5 header. It seems to work OK for me.

Edit: in case people missed it, the patch is attached to the reply. You might need to be logged in to see it.