S3Sync.net
February 02, 2014, 01:22:31 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: Price of s3sync  (Read 3607 times)
Trump
Newbie
*
Posts: 2


View Profile
« on: January 29, 2009, 09:51:06 AM »

In looking at the pricing line items for S3, I noticed the following:

     $0.01 per 1,000 PUT, COPY, POST, or LIST requests

Which is fine. My only concern is, I would like to run a recursive s3sync on lots of objects on a scheduled cron job every 3 hours (or less.)

I understand new files will need to be transferred, and a penny is expected to be spent there. But what about the checks s3sync does on existing files? Is that a LIST for each file regardless of if it exists or not? How does that work; I'd like to better understand what happens when you recursively run s3sync. I have one bucket, and many many objects in multiple directories and subdirectories (I guess they're all just keys.)

Thanks in advance for any clarification on this. My next step would be to learn a bit more about ruby and figure it out for myself, but I'd really rather not. (:

-Trump
Logged
assettv
Newbie
*
Posts: 2


View Profile
« Reply #1 on: January 29, 2009, 12:52:46 PM »

Each LIST request will return something like 1,000 files, with multiple requests needed to fetch details of further files.

As a result, say you have 3,000 files that would require 3 LIST requests to fetch details about each. This is needed on each s3sync invocation to build the server list, for comparison with the local list.

The cost of running s3sync is:

Cost of LIST requests (which depends on the number of files already on the server)
+
Cost of files to upload (new files not on the server)

With the former always applying even if there are no new files to upload.
Logged
Trump
Newbie
*
Posts: 2


View Profile
« Reply #2 on: January 29, 2009, 09:46:55 PM »

Thanks assettv.

Also, thanks to the timeliness of the usage reports amazon does, I was able to figure out that it cost about $0.055 per recursive sync. Your information helped as well.

-Trump
Logged
danm2
Newbie
*
Posts: 2


View Profile
« Reply #3 on: February 19, 2009, 01:18:00 AM »

Hi,

Whats the volume of your directory and how many files it contain?
Could you clarify you calculation to amazon cost?

Thanks
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!