S3Sync.net
February 02, 2014, 01:23:33 PM
Welcome,
Guest
. Please
login
or
register
.
1 Hour
1 Day
1 Week
1 Month
Forever
Login with username, password and session length
Home
Help
Search
Login
Register
S3Sync.net
>
General Category
>
Questions
>
How do I do incremantals with S3Sync?
Pages: [
1
]
« previous
next »
Print
Author
Topic: How do I do incremantals with S3Sync? (Read 4387 times)
scotch
Newbie
Posts: 3
How do I do incremantals with S3Sync?
«
on:
September 23, 2010, 12:41:35 PM »
Hello,
I just got S3Sync up and running on a Linux server and just did a full backup of files to my S3 bucket. But I can't seem to find anything in the documentation about doing incrementals. Could someone steer me toward any info on this?
Thanks,
scotch
Logged
ferrix
Sr. Member
Posts: 363
(I am greg13070 on AWS forum)
Re: How do I do incremantals with S3Sync?
«
Reply #1 on:
September 24, 2010, 03:55:04 PM »
Like rsync, it detects whether each item needs to be changed. By default it uses the md5 values. Sometimes that's too expensive, so md5 checking can be turned off and then it will rely on file size and (if I recall correctly) upload time.
Logged
scotch
Newbie
Posts: 3
Re: How do I do incremantals with S3Sync?
«
Reply #2 on:
October 04, 2010, 12:49:27 PM »
So by default it essentially does incrementals by checking to see if files have been changed and then backing those up? If that's the case, then how do I do a full backup? I was hoping to have once script for daily incrementals and one script for full backups every friday night, so something like:
ruby s3sync.rb -r -s -v "--incremental_switch" /var/www/ my_s3_bucket_incrementals:www_inc
ruby s3sync.rb -r -s -v "--full_switch" /var/www/ my_s3_bucket_fulls:www__full
Thanks for your help.
Logged
ferrix
Sr. Member
Posts: 363
(I am greg13070 on AWS forum)
Re: How do I do incremantals with S3Sync?
«
Reply #3 on:
October 05, 2010, 07:37:22 AM »
By nature sync overwrites older stuff with the newest changes. The nature of "incremental backup" is to preserve the older state AND save the new stuff off to the side.
The point being, you want to minimize storage but be able to get access to 7 days of data. You can't have both with a simple sync-like solution. I'd recommend 7 different save locations, rotating each day which one you write to. Otherwise you will only have the latest day of backup.
Logged
scotch
Newbie
Posts: 3
Re: How do I do incremantals with S3Sync?
«
Reply #4 on:
October 05, 2010, 11:13:32 AM »
Quote
I'd recommend 7 different save locations, rotating each day which one you write to.
Ah, that's a great suggestion. I'll just have a cron job for each day, backing up to that day's bucket. Thanks very much.
Logged
ferrix
Sr. Member
Posts: 363
(I am greg13070 on AWS forum)
Re: How do I do incremantals with S3Sync?
«
Reply #5 on:
October 05, 2010, 08:30:35 PM »
Yep that's how I do it.
Once I rewrite s3sync completely and take advantage of copy/rename, we can do it in a more rsync-like way that doesn't require uploading every change 7 times.
Logged
Pages: [
1
]
Print
« previous
next »
Jump to:
Please select a destination:
-----------------------------
General Category
-----------------------------
=> Announcements
=> Questions
=> General Discussion
=> Report Bugs
===> Closed Bugs
=> Feature Requests
Loading...