S3Sync.net

General Category => Questions => Topic started by: seengee on November 19, 2007, 09:37:40 AM



Title: Create multiple backup buckets
Post by: seengee on November 19, 2007, 09:37:40 AM
Hi,

I would like to be able to create a number of backups within S3, e.g. Monday_01_01_07  / Tuesday_02_01_07 etc etc so i can create a series of backups for a week, when i hit 7 days i start deleting the oldest backup and replacing with a new one. There will be approx 10Gb of data so obviously i do not want to do a complete upload every day. Ideally I would copy the backup from the previous day and then s3sync to this, is this possible as i have seen no reference to a copy command for s3  or s3sync, if not does anyone have any idea how else this could be achieved.

thanks a lot.

.gee


Title: Re: Create multiple backup buckets
Post by: ferrix on November 19, 2007, 08:24:15 PM
In the S3 paradigm it is not possible to copy a node without re-transferring it.


Title: Re: Create multiple backup buckets
Post by: seengee on November 20, 2007, 04:54:29 AM
thanks for your response. That does seem to be a major issue for me, i will have to look elsewhere for a remote backup solution unless there is some kind of workaround.


Title: Re: Create multiple backup buckets
Post by: ferrix on November 20, 2007, 02:30:46 PM
It's not perfect but:
I keep one s3sync target copy named for each day of the week and sync to that copy on its respective day.  Initial upload overhead is 7x total, but after that it's just updating the Tuesday prefix every Tuesday (the sync would only transfer items that changed since the previous Tuesday).

The Better solution would be to build a block device abstraction layer on top of S3 (which I intend to do eventually)


Title: Re: Create multiple backup buckets
Post by: seengee on November 21, 2007, 06:53:45 AM
hmmm..

thats an interesting idea which i might look into actually.

thanks a lot.