S3Sync.net
February 02, 2014, 01:18:52 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
  Home Help Search Login Register  
  Show Posts
Pages: [1] 2 3 ... 25
1  General Category / Questions / Re: extrange files created when upload to s3 on: February 19, 2011, 01:22:26 PM
Those are nodes that s3sync uses to denote a folder/directory.  The md5 is well-known so a list operation can quickly know that it's a directory name and not a file.  They need their own node in order to store permission and owner data. 

Other tools after s3sync came along and used different standards.  Such as not storing any dir nodes, or using a node whose name trails a "/" and has empty contents.  These other tools don't really understand what s3sync is doing with its dir nodes, which is why it shows them to you in a manner that you think they are something strange.  Don't worry, they are just normal.
2  General Category / Report Bugs / Re: S3Sync Stopped wWorking on: February 07, 2011, 11:48:59 AM
Like some file or dir that used to be there is now gone?

Would help to see your command lines, and see an output with -d
3  General Category / Report Bugs / Re: s3sync fails on 400 Bad Request on: February 03, 2011, 08:28:13 AM
Usually a 400 error means the client side has done something wrong, so that's why we don't just keep retrying it.  (Retries are done to 500 errors because amazon asks us to). 

If you want to change the range of errors we retry on, go into s3try.rb and change

Code:
break unless ((500...600).include? result.http_response.code.to_i) or forceRetry

to be 400...600 (for example)
4  General Category / Questions / Re: s3sync support multipart? on: December 23, 2010, 11:51:19 AM
No but it definitely needs to.  I'm trying to find some time to do a rewrite.
5  General Category / Questions / Re: full backup everytime? on: October 08, 2010, 09:53:01 AM
sync is always "changes" only.  If you want to also remove files from s3 that are no longer on D:, you need --delete too.
6  General Category / Questions / Re: How do I do incremantals with S3Sync? on: October 05, 2010, 08:30:35 PM
Yep that's how I do it.

Once I rewrite s3sync completely and take advantage of copy/rename, we can do it in a more rsync-like way that doesn't require uploading every change 7 times.
7  General Category / Questions / Re: How do I do incremantals with S3Sync? on: October 05, 2010, 07:37:22 AM
By nature sync overwrites older stuff with the newest changes.  The nature of "incremental backup" is to preserve the older state AND save the new stuff off to the side. 

The point being, you want to minimize storage but be able to get access to 7 days of data.  You can't have both with a simple sync-like solution.  I'd recommend 7 different save locations, rotating each day which one you write to.  Otherwise you will only have the latest day of backup.
8  General Category / Questions / Re: How do I do incremantals with S3Sync? on: September 24, 2010, 03:55:04 PM
Like rsync, it detects whether each item needs to be changed.  By default it uses the md5 values.  Sometimes that's too expensive, so md5 checking can be turned off and then it will rely on file size and (if I recall correctly) upload time.
9  General Category / Feature Requests / Re: leave old backups on S3 -- i.e. myfile.old1 when reuploading myfile on: September 07, 2010, 12:15:59 PM
The poor project has been neglected for a couple years because of lack of time.  I plan a complete rewrite, avoiding ruby in favor of either lua and C, or java.  But it has not happened yet.
10  General Category / Questions / Re: s3sync script for EC2 windows instance on: August 30, 2010, 02:55:32 PM
Well I'm not sure if it's any faster or better than other tools.. it used to be a long time ago but it has not been worked on very actively for a couple years.  Other tools may very well have surpassed it by now.  Shop around!

For the directory thing, you can try the --make-dirs argument to have the local machine make directories when they don't exist.  That might be the reason for that error. 

For the Iconv error, do you have your S3SYNC_NATIVE_CHARSET set properly (to Windows-1252)?  If so, then maybe the file names are screwy on the S3 side, i.e. if the tool used to upload them did a bad job translating them into UTF-8 to be compliant with S3.  The iconv error has to do with converting strings between different character sets
11  General Category / Questions / Re: s3sync script for EC2 windows instance on: August 27, 2010, 07:41:54 PM
I could probably offer advice about errors, if you post some details.

s3sync doesn't have a way to skip errors and keep going.  But you could exclude things which fail and start it again.  Also it only transfers one thing at a time.  It seems like you could use something more brute force.
12  General Category / Questions / Re: Making existing items public on destination on: July 13, 2010, 04:52:34 PM
Not really I just wish for more time myself.  It makes me sad that this and some other projects are stagnant due to a limited amount of my time.
13  General Category / Questions / Re: Making existing items public on destination on: July 10, 2010, 02:45:34 PM
Nope, this feature is lacking in my tools.  As are a bunch of others these days.  Oh for an infinite amount of free time....
14  General Category / Questions / Re: s3sync ssh connection reset loop and slow performance overall on: May 27, 2010, 02:50:31 PM
I don't know what I could do to help with this.  The http transfer stuff is all down in the ruby library.
15  General Category / Feature Requests / Re: support for amazon reduced redundancy storage (rrs) on: May 25, 2010, 08:02:05 AM
Not actively, but I still want to make a new one.  With a more pluggable design, too.
Pages: [1] 2 3 ... 25
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!