S3Sync.net
February 02, 2014, 01:30:52 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
  Home Help Search Login Register  
  Show Posts
Pages: [1]
1  General Category / Questions / Re: Help needed: "no such file to load -- iconv (LoadError)" on: January 15, 2010, 09:34:57 PM
Found this on a different forum:

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
It's here:
/usr/ports/converters/ruby-iconv

If you are new to FreeBSD [ like me ],
/usr/ports/
is loaded up with a bunch of software which you can install.

Usually software I find there installs cleanly with a simple
make
make install

So, I installed ruby-iconv and now
require 'iconv'
returns true rather than an exception.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

hope it works for you.
2  General Category / Questions / Re: S3Sync & Ruby 1.9.1 on: January 15, 2010, 09:27:45 PM
Sounds like you are missing the md5.rb library file.  You should first verify you have it on your system. I too am running version 1.8.X and it is found on my system here:  /usr/lib/ruby/1.8/md5.rb

I read on a different forum that you may need to update line 23 of s3sync.rb from:
require 'md5'
to
require 'digest/md5'

because the md5.rb file is in the sub directory for ruby 1.9.

Hope that helps.
3  General Category / Questions / Re: s3sync randomly re-updates nodes [corruption occurring] on: January 10, 2010, 12:15:12 PM
Well, I found out more information and it is not good news. It turns out that a small portion of my data is getting corrupt.  Boy am I glad I decided to investigate further.  I STRONGLY URGE EVERYONE TO VERIFY THEIR TRANSFERS ARE NOT RANDOMLY RE-UPDATING BY USING THE -V FLAG. 

What I did was I starting double checking all of the md5sums and there were indeed some that were different.  I then starting doing binary differing and the corrupt files were only a few bytes different.  The file sizes were all equal but in all cases there were 1-4 extra bytes zeroed out at the end of the file.  Also, in about half of the cases there were also 2-10 consecutive bytes in the middle of the file that were just wrong.

I tried removing the the --ssl option to see if that had something to do with it but it made no different.  I also tried adding in the the '--content-md5' patch from the 'http://s3sync.net/forum/index.php?topic=206.0' topic.  Using that new command line option made no difference either. 

My only conclusion is since I'm doing md5 checks that somehow the file is corrupted prior to the actual upload.  Perhaps this is caused by a bad kernel compile or bad memory, but I'm surprise I'm not having had more serious issues then.  It is also possible that amazon made a change that s3sync is not completely compatible.  I noticed that there hasn't been any new development since June 2008--I would think that the content-md5 patch would be included by now. 

Well, I'm going to start trying some alternative s3 backup methods just to see if I have similar problems.   I do really enjoy s3sync as does exactly what I want but I can't deal with possible corruptions occurring and I don't think I can troubleshoot this much further.
4  General Category / Questions / s3sync randomly re-updates nodes [corruption occurring] on: January 07, 2010, 02:30:26 AM
Has anyone experienced s3sync re-updating nodes when synch'ing even though the file has not changed?  I read in a different topic where this may happen for ever file if you are using trailing forward slashes on directory paths, but this is not the case for me.  I'm trying to sync a directory full of pictures and about 1 in 100 will re-synch.  Everything still works, but I'd like to minimize my bandwidth usage if possible.

Does anyone have any suggestions?

This is the command I'm using to do the work for me:
s3sync.rb -r -v --ssl --delete /raidset/storage/Saves mybucket:Saves

This is sample output:
Update node 1998-2002/2000-11 Thanksgiving & Christmas/101-0115_IMG.JPG
Update node 1998-2002/2000-11 Thanksgiving & Christmas/101-0139_IMG.JPG
Update node 1998-2002/2002-10-31 Office Halloween/IMG_0003.JPG
Update node 1998-2002/2002-11 Caribbean Vacation/3 - Barbados/314.JPG

Thanks
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!