S3Sync.net
February 02, 2014, 01:33:26 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
 
  Home Help Search Login Register  
  Show Posts
Pages: [1]
1  General Category / Questions / Re: Files being modified during upload on: May 05, 2007, 02:32:45 AM
ferrix, short and to the point responses like that usually suggest that someone's been irritated by something they consider to be a stupid  question or unreasonable request, such that it doesn't warrant anything more.  If you revisit my post you'll see that I was aware it might not be an appropriate request and so wasn't directly doing that, I was fully aware that this wasn't a straight forward matter and I was simply looking to try and clarify the context of the issue I'm trying to solve.  So, I apologise if I rubbed you up the wrong way and that's how it came out.

  • If it's out of the scope of this application and doesn't belong in it (this is after all a syncing application and I'm looking to backup with it), then fine, I understand that it's something I should not expect of it.
  • If it's something that's a very complex issue due to the underlying manner in which the OS deals with file locking then that tells me I'm not likely to find a simple solution and I should stop looking and go with archiving the rotated files like suggested, or archiving them first.
  • Even if simply it's something you for your own reasons don't want in the app then again, no problem, it's your work and it's your prerogative so I wouldn't argue it.

But please don't take me for one of those irritating users expecting you to make your application fit my needs or hand hold me through solving my problems. I appreciate the work of everyone who gives their time to open source projects and never at any point feel like they're obliged to do what we users request of them. I'm simply trying to answer some questions I've posed to myself and get a greater understanding of my situation so I can find my way to the most appropriate solution for my needs.

lowflyinghawk, thanks for the pointers.  I've checked the man pages but it's my lack of detailed understanding of the terminology and concepts referred to that's been hampering my investigations.  The logic of my thinking is that if something as simple as tar is able to continue archiving a file as it's contents is on opening but while writes continue on it, it suggests to me that the this is a matter that has a solution within the standard locking mechanisms.  But I imagine there's a lot more to it than that.

Never mind though, I think I'll simply go with archiving inboxes and logs by tarring them first.

Cheers,

Jon
2  General Category / Questions / Re: Does s3sync preserve metadata? on: May 04, 2007, 09:29:27 AM
Ahhh, interesting.  No standards being set out is a bit of a disappointment - seems like just asking for problems!  I imagine it's paths and defining directories, etc where the confusion comes in?

Yes, s3fox is what I've been using so far as it seems like a nice light weight tool and I like anything that I can use on my Mac at home and on my work PC (extensions avoid installation problems with locked down desktops too!), real shame it's closed source.  Like you say, it could do with the benefit of some extra help.  One of the most annoying things is that listing long buckets can lock my browser up for quite a while sometimes.

I love the command line, but sometimes you don't feel like doing anything more than point and click eh!
3  General Category / Questions / Re: Files being modified during upload on: May 04, 2007, 09:11:00 AM
Hi ferrix, thanks for the response.

I was going to ask if some sort of file locking would make for a reasonable feature request however, to quote a colleague of mine who's more experienced in that sort of thing, it's apparently 'a bit of a minefield'!  A bit of Googling turns up some info which suggests Ruby has various file locking mechanisms which are fairly easy to implement, but I was unable to tell exactly how any of them work.

For some reason I thought there was some sort of lock that enabled you to read from a file while writes continued to be performed on it, sort of like you were viewing a snapshot of it as it was on opening.  Thinking about that though, that seems unlikely and it's most likely it's something that would be implemented on an application level using temp copies, etc.

If all else fails I guess, I'll have to go with making sure that whatever files I backup are definitely not being written to before syncing them however this is something I've always looked for when it's come to backing up.  It's partially so I can make sure I backup the most up to date data, but also so I can keep backups as simple as possible i.e. a straight forward backup of a whole directory tree, without having to note certain exceptions.

Cheers,

Jon
4  General Category / Questions / Re: Does s3sync preserve metadata? on: May 02, 2007, 05:32:41 AM
Since my question is related to metadata I thought I'd bump this thread rather than start a new one.  Smiley

Where / how is the metadata stored?

There has been mention of using S3Pilot to browse the data - is everything stored in such a way that any other S3 tool should be able to browse the files?

Thanks,

Jon
5  General Category / Questions / Files being modified during upload on: May 02, 2007, 05:27:29 AM
Hi all,

What's the expected behaviour of s3sync if I try to upload a file and it gets updated during the process?

I only ask as one of the things I plan to do is just keep a sync of /var/log on S3, and while testing the other day it kept hanging on /var/log/messages which I presume was due to it being updated while uploading.  It detected the failure and said it was retrying, however then sat there for some time.  As a crude way to check if it was doing I simply looked for it taking any load in top and found it looking pretty inactive.

Do I have any choice other than to copy / tar it out elsewhere then do the upload?

Thanks,

Jon
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!