S3Sync.net

General Category => Feature Requests => Topic started by: ferrix on February 16, 2007, 03:27:55 AM



Title: --exclude
Post by: ferrix on February 16, 2007, 03:27:55 AM
subject says it all.


Title: Re: --exclude
Post by: wantmoore on February 16, 2007, 03:37:23 PM
Yes, please! I recall seeing in the forever-long AWS thread that you had a patch - I'd be willing to test if you'd send me a new version of the script. I would love this even more if there were a way to exclude a given directory name, in all subdirectories.

Example:
Given this directory structure:
Code:
/home/user1/dir1
/home/user1/dir2
/home/user1/dir3
/home/user2/dir1
/home/user2/dir2
/home/user2/dir3

And this command:
Code:
~>./s3sync.rb -r --delete --exclude dir2 /home mybucket:

Everything except /home/user1/dir2 and /home/user2/dir2 would be sync'd to S3.

Can we make this happen?


Title: Re: --exclude
Post by: ferrix on February 16, 2007, 03:44:20 PM
Here is the patch I was sent by Benjamin Jackson.  I have been so busy that I literally haven't even looked at it yet.

You (anyone) are welcome to poke at it and let me know what you think.  I can't recommend or anti-recommend it yet.


Title: Re: --exclude
Post by: maelcum on February 18, 2007, 12:06:52 PM
I second this.
It's one of *the* most important features I'm missing.


Title: Re: --exclude
Post by: ferrix on February 19, 2007, 08:14:08 AM
Done.


Title: Re: --exclude
Post by: jeffpar on April 21, 2007, 11:20:15 PM
I'm trying to use --exclude like so:

Code:
ruby s3sync.rb -n -r --exclude="\.svn/" --progress (source) (dest)

and it still wants to process ".svn" folders.

Am I misunderstanding the syntax of --exclude="<regexp>" or some peculiarity of Ruby regular expressions?

Thanks,
Jeff


Title: Re: --exclude
Post by: ferrix on April 22, 2007, 01:24:16 AM
Nope you're fine.  Are you sure it's "processing" them, or just making a node for the directory and then skipping its contents?  Because that's what your regex is telling it to do.

If you want to skip the folder itself, as well as its contents, you should trail with /|$  instead of just /
(that's "slash, or the end of the string")


Title: Re: --exclude
Post by: maelcum on April 23, 2007, 10:23:34 AM
After a long stay in the hospital I'm kinda up and running.
Thank you so much for implementing --exclude. It's a boon!!


Title: Re: --exclude
Post by: michela on October 30, 2007, 03:44:35 PM
Nope you're fine.  Are you sure it's "processing" them, or just making a node for the directory and then skipping its contents?  Because that's what your regex is telling it to do.

If you want to skip the folder itself, as well as its contents, you should trail with /|$  instead of just /
(that's "slash, or the end of the string")

This is not working for me. This isn't skipping .svn folders, it's skipping

What does the trailing |$ supposed to do? The following cmd is excluding all subdirectories.

./s3sync.rb -rd --delete --progress --exclude="\.svn/|$" -n  /etc <bucket>:


Here's the debug from a test folder structure I created

localTreeRecurse /tmp/testing
Test /tmp/testing/bli1
Test /tmp/testing/bli3
Test /tmp/testing/bli2
Test /tmp/testing/.svn
Adding it as a dir node
skipping local item /.svn because of --exclude
.svn is a dir node
skipping local item /.svn because of --exclude
skipping local item /bli1 because of --exclude
skipping local item /bli2 because of --exclude
skipping local item /bli3 because of --exclude

In this example (according to your post) the following would be synch'ed

testing/
 bli1
 bli2
 bli3

and the following would be skipped

testing/
 .svn/
  blah1
  blah2
  blah3

but all you get is a file node 'testing'






Title: Re: --exclude
Post by: ferrix on October 31, 2007, 04:37:55 AM
"testing" is probably a directory node not a file node.  But that's beside the point.

I suppose the alternation in the regex (the pipe symbol) must be grouping wrong.
Maybe "\.svn(/|$)"

The idea of the last part is to say "slash, or end of string".  In other words match it when there's more after the .svn or when the "n" is the last character of the whole string.

Didn't test this, just thought experiment.