--exclude

[ferrix]

subject says it all.

[wantmoore]

Yes, please! I recall seeing in the forever-long AWS thread that you had a patch - I'd be willing to test if you'd send me a new version of the script. I would love this even more if there were a way to exclude a given directory name, in all subdirectories.

Example:
Given this directory structure:

Code:

/home/user1/dir1
/home/user1/dir2
/home/user1/dir3
/home/user2/dir1
/home/user2/dir2
/home/user2/dir3

And this command:

Code:

~>./s3sync.rb -r --delete --exclude dir2 /home mybucket:

Everything except /home/user1/dir2 and /home/user2/dir2 would be sync'd to S3.

Can we make this happen?

[ferrix]

Here is the patch I was sent by Benjamin Jackson.  I have been so busy that I literally haven't even looked at it yet.

You (anyone) are welcome to poke at it and let me know what you think.  I can't recommend or anti-recommend it yet.

[maelcum]

I second this.
It's one of *the* most important features I'm missing.

[ferrix]

Done.

[jeffpar]

I'm trying to use --exclude like so:

Code:

ruby s3sync.rb -n -r --exclude="\.svn/" --progress (source) (dest)

and it still wants to process ".svn" folders.

Am I misunderstanding the syntax of --exclude="<regexp>" or some peculiarity of Ruby regular expressions?

Thanks,
Jeff

[ferrix]

Nope you're fine.  Are you sure it's "processing" them, or just making a node for the directory and then skipping its contents?  Because that's what your regex is telling it to do.

If you want to skip the folder itself, as well as its contents, you should trail with /|$  instead of just /
(that's "slash, or the end of the string")

[maelcum]

After a long stay in the hospital I'm kinda up and running.
Thank you so much for implementing --exclude. It's a boon!!

[michela]

Nope you're fine.  Are you sure it's "processing" them, or just making a node for the directory and then skipping its contents?  Because that's what your regex is telling it to do.

If you want to skip the folder itself, as well as its contents, you should trail with /|$  instead of just /
(that's "slash, or the end of the string")

This is not working for me. This isn't skipping .svn folders, it's skipping

What does the trailing |$ supposed to do? The following cmd is excluding all subdirectories.

./s3sync.rb -rd --delete --progress --exclude="\.svn/|$" -n  /etc <bucket>:


Here's the debug from a test folder structure I created

localTreeRecurse /tmp/testing
Test /tmp/testing/bli1
Test /tmp/testing/bli3
Test /tmp/testing/bli2
Test /tmp/testing/.svn
Adding it as a dir node
skipping local item /.svn because of --exclude
.svn is a dir node
skipping local item /.svn because of --exclude
skipping local item /bli1 because of --exclude
skipping local item /bli2 because of --exclude
skipping local item /bli3 because of --exclude

In this example (according to your post) the following would be synch'ed

testing/
 bli1
 bli2
 bli3

and the following would be skipped

testing/
 .svn/
  blah1
  blah2
  blah3

but all you get is a file node 'testing'

[ferrix]

"testing" is probably a directory node not a file node.  But that's beside the point.

I suppose the alternation in the regex (the pipe symbol) must be grouping wrong.
Maybe "\.svn(/|$)"

The idea of the last part is to say "slash, or end of string".  In other words match it when there's more after the .svn or when the "n" is the last character of the whole string.

Didn't test this, just thought experiment.