Shell command to tar directory excluding certain files/folders?

Question

Is there a simple shell command/script that supports excluding certain files/folders from being archived? I have a directory that need to be archived with a sub directory that has a number of very large files I do not need to backup. **Not quite solutions:** The `tar --exclude=PATTERN` command matches the given pattern and excludes those files, but I need specific files & folders to be ignored (full file path), otherwise valid files might be excluded. I could also use the find command to create a list of files and exclude the ones I don't want to archive and pass the list to tar, but that only works with for a small amount of files. I have tens of thousands. I'm beginning to think the only solution is to create a file with a list of files/folders to be excluded, then use rsync with `--exclude-from=file` to copy all the files to a tmp directory, and then use tar to archive that directory. Can anybody think of a better/more efficient solution? EDIT: **cma**'s solution works well. The big gotcha is that the `--exclude='./folder'` **MUST** be at the beginning of the tar command. Full command (cd first, so backup is relative to that directory): cd /folder_to_backup tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .

Answer 1

I like your thought process, but I wouldn't over-think this ... you're very close

to a real solution!

In the spirit of "separation of concerns", yes, I like your idea of generating a

file containing the list files to be archived by tar.

But let's not even get that specific yet. Let's just say you want to

commit to NOT burdening tar (the archiver) with determining what needs to be archived.

How would you implement the logic that generates a list of files

under a given directory that meet your archiving criteria?

Why not use the obvious-choice commands like find, grep, etc.?

As an example, if your backup requirements were limited to:

- all C language source files that are less than 4MB

- all Python source files that are less than 4MB

the following one-line script would work:

find $PWD -type f -size -4M -iname "*.py" -o -iname *.c > /tmp/tar/ToBeArchived.txt

Now that we have a list, we can use tar to draw from that list as follows:

tar cvf /tmp/tar/archive.tar -T /tmp/tar/ToBeArchived.txt

BTW, we generally could do something like this in one step using command substitution ...

tar cvf /tmp/tar/archive.tar $(scriptThatGeneratesList)

but if the list of files is substantial, the buffer size limit

allocated for command line parameters might get exhausted.

Using tar's "-T file" option is the safer bet ...