[BackupPC-users] Skipping directories with .nobackup files (creating dynamic file exclude/include lists)

[John Rouillard]

Hi all:

Our current backup system (hdup) allows the user to prune directories
from the backups. So I can backup the /home share and have a user
create

/home/user/data/lots_of_junk_data_that_doesnt_need_backup/.nobackup.

that file will prevent the backup of

/home/user/data/lots_of_junk_data_that_doesnt_need_backup

recursively. To do this with backuppc, I was planning on using a
DumpPreShareCmd to create /tmp/share.backup.exclude.<random_id> file
that looks like:

/user/data/lots_of_junk_data_that_doesnt_need_backup

and changing the rsync command options to include:

--exclude-from /tmp/share.backup.exclude

Does anybody see an issue with this? Does anybody do something
similar?

It would be useful to have the main config file settings available in
the per host config files. That way a per host config file could use a
default "homes.pm" file that adds /homes to the default share list
from config.pl and could also augment the default RsyncArgs to add the
--exclude-file option as well as augmenting the DumpPreShareCmd. So you
could do a:

$Conf{DumpPreShareCmd} .= "&& /etc/admin/bin/mkdirexclude $share";

[Rob Owens]

When I backup /home, I exclude /*/tmp/

Anything that should not get backed up goes in /home/username/tmp, or
any subdirectory of that folder. It's not exactly what you're looking
for, but I've found it to be a workable solution.

-Rob

Post by John Rouillard
Our current backup system (hdup) allows the user to prune directories
from the backups. So I can backup the /home share and have a user
create
/home/user/data/lots_of_junk_data_that_doesnt_need_backup/.nobackup.
that file will prevent the backup of
/home/user/data/lots_of_junk_data_that_doesnt_need_backup
recursively. To do this with backuppc, I was planning on using a
DumpPreShareCmd to create /tmp/share.backup.exclude.<random_id> file
/user/data/lots_of_junk_data_that_doesnt_need_backup
--exclude-from /tmp/share.backup.exclude
Does anybody see an issue with this? Does anybody do something
similar?
It would be useful to have the main config file settings available in
the per host config files. That way a per host config file could use a
default "homes.pm" file that adds /homes to the default share list
from config.pl and could also augment the default RsyncArgs to add the
--exclude-file option as well as augmenting the DumpPreShareCmd. So you
$Conf{DumpPreShareCmd} .= "&& /etc/admin/bin/mkdirexclude $share";

[Rich Rauenzahn]

Post by John Rouillard
Our current backup system (hdup) allows the user to prune directories
from the backups. So I can backup the /home share and have a user
create
/home/user/data/lots_of_junk_data_that_doesnt_need_backup/.nobackup.
that file will prevent the backup of
/home/user/data/lots_of_junk_data_that_doesnt_need_backup

Have you tried the -F option in rsync? Seems to work for me. I added
-F to the rsync remote options and then dropped a .rsync-filter file in
the directory. The contents of .rsync-filter is just one line,

- *

It is a little more work than just 'touch .nobackup'. On the other
hand, maybe a cronjob could be made with an appropriate find that
sprinkles these wherever the .nobackup files exists. It could be
followed by another find for .rsync-filter files and removes them if a
.nobackup files doesn't exist as well (maybe that's not such a good idea
if others are using rsync filters on the box independently of you...).
Anyway, I'd suggest doing this in File::Find in perl.

Rich

[John Rouillard]

Post by Rich Rauenzahn

Post by John Rouillard
Our current backup system (hdup) allows the user to prune directories
from the backups. So I can backup the /home share and have a user
create
/home/user/data/lots_of_junk_data_that_doesnt_need_backup/.nobackup.
that file will prevent the backup of
/home/user/data/lots_of_junk_data_that_doesnt_need_backup

Have you tried the -F option in rsync? Seems to work for me.

No I haven't. Was using rsycn 2.6.3 till recently and -F/--filter
isn't in that version.

Post by Rich Rauenzahn
I added
-F to the rsync remote options and then dropped a .rsync-filter file in
the directory. The contents of .rsync-filter is just one line,
- *
It is a little more work than just 'touch .nobackup'.

True but as you explained could be easily automated and actually
provides better control.

Thanks for the pointer.