Discussion on Zipping/Tar Files Actively Being Written

[PBXMePlz]

So I'm writing a backup/restore script, and it's getting a little intense in my head.
I mean seams simple enough, tar up the files, restore them on a new install. The restore will/can be relatively destructive, it is after all a restore. The backup though I want to be able to run as often as possible if necessary. Which brings me to this question:
What happens when tar attempts to archive something actively being written (say someone recording a voicemail, or something)? Will this just bomb out? Will it be destructive to the source file at all?

And then there's PGDump. I'm thinking this is less likely to be problematic, but same question applies.

Thoughts on the matter?

 

[Adrian Fretwell]

That is a good question. I don' t know about PGDump but I guess it will have a mechanism to ensure consistency.

In my experience tar normally fails if you instruct it to archive a directory that has files open in it. There is a command line switch to instruct tar to ignore the error, but that is all it does, it just ignores the error, it does not rectify the problem. I think it is something like tar --warning=no-file-changed but check your man pages for your tar version.

For the kind of regular backup, you are proposing I would prefer to use rsync with the -avz switches. rsync will look after open files and file changes.

 

[PBXMePlz]

Rsync requires an active server though, I'm writing it with rclone to double as a backup option.
I don't technically have to tar the directories I suppose but the compression to save space, and the simplicity of one file will allow me to maintain a historical reference easily enough. Granted I suppose realistically a historical reference may only be so good since the new environment is likely to be the most up to date.

I don't necessarily have an issue with tar failing on occasion to take a backup. The big thing is whether the process might manage to hinder production in some manner. Which it sounds like it won't to your point, as it would just fail its attempt.

 

[Adrian Fretwell]

Probably a bit overkill but you could rsync to a backup directory structure on the same box and then tar that. I know it wastes space but disks are cheap enough these days.

 

[PBXMePlz]

Probably a bit overkill but you could rsync to a backup directory structure on the same box and then tar that. I know it wastes space but disks are cheap enough these days.

Hmmmm...agreed on disks are cheap anymore. Trying to get all this into Wasabi.
Might be overkill, but I guess what you're saying is: rsync would wait on a file to then copy the most up to date; thus ensuring that (assuming the process completes) rsync pretty much guaranteed has everything and the most up to date version from when it was run?

Probably going to stick with just tar'ing assuming we're not worried about it effecting production files being written, but I might also play with mounting a Wasabi bucket as a drive and then rsyncing to that, then tar'ing and copying to another bucket.