Backup / Restore - Working Script?

[Andrew Byrd]

I have tried to use the backup / restore scripts on Fusionpbx docs and I am told they are out of date and not relevant. I have yet to see a script to back up and restore that actually works

I am willing to pay someone ($100? $200 - you tell me) to write me a back up and restore script that will enable me to do the following:

1) Back up a current in production Debian 10 Fusionpbx server

2) Be able to restore that back up to another offsite server running Debian 10. I can install Fusionpbx current on the new server and have it ready for the restore.

As I said, I have tried the scripts on the Fusionpbx site and they have lots of errors.

Please contact me if you believe you can do this project for me. I will be more than happy to pay you

 

[roger_roger]

I have a backup script that I've been using for years and you can have it for free. Please note that I use .pgpass so I don't have to enter the password in the script. You can look at the following link to see how to configure that.

https://www.postgresql.org/docs/9.3/libpq-pgpass.html

The backup script:

#!/bin/sh
now=$(date -d "yesterday" +%Y-%m-%d)
#delete backups older 7 days
find /var/backups/fusionpbx/postgresql/fusionpbx_pgsql* -mtime +7 -exec rm {} \;
find /var/backups/fusionpbx/*.tgz -mtime +7 -exec rm {} \;
#delete freeswitch logs older 7 days
find /usr/local/freeswitch/log/freeswitch.log.* -mtime +7 -exec rm {} \;
#echo "Starting Backup"
#backup the database
pg_dump -Fc -w -h 127.0.0.1 -U fusionpbx -f /var/backups/fusionpbx/postgresql/fusionpbx_pgsql_$now.sql
#
#backup the files and directories
tar -zvcf /var/backups/fusionpbx/pbx1-backup_$now.tgz /var/lib/freeswitch/recordings /var/www/fusionpbx /usr/local/freeswitch/scripts /var/lib/freeswitch/storage /etc/fusionpbx /usr/local/freeswitch/conf /var/scripts /var/backups/fusionpbx/postgresql/fusionpbx_pgsql_$now.sql
echo "upload to backup storage"
sshpass -p '<your_password>' scp /var/backups/fusionpbx/pbx1-backup_$now.tgz bruceg@100.64.50.137:/var/backups/fusionpbx/nightly
echo "Backup Complete";

Restore script - please note that all lines are commented out; I once ran it by mistake

#!/bin/sh
#now=$(date +%Y-%m-%d)
#remote_server=162.208.16.3
#delete freeswitch logs older 7 days
#find /usr/local/freeswitch/log/freeswitch.log.* -mtime +7 -exec rm {} \;
#message to the console
#echo "Restoring the Backup"
#extract the backup from the tgz file
#tar -xvpzf /var/backups/fusionpbx/pbx1-backup_$now.tgz -C /
#remove the old database
#psql --host=127.0.0.1 --username=fusionpbx -c 'drop schema public cascade;'
#psql --host=127.0.0.1 --username=fusionpbx -c 'create schema public;'
#restore the database
#pg_restore -Fc --host=127.0.0.1 --dbname=fusionpbx --username=fusionpbx /var/backups/fusionpbx/postgresql/fusionpbx_pgsql_$now.sql
#backup the files and directories
#echo "Restore Complete";

 

[Andrew Byrd]

Should I modify the .pgpass on both servers so that the restore will function as well? If yes, then do I use Server A's db password on server B?

 

[Andrew Byrd]

The very beginning of the backup script fails and this is the result

root@abnetworks1:~# #!/bin/sh
root@abnetworks1:~# now=$(date -d "yesterday" +%Y-%m-%d)
root@abnetworks1:~# #delete backups older 7 days
root@abnetworks1:~# find /var/backups/fusionpbx/postgresql/fusionpbx_pgsql* -mtime +7 -exec rm {} \;
#delete freeswitch logs older 7 days
find /usr/local/freeswitch/log/freeswitch.log.* -mtime +7 -exec rm {} \;root@abnetworks1:~# find /var/backups/fusionpbx/*.tgz -mtime +7 -exec rm {} \;
root@abnetworks1:~# #delete freeswitch logs older 7 days
root@abnetworks1:~# find /usr/local/freeswitch/log/freeswitch.log.* -mtime +7 -exec rm {} \;
find: ‘/usr/local/freeswitch/log/freeswitch.log.*’: No such file or directory

 

[roger_roger]

Check the local of your freeswitch logs - they could be in /var/log/freeswitch

Both fusionpbx postgres databases should have the same password, so yes.

 

[markjcrane]

I just updated the documentation at https://docs.fusionpbx.com/en/latest/getting_started/backup.html
This is a working backup script that I use just about every week for people using Debian

 

[Andrew Byrd]

Thank you Mark - What about an update restore script? The restore script on the docs does not work and returns errors

I do appreciate you updating this

 

[Andrew Byrd]

Mark - Can you update the restore script? I am fascinated at how the emphasis is on back up but a backup does absolutely no good if it can be restored. My biggest issue is the error I get at the end of the restore script saying there is a fatal error: psql: error: could not connect to server: FATAL: password authentication failed for user "fusionpbx"

The .pgpass is the same on server B that I am restoring to as it is on server A where I did the backup

By the time the script gets to this point it has already changed the /etc/fusionpbx/config.php db password to match server A.

So Server A config.php and Server B config.php and both .pgpass files on both servers match server A

I don't understand why the db password is incorrect

This issue surfaces when the script gets to this point:

#remove the old database
psql --host=$database_host --port=$database_port --username=fusionpbx -c 'drop schema public cascade;'
psql --host=$database_host --port=$database_port --username=fusionpbx -c 'create schema public

This is the result after it gets to this point:

psql: error: could not connect to server: FATAL: password authentication failed for user "fusionpbx"
FATAL: password authentication failed for user "fusionpbx"
psql: error: could not connect to server: FATAL: password authentication failed for user "fusionpbx"
FATAL: password authentication failed for user "fusionpbx"
pg_restore: error: connection to database "fusionpbx" failed: FATAL: password authentication failed for user "fusionpbx"
FATAL: password authentication failed for user "fusionpbx"
Restore Complete

 

[ad5ou]

The documented restore scripts will work when the database password is correct.
There are multiple ways and reasons to restore a backup, so a “one size fits all” script isn’t practical.
The tricky part in your scenario as I read it, is you have two different database passwords to deal with. By default the fusion install scripts set a random database password. Your backup contains config info for the old database password from the source server.

If starting with a blank slate on a new server, I would set the DB password the same as original server by editing “config.sh” in the install resources directory before running install.sh

A slightly harder option would be to modify the DB password from pgsql cli

Another option is to install fresh copy as normal, then copy contents of /etc/fusionpbx to a safe location so the fresh config files aren’t lost during restore process. Pull DB password from the fresh config.php file and paste into the restore script. Once script finishes, copy the saved /etc/fusionpbx files back where they were and overwrite the restored config files.

I have a backup of my production primary server sent to a spare machine at my office nightly. The restore script that has been posted is run on the spare server every morning without issue.

 

[s2svoip]

@ad5ou thanks for this post, very helpful - this is what I am looking for

I managed to custom the script to pull my backup from storage and restore it, I want to do this daily so put it in the cron folder - just need to work out how to ensure the cron daily restore runs after the cron daily backup!

Do you happen to know the easiest way I could get it to email the output from the script as an attachment when it's done, so I have a log of the restores happening each day and any errors that occurred

 

[ad5ou]

There are a few ways to tackle it.
If you have ssh key authentication available between servers, you could execute everything including the restore functions with one script
A probably easier route is to set the cron job times yourself instead of relying on cron.daily. Set backup at say 1am, then on other server set restore script to run at 3 am.

There are plenty of tutorials for setting up cron to send out emails after jobs etc. Personally I get enough email and just have my backup scripts echo to a local file info I might want to check on. For one script I run nightly to convert call recordings, I log the time it started, then the end time.

echo $(date -u) "conversions started" >> /root/convert.log
<stuff happening>
echo $(date -u) "conversions complete" >> /root/convert.log

Gives me:

Sat 06 Feb 2021 07:00:01 AM UTC conversions started
Sun 07 Feb 2021 07:00:01 AM UTC conversions started
Sun 07 Feb 2021 07:02:02 AM UTC conversions complete
Mon 08 Feb 2021 07:00:01 AM UTC conversions started
Mon 08 Feb 2021 07:02:22 AM UTC conversions complete
Tue 09 Feb 2021 07:00:01 AM UTC conversions started
Tue 09 Feb 2021 07:34:32 AM UTC conversions complete
Wed 10 Feb 2021 07:00:01 AM UTC conversions started
Wed 10 Feb 2021 07:40:36 AM UTC conversions complete

 

[s2svoip]

thanks, I see that allows you to echo single lines to the file, I was hoping to echo everything you see when you run the file from CLI, obviously its huge because of the zip extraction and database restore but would be the most comprehensive log of the restore so you can see if there was any issues

edit. sorry I am being silly, I understand how the echo redirect works, thanks!

 

[ad5ou]

I have more faith in scripts once they work once but you can always have cron redirect things to a file if you are worried

 

[Scott Williams]

I found that /etc/fusionpbx/config.lua stores the database password as well, so I needed to put the new database password in there. Wish I had made the new database password the same as the old, would have been simpler.

 

[Davesworld]

I just updated the documentation at https://docs.fusionpbx.com/en/latest/getting_started/backup.html
This is a working backup script that I use just about every week for people using Debian

I have always gotten a backup and that script is already in daily cron on my Debian installation however, it does not contain the commented out restore script posted by another poster and reposted here:

#!/bin/sh

#now=$(date +%Y-%m-%d)

#remote_server=162.208.16.3

#delete freeswitch logs older 7 days

#find /usr/local/freeswitch/log/freeswitch.log.* -mtime +7 -exec rm {} \;

#message to the console

#echo "Restoring the Backup"

#extract the backup from the tgz file

#tar -xvpzf /var/backups/fusionpbx/pbx1-backup_$now.tgz -C /

#remove the old database

#psql --host=127.0.0.1 --username=fusionpbx -c 'drop schema public cascade;'

#psql --host=127.0.0.1 --username=fusionpbx -c 'create schema public;'

#restore the database

#pg_restore -Fc --host=127.0.0.1 --dbname=fusionpbx --username=fusionpbx /var/backups/fusionpbx/postgresql/fusionpbx_pgsql_$now.sql

#backup the files and directories

#echo "Restore Complete";

I do not understand the remote server entry.

 

[ad5ou]

I do not understand the remote server entry.

That looks like an entry the original poster forgot to delete before posting the script. My guess is there used to have additional lines to the script to automate pulling a backup from said remote server.

Based on what you posted above, that variable is not used.

 

[PBXMePlz]

I found that /etc/fusionpbx/config.lua stores the database password as well, so I needed to put the new database password in there. Wish I had made the new database password the same as the old, would have been simpler.

This is what I needed! I'm writing a script now that is a slight modification on @markjcrane or at least the inspiration was drawn from his. This one will use rclone to backup-to/restore-from a cloud repository.