[Tutorial] Csync2 for Filesystem Replication

[DigitalDaz]

On each node ensure that the remote node is accessible via the hosts file for the short name.

They do not need to be resolvable externally so in our lab case we will simply add node1 and node2 to each file:

So modify /etc/hosts to suit you IPs and chosen names.

Add for example:

79.137.83.17 node1.local node1
79.137.83.18 node2.local node2

Check both hosts are pingable by short name:

root@vps367591:/usr/src/fusionpbx-sce-install/debian# ping node1
PING node1.local (79.137.83.17) 56(84) bytes of data.
64 bytes from node1.local (79.137.83.17): icmp_seq=1 ttl=64 time=0.104 ms

root@vps367591:/usr/src/fusionpbx-sce-install/debian# ping node2
PING node2.local (79.137.83.18) 56(84) bytes of data.
64 bytes from node2.local (79.137.83.18): icmp_seq=1 ttl=57 time=0.562 ms


On each node edit /etc/hostname and set to your node name as set in the hosts file, eg:

One node1:

echo node1.local > /etc/hostname

One node2:

echo node2.local > /etc/hostname

Then reboot

On each node:

apt-get update
apt-get install csync2

On node1 generate the needed key files:

openssl genrsa -out /etc/csync2_ssl_key.pem 1024
openssl req -batch -new -key /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.csr
openssl x509 -req -days 3600 -in /etc/csync2_ssl_cert.csr -signkey /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.pem
csync2 -k /etc/csync2.key

On node1, edit /etc/csync2.cfg remembering to change host to suit your node names and make it like so:

# please see the REAMDE file how to configure csync2
group ClusterSync
{
host node1.local;
host node2.local;
key /etc/csync2.key;
include /var/lib/freeswitch/recordings;
include /usr/share/freeswitch/sounds;
include /var/lib/freeswitch/storage;
exclude *~ .*;
}

Now copy the csync config, certs and keys to the other node, in my case:

scp /etc/csync2* root@node2:/etc/

On node1 now do:

csync2 -fr /

Follow this with:

csync2 -xv

Now repeat the above again:

On node1 now do:

csync2 -fr /

Follow this with:

csync2 -xv


This will take a while as it is an initial sync and will be verbose.

If anyone knows why we need to run it twice, please comment.


On each server do crontab -e and set up a job to suit you, I use:

0-59/2 * * * * /usr/sbin/csync2 -x

on one server, and on the other:

1-59/2 * * * * /usr/sbin/csync2 -x

This means each minute, one server syncs then the other on the second minute, seems to work great for me.


If you wish to test:

On node one:

touch /var/lib/freeswitch/recordings/test

Give it a couple of minutes then on node two you should see the file:

ls /var/lib/freeswitch/recordings/test

root@node2:~# ls /var/lib/freeswitch/recordings/test
ls: cannot access /var/lib/freeswitch/recordings/test: No such file or directory
root@node2:~# ls /var/lib/freeswitch/recordings/test
/var/lib/freeswitch/recordings/test
root@node2:~#

Now remove the file from node two and give it another couple of minutes:

rm /var/lib/freeswitch/recordings/test

One node one:

ls /var/lib/freeswitch/recordings/test

root@node1:~# ls /var/lib/freeswitch/recordings/test
/var/lib/freeswitch/recordings/test
root@node1:~# ls /var/lib/freeswitch/recordings/test
ls: cannot access /var/lib/freeswitch/recordings/test: No such file or directory
root@node1:~#

All done

Csync2 is a very powerful tool and we have only touched the surface here. For example actions can be triggered upon file changes etc.

I encourage everyone to read http://oss.linbit.com/csync2/paper.pdf and if you do come up with any weird and wonderful configs, please share them with the community.

 

[Luis Daniel Lucio Quiroz]

I already published this guide, it complements yours

http://inside-out.xyz/technology/how-to-sync-files-among-your-cluster-with-csync2.html

The big advantage of csync2 is the little footprint, but the files gets very easily in dirty state when csync2 runs simultaneously on both servers. It needs babysitting.

 

[DigitalDaz]

Luis, I based my guide on yours

 

[Luis Daniel Lucio Quiroz]

You should put my link as reference as I do when I read things from other sources.

 

[Alessandro]

Hello @DigitalDaz is there a way to sync extensions and all PBX settings from node1 to node2 . These informations are stored in one or more paths?

Thank you @DigitalDaz

 

[DigitalDaz]

You wouldn't need to do this with csync, the db replication already takes care of that.

 

[Alessandro]

I tried this procedure:

1. i created an extension on node1
2. node1 with postres user: pg_dumpall -U postgres > /tmp/all.bak
3. scp copied all.bak to node2
4. nod2 with postreg user: psql -f /tmp/all.bak postgres

Then i had a connection problem with fusionpbx webinterface (wrong username)

5. i copied from node1 /etc/fusionpbx/config.php $db_password to the same file on node2 and web interface become available but i can't find the extension. Instead now i have all menu duplicated like attached.

Any idea? Thank you.

 

[ppwicho]

@DigitalDaz Just for reference: Do you have a guide for db replication for fussionpbx?.

Also thanks for this guide, I dind't know csync2 !!!

 

[DigitalDaz]

@DigitalDaz Just for reference: Do you have a guide for db replication for fussionpbx?.

Also thanks for this guide, I dind't know csync2 !!!

Look inside the clustering script

 

[krooney]

@DigitalDaz changes to my /etc/hosts file are not saving after reboot any ideas

 

[DigitalDaz]

I would guess at that you are using some cloud based vps. There will likely be some sort of cloud init, this is definitely just some mundane linux problem that the solution for doesn't lie here.

 

[krooney]

I would guess at that you are using some cloud based vps. There will likely be some sort of cloud init, this is definitely just some mundane linux problem that the solution for doesn't lie here.

You are right problem fixed thanks

 

[Andrew Pickett]

Hi - Ive run this tutorial before and had it working. Ive just set this up again but on running the first sync i get the follwing errors: -

ERROR from peer node2.local: Identification failed!
While syncing file /var/lib/freeswitch/recordings:
ERROR from peer node2.local: Identification failed!
While syncing file /var/lib/freeswitch/storage:
ERROR from peer node2.local: Identification failed!
Finished with 4973 errors.

Any ideas welcome. Ive ensure hostname are perfectly set and files on both are identical.

Thanks in advance!

 

[krooney]

Hi - Ive run this tutorial before and had it working. Ive just set this up again but on running the first sync i get the follwing errors: -

ERROR from peer node2.local: Identification failed!
While syncing file /var/lib/freeswitch/recordings:
ERROR from peer node2.local: Identification failed!
While syncing file /var/lib/freeswitch/storage:
ERROR from peer node2.local: Identification failed!
Finished with 4973 errors.

Any ideas welcome. Ive ensure hostname are perfectly set and files on both are identical.

Thanks in advance!

did you run sync twice

 

[DigitalDaz]

Can you ping node2.local from each machine?

 

[Andrew Pickett]

Yes tried twice as described in tutorial and can ping from each machine with no issue