Freeswitch(fusionpbx) performancs on WMware ESX vs Proxmox

[Maani]

Hi
At first glance it seems because of container support proxmox should have higher performance but Proxmox runs over another debian kernel but wmware esx runs on metal.
Please share your real world test results of freeswitch(fusionpbx) performance on wmware esx vs proxmox. Please tell detail about hardware specifications, caps...

 

[djzort]

If you run in a Proxmox container, the container uses the hosts kernel. Its just the Linux kernel on metal.

If you run VMware then there is obviously the VMware hypervisor running on the metal, and your kernel running on top of that.

KVM on Proxmox is differently structured than VMware, the Linux kernel hosts another Linux kernel running in a VM, but has conceptually the same draw backs.

One major advantage of the Proxmox container is that you dont have to worry about clock drift, and adding memory/cpu/etc is more easily done on the fly.

I personally run FusionPBX on Debian 8 inside Proxmox container. It runs perfectly fine - hardly uses any CPU and just to be sure I have that containers CPU priority 4x higher than the others running on the same host.

 

[DigitalDaz]

And that's how the theory goes.....

You can imagine my shock, when in performance tests, I found ESXi handled more than LXC on Proxmox.

I think the reality is, virtualisation is that good these days that it doesn't really matter until you get under serious load you won't see a difference and if yor load is that high you should probably be baremettaling it anyway.

 

[djzort]

Probably something to do with tunables. Linux out of the box is tuned for... well nothing TBH.

Personally, I consider VMware to be a legacy product. Virtualization is now (as it was originally) a native part of OS's

 

[elkato]

I have that containers CPU priority 4x higher than the others running on the same host.

Hi djzort, how do you get that?
I'm using last Proxmox 5.2, and incresed CPU units to Fusion LXC to 16384. The same for virtual router (pfsense in KVM on same physical server) with 16384. About 3 Elastix VMs with 4096 and the rest of VMs/LXC (about 6 more) with default 1000 units. Overall CPU of server is 15% usage, full of RAM free and IO ok too.

Fusion LXC is running with 4 cores, less than 5% usage, 50 domains and peaks of 20 simultaneous calls.

But still getting clock drifts and we can hear some ticks on a call

 

[bcmike]

Old thread but I'm planning on using FusionPBX on Proxmox KVM. I've had issues in the past with leaky containers and choose to go full hypervisor. I haven't put the test boxes under load yet.

Has anybody had any experience with this type of setup?

 

[DigitalDaz]

Many of us are running KVM with Proxmox, I have for the last 2 years plus

 

[djzort]

Im not certain what youve experienced with 'leaky containers', but i have fusionpbx running in a container on proxmox and it runs perfectly.

I have it on an atom C2550, with the container having just two cores, cpuunits=2048 (other containers are 1024) and just 2gb of ram. postgresql uses the most memory, followed by freeswitch, etc for a combined usage of less than 256mb. there are currently 29 registered devices all provisioned out of fusionpbx and a reasonably complicated configuration of huntgroups, timers, parking etc. the trunk is G711 with most of the handsets on the same premise using the same codec, so they are passing through. there are two remote sites which make up about 10 of the extensions. their handsets are doing G729 and freeswitch (fusionpbx) is transcoding them. fwiw the handsets are mostly snom 710 and snom 720s (some with D7), a panasonic cordless dect with three handsets/extensions, and some grandstream HT801 and HT802's running fax machines (a lot of PO's and what not are stilling coming in and out via fax... *shrug*)

I dont know what a "high" call volume is, but the calls / business hours is about a call every 2 minutes with an average call duration of 90 seconds. these may be severally skewed by heavy use of call parking.

 

[Ian Oakes]

I run almost entirely in containers and have had very few issues I could blame on LXC, or proxmox, on high volume servers too. IO wait, and proxmox annoying cache usage stuff were about all I encountered, other than actual hardware failure.

 

[DigitalDaz]

I get crashing containers running next to nothing with LXC. Two with just nginx on, one serving my repo, the other serving cgrates site just become unresponsive and need rebooting.

 

[Ian Oakes]

I just havent seen it, and you've seen how much stuff I have in LXC - both traffic and density on the hosts.

 

[DigitalDaz]

Yep, I know, I cannot understand it.

 

[bcmike]

I understand Asterisk is a completely different animal but we used to have a lot of IO and clock problems with it in LXC containers, also at times security issues but that was many moons ago. So now I'm gun shy and will trade a bit of performance for the abstraction I get from KVM.

Djzort thanks for the dimensioning info. I'm basically looking for stories on what resources are committed to concurrent calls and/or mutli domain setups.

Thanks for the info so far.

 

[djzort]

DigitalDaz are you running CentOS/RHEL in a container? You may need to set "vsyscall=emulate" in your proxmox kernel options. See also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847154 and many other links that google will provide on the topic.

Regarding performance, the Linux kernel ships tuned for high performance in no known scenario (i.e. it's no good for desktops, it's also no good for servers, what's it tuned for?)

Redhat have finally realized this and released some tuning tools to help. Other areas of the kernel are autotuning more (some network settings). But in general its still up to you to install settings that unleash your hardware without overwhelming it.

I have a non public collection of notes regarding every setting i have ever found useful. However it links to some of these pages which cover most of it:

• https://wiki.mikejung.biz/Ubuntu_Performance_Tuning
• https://wiki.debian.org/SSDOptimization
• https://wiki.debian.org/KVM#Between_VM_host.2C_guests_and_the_world

Disks especially aren't tuned well by default. In general you should increase (possibly max out) the read ahead and increase the nr_requests quite a bit.

This is departing from the original topic though.

If youre going to run Linux on KVM be sure to look in to the ptp_kvm driver, it will help enormously with timing and clock drift. Redhat as some suggestions based on their kernels (so ymmv on other linux) at https://access.redhat.com/documenta...anagement-required_parameters_for_rhel_guests

 

[djzort]

bcmike im curious to know what sort of size you are looking at?

 

[bcmike]

I'm in the build it and they will come stage but I'm trying to get a handle on what to expect.

I'm thinking a server with twenty domains, 500 registered phones and maybe 75 to a hundred concurrent calls at peak. All calls are G711. no video. may also have to mux a couple conferences.

 

[djzort]

If you would need more than 8GB of RAM i would be surprised, 4GB is probably more than enough.

I would lean towards 8 cores and see how that goes, although 4 is probably fine.

 

[djzort]

If there are other containers or vm's running, make sure you set the cpuunits appropriately so that the pbx gets priority over other things which are less time sensitive (i.e. file shares)

 

[DigitalDaz]

@djzort I have never even looked at these tuning options. Being as I am using Proxmox which is a virtualisation product I made the assumption that they would do this for you.

 

[djzort]

Many of them are hardware dependent and YMMV.

Saying that I do find that Proxmox is a bit thin on details like those above.