Strange registration issue

[stin]

Hello,

I am having some strange times with FREESwitch and FusionPBX.

Originally, I had a remote phone which I was trying to get working over NAT as my internal ones worked fine (I have since found out - not from the horse's mouth - that there is a good chance I have a SIP ALG enabled on my SkyQ router without the option to turn it off!!) but it has developed into a bit of a brain-bender for me.

Some extensions will randomly not be able to make outgoing calls, presence subscriptions randomly become unknown or known, interdomain calls sometimes work with video, sometimes not and gateway registrations are unstable. It's as if it has a mind of it's own, or it's having a fit.

Maybe I'm just getting too old for all this mental running around now, but I can't find a pattern at all. Aside from a possible SIP ALG I can't get rid of.

It feels like there's too much 'intelligent' software in my network and they're all outsmarting each other - The phones, the PBX server, the router's ALG, the NAT-ing in general. I temporarily changed out my networking gear (not the router, unfortunately) to see if it was a switch, but that's when things just became even weirder.

Just in case anyone wants something to get their teeth into because I am ready to give it all up as another promising, yet ridiculous waste of my time. I feel that, while the standards we use are well designed, they are employed by relative Neanderthals, resulting in the minefield of inter-operability issues we have today.

I have attached some configs and logs to see if they'll help at all.

It's a shame there are no comprehensive knowledge resources for FS. It's not like I haven't informed the FS team of this. I keep thinking "If there's no accurate, current documentation, what actual use can I put this code to?" idk :/

I hope someone is able to shed a modicum of ambient light on the matter, if nothing else.

I would like to make this a viable business opportunity. I even have a guinea pig business for testing it with, provided I don't ruin him lol. If I can get this thing to at least behave more predictably, like it seems to do for some others, I would be over the moon. However, if I can't get it functioning at a usable level, I may well just have to give it up as poorly-judged idea.

Outgoing call log from extension 2001: (This is a good one - the domain name changes just in time for the ACL check and then changes back!)
2020-07-14 17:33:52.595928 [NOTICE] switch_channel.c:1118 New Channel sofia/internal/2001@sip.sntgroundworkspecialists.uk [c3eef6d6-c094-4562-8f7a-38691667809b]
2020-07-14 17:33:52.595928 [DEBUG] switch_core_state_machine.c:585 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Running State Change CS_NEW (Cur 1 Tot 22)
2020-07-14 17:33:52.595928 [DEBUG] sofia.c:10279 sofia/internal/2001@sip.sntgroundworkspecialists.uk receiving invite from 192.168.42.245:5060 version: 1.10.3 -release-15-129de34d84 64bit
2020-07-14 17:33:52.595928 [DEBUG] sofia.c:10373 verifying acl "domains" for ip/port 192.168.42.245:0.
2020-07-14 17:33:52.614993 [WARNING] sofia_reg.c:2930 Can't find user [2001@fs-pbx.sandalcomputers.co.uk] from 192.168.42.245
You must define a domain called 'fs-pbx.sandalcomputers.co.uk' in your directory and add a user with the id="2001" attribute
and you must configure your device to use the proper domain in its authentication credentials.
2020-07-14 17:33:52.614993 [WARNING] sofia_reg.c:1739 SIP auth failure (INVITE) on sofia profile 'internal' for [*9172@sip.sntgroundworkspecialists.uk] from ip 192.168.42.245
2020-07-14 17:33:52.614993 [DEBUG] switch_core_state_machine.c:604 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State NEW
2020-07-14 17:33:52.614993 [NOTICE] sofia.c:2432 Hangup sofia/internal/2001@sip.sntgroundworkspecialists.uk [CS_NEW] [CALL_REJECTED]
2020-07-14 17:33:52.634540 [DEBUG] sofia.c:1548 Channel is already hungup.
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:585 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Running State Change CS_HANGUP (Cur 1 Tot 22)
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:848 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Callstate Change DOWN -> HANGUP
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:850 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State HANGUP
2020-07-14 17:33:52.634540 [DEBUG] mod_sofia.c:453 Channel sofia/internal/2001@sip.sntgroundworkspecialists.uk hanging up, cause: CALL_REJECTED
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:60 sofia/internal/2001@sip.sntgroundworkspecialists.uk Standard HANGUP, cause: CALL_REJECTED
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:850 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State HANGUP going to sleep
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:620 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State Change CS_HANGUP -> CS_REPORTING
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:585 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Running State Change CS_REPORTING (Cur 1 Tot 22)
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:936 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State REPORTING
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:174 sofia/internal/2001@sip.sntgroundworkspecialists.uk Standard REPORTING, cause: CALL_REJECTED
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:936 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State REPORTING going to sleep
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:611 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State Change CS_REPORTING -> CS_DESTROY
2020-07-14 17:33:52.634540 [DEBUG] switch_core_session.c:1726 Session 22 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Locked, Waiting on external entities
2020-07-14 17:33:52.634540 [NOTICE] switch_core_session.c:1744 Session 22 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Ended
2020-07-14 17:33:52.634540 [NOTICE] switch_core_session.c:1748 Close Channel sofia/internal/2001@sip.sntgroundworkspecialists.uk [CS_DESTROY]
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:739 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) Running State Change CS_DESTROY (Cur 0 Tot 22)
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:749 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State DESTROY
2020-07-14 17:33:52.634540 [DEBUG] mod_sofia.c:364 sofia/internal/2001@sip.sntgroundworkspecialists.uk SOFIA DESTROY
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:181 sofia/internal/2001@sip.sntgroundworkspecialists.uk Standard DESTROY
2020-07-14 17:33:52.634540 [DEBUG] switch_core_state_machine.c:749 (sofia/internal/2001@sip.sntgroundworkspecialists.uk) State DESTROY going to sleep

If you need packet/SIP traces, ladder diagrams or more logs, I should be able to get some. I think there are some things perceived idiosyncrasies which are currently eluding me.

I was going to attach my FS config XML, but I am using FPBX on this server and don't know how to export or backup my config - I can't find a menu option for downloading/uploading it anywhere. The freeswitch.xml.fsxml file doesn't seem to contain my configuration changes. I gather FPBX updates a database instead of the XML files directly? If so, why didn't they choose to include a config backup/restore option in the Interface? So many questions.

Also, can anyone recommend a resource to learn about current Fusion PBX and FREESwitch versions in detail, please, preferably with a functional overview and categorised reference? It's hard without documentation and my social skills mean I regularly commit 'faux-pas' in forums without realising it, so I apologise if I have done that here. I have looked all over for consice and accurate documentation, but have found FS's site to be filled with marketing blurb and little to no technical information, including a blank section on video calls lol

Many thanks for any information or insights you can give me. I understand it's a forum, so I am not expecting an expert solution, but would like to know if there is an end in sight for this installation. I would like to use and promote this software if that is going to be possible.

Many, many thanks again.

 

[Adrian Fretwell]

It sounds like you are having a bad time...
Bottom line: You will always struggle if you cannot turn SIP ALG off. Treat yourself to a decent router.

You may be able to circumvent the SIP ALG if you get your endpoints to connect using TLS, but I'm not recommending this for you right now.

If you are going to have customers connecting to your service, you would be much better placed in a data centre.

The IP address that the domain name resolves to (5.70.xx.xx), is that the IP on the WAN side of your NAT?

It will help us if you can explain your system and network topology and also explain exactly what you are trying to achieve.

Documentation:
https://freeswitch.org/confluence/display/FREESWITCH/FreeSWITCH+Explained
https://docs.fusionpbx.com/en/latest/
https://tools.ietf.org/html/rfc3261

Also checkout FusionPBX dot com for training resources and membership benefits:
https://www.fusionpbx.com/

 

[stin]

Thank you very much for your input, Adrian. It is very much appreciated.

Apologies for the length of my posts, but I tend to think having too much information is better than not enough. Also, I haven't had anyone who I can talk with about technical stuff for a few years, which won't help much :/

It sounds like you are having a bad time...
Bottom line: You will always struggle if you cannot turn SIP ALG off. Treat yourself to a decent router.

Thank you for your sympathies It's one of the warmest welcomes to an on-line support forum I've ever had lol

I will be enquiring with Sky as to the possibility of replacing the Q 'router' with a real one. If that's not possible, I will be looking at changing ISP. The last resort would be renting a VPS for a few months or so to see if that improves the situation.

You may be able to circumvent the SIP ALG if you get your endpoints to connect using TLS, but I'm not recommending this for you right now.

Aaaah... Yeah... Certificates... The eventual goal is encrypted A/V SIP provision which I could sell to small businesses. Do you know if these certs/keys need to be a certain type, or will LetsEncrypt do for them? I know I do need to look into this aspect deeper.

Documentation:
https://freeswitch.org/confluence/display/FREESWITCH/FreeSWITCH+Explained
https://docs.fusionpbx.com/en/latest/
https://tools.ietf.org/html/rfc3261

Many thanks for the links. I have found the FreeSWITCH 'confluence' documentation to have large holes and missing explanations, which doesn't really help me. IMHO, it's also terrible for organised reference. The ball is firmly in the core team's court with that one, fully assisting their community is their choice :/

The FPBX documentation is better, but still lacks organised reference and full explanations - I am used to technical reference books and manuals. I would gladly write one for FreeSWITCH, but I am the one of the many who needs to read it lol

The only problem I have with using FreeSWITCH documentation with FPBX is the disconnect between the two - Where does the XML 'file' go in FPBX's interface?

Now I mention it, my new dialplan shows nothing in the FPBX interface, only the raw XML in the XML editor. There was a 'de-sync' between the interface's XML table and the XML contents of the 'file', so I removed the rows from the interface and edited the XML directly. FPBX never recreated the list/table from the XML file. I wonder if this is a known issue with FPBX, or if it is seen as an issue. idk

Do you think they would train me up so I could write the manual for the community, or does that conflict with their interests in providing commercial support? Hmmm lol

Also checkout FusionPBX dot com for training resources and membership benefits:
https://www.fusionpbx.com/

Well, the initial intention was to do some research, read up on it, do some mock deployments and test the software, then get income from renting the service. I have since found out that the reading-up only happens when there are things to be read. But, I will check out the pricing. It's gotta be better than the rumoured $5000 pcm from FreeSWITCH lol IMHO uninformed users, who do not know how to use their software, are their revenue stream lol

The IP address that the domain name resolves to (5.70.xx.xx), is that the IP on the WAN side of your NAT?

Yes. Sky have a couple of strange ranges. Recently, my public IP was something like 95.x.x.0! Not sure how that one works - AFAIK that's a subnet/network ID lol

(I used to use LogMeIn Hamachi VPN on Windows years ago and wondered why it stopped working. 5.0.0.0/8 was the range LogMeIn used for that. Sky must own it now. idk)

It will help us if you can explain your system and network topology and also explain exactly what you are trying to achieve.

The final idea is: Provide secure (E2EE?) SIP A/V calls and conferencing to small, local businesses. Looking into Skype/Zoom interconnectivity in future.

Currently, I am testing the software (and my ISP's provision, apparently) to get a feel for it. I have installed FS 1.8 (IIRC) via bare-metal config files, so FPBX feels more 'removed' in that regard, but I am mostly enjoying it.

I would love to draw you a network diagram: That leads me onto something else (indirectly related) I have been trying to get hold of for years, now - Does anyone know of a usable diagramming application for Linux, along the lines of M$ Visio, with a library of symbols/images/clipart and connectors with good routing code? I have tried a few over the years, but they each lack something vital to creating a good diagram (guides, alignment, connector routing, etc) and have had to resort to hand-drawing network diagrams

I do have reservations regarding how FPBX implements configuration changes in FreeSWITCH. API? Generation of a freeswitch.xml.fsxml file from a DB? idk - I can't find a technical descriptions or details of how it actually works 'under the bonnet' :/

And it looks a little bit like this at the moment:

[E20(dn2), 192.168.0.250] <--eth--> [SKY Q ROUTER, DHCP WAN IP] <-- NAT+P/Fwd --> {SKY NETWORK} <-- NAT+P/Fwd --> [SKY Q ROUTER] <--eth--> [PBX, 192.168.42.249] <--eth--> [2 x E20s at (dn1) 192.168.42.244 + (dn2) 192.168.42.245]

Please excuse the low-tech approach. I am no ASCII artist, either lol

The remote Tandberg/CISCO E20 and one local E20 are on one domain (dn2);
One other local E20 is on a different domain (dn1).

Whilst I am no expert in SDP/SIP/RTP protocols, or even some of the more esoteric flavours of NAT algorithm, I do have decades of non-professional software development and network, server and desktop support experience. I started out with Microsoft and Cisco certifications (now completely worthless lol) but have received no formal training in SIP/VoIP deployments.

The connection between the routers (thankfully?) is a single(?) ISP network. I think any issues I am having are likely caused by Sky and their consumer 'router'. (I wouldn't even consider it as consumer-level equipment, they've hidden ALL the options! How do home customers live like this?! lol)

I have added a dialplan rule for inter-domain dialling, based upon a forum post for an inter-tenant dialplan but with a structural modification which provides inter-domain dialling codes, prefixed with *5:

<extension name="interdomain_call" continue="false" uuid="b299e8c7-e6db-474d-93ce-cba73926dcea">
<condition field="destination_number" expression="^(\*5(\d{2})(\d{4}))$">
<action application="set" data="interdomain_domain_id=$2" inline="true"/>
<action application="set" data="callee_id_number=$3" inline="true"/>
<action application="set" data="sip_h_X-accountcode=${accountcode}"/>
<action application="set" data="call_direction=local"/>
<action application="set" data="hangup_after_bridge=true"/>
<action application="set" data="inherit_codec=true"/>
<action application="set" data="effective_caller_id_name=${caller_id_name}"/>
<action application="set" data="effective_caller_id_number=${caller_id_number}"/>
<action application="set" data="ignore_display_updates=true"/>
<action application="set" data="continue_on_fail=true"/>
<condition field="${interdomain_domain_id}" expression="01" break="on-true">
<action application="set" data="domain_name=fs-pbx.sandalcomputers.co.uk" inline="true"/>
<action application="set" data="domain_uuid=eb464dcb-7ee0-4892-84ab-6c6a5ac3390d" inline="true"/>
<action application="transfer" data="${callee_id_number} XML ${domain_name}"/>
</condition>
<condition field="${interdomain_domain_id}" expression="02" break="on-true">
<action application="set" data="domain_name=sip.sntgroundworkspecialists.uk" inline="true"/>
<action application="set" data="domain_uuid=8929178a-b5ed-40d8-92be-ab5452ab2fea" inline="true"/>
<action application="transfer" data="${callee_id_number} XML ${domain_name}"/>
</condition>
<condition field="${interdomain_domain_id}" expression="03" break="on-true">
<action application="set" data="domain_name=sip.last-domain.uk" inline="true"/>
<action application="set" data="domain_uuid=00000000-0000-0000-0000-000000000000" inline="true"/>
<action application="transfer" data="${callee_id_number} XML ${domain_name}"/>
</condition>
</condition>
</extension>

(the 3rd domain is just a place-holder)
I am sure a dialplan expert will tear this abomination to shreds in seconds few.

Apologies if I have missed something out. Is there anything else you would like to know?

Best regards and many thanks.

[Edits: To correct errors and expand on a few points.]

 

[Adrian Fretwell]

The last resort would be renting a VPS for a few months or so to see if that improves the situation.

Don't make this your last resort, it doesn't cost that much and will take a lot of variables out of the equation. There are many people out there running successful deployments on AWS and Digital Ocean etc... We run our own bare metal in the DCs and use XCP-ng as our hypervisor, all our deployments are virtual machines.

Try to make your first test bed as simple as possible and understand what is going on. You will need to understand SIP and SDP and be able to read packet captures in order to diagnose faults.

I don't have too much time right now, so I will come back on some of the points later, but to answer your immediate questions:

Where does the XML 'file' go in FPBX's interface?

Don't get too hung up on this. FreeSWITCH has an embedded language called Lua. Lua is used quite a lot for many different things in the FusionPBX front end. Lua is also used to serve XML information from the database into FreeSWITCH using XML Handlers. Files for points of reference are:
/etc/freeswitch/autoload_configs/lua.conf.xml
/usr/share/freeswitch/scripts/app/xml_handler/indes.lua

Describing exactly ho wit works is beyond the scope of this message but this mechanism coupled with messages sent to the FreeSWITCH event socket is how dialplan configuration and updates take place.

Do you think they would train me up so I could write the manual for the community,

A community based manual to supplement the existing publically available documentation would be a great idea, especially if it gave lots of examples for settings. In the early days I would have struggled without this forum. I think @DigitalDaz has some documentation that someone put together, but we do need to be careful not to undermine the membership offerings that support the Fusion team.

ack something vital to creating a good diagram

I tend to get by using the draw package in LibreOffice.

How do home customers live like this?

It can be a minefield, if we provide a hosted solution, then we normally install our own ADSL/Fibre and router so we have as much control as possible (We are a direct Openreach B2B customer).

I have added a dialplan rule for inter-domain dialling

Get the basics working before doing anything more complicated. You can enter XML directly in the Fusion interface but if you are new to it stick to doing everything in the GUI with the Dialplan/Dialplan-Details pages.

I hope that helps a little for now.

PS: Its great, and I like to see plenty of information in requests for help, but if they get too large it does put people off reading them.

Cheers,
Adrian.

 

[stin]

You will need to understand SIP and SDP and be able to read packet captures in order to diagnose faults.

I've done a few packet traces before and I reckon I can read them well enough. I have recently scanned some info on SDP and SIP, I drew out some ladder diagrams to better help me see the conversation and relevant SDP media descriptors to check ports. It threw me for a second when I saw my phone's part of the conversation suddenly continue from the router, but realised that it was being NAT-reflected as my phone, inside, started to send it's data to the external IP.

Don't get too hung up on this. FreeSWITCH has an embedded language called Lua. Lua is used quite a lot for many different things in the FusionPBX front end. Lua is also used to serve XML information from the database into FreeSWITCH using XML Handlers. Files for points of reference are:
/etc/freeswitch/autoload_configs/lua.conf.xml
/usr/share/freeswitch/scripts/app/xml_handler/indes.lua

Describing exactly ho wit works is beyond the scope of this message but this mechanism coupled with messages sent to the FreeSWITCH event socket is how dialplan configuration and updates take place.

Thank you very much for this, honestly. I think I'm a bit weird in that I 'feel' code and software architecture, it's almost like a tangible sensation for me and the void or floating sensation I get when contemplating issues with FreeSWITCH/FusionPBX is quite unsettling for me. Now I know that, It doesn't feel as distant from FreeSWITCH. Also, I can now go and learn about Lua and how it behaves

Get the basics working before doing anything more complicated. You can enter XML directly in the Fusion interface but if you are new to it stick to doing everything in the GUI with the Dialplan/Dialplan-Details pages.

Believe me, it started-off simply, but I fear a lack of knowledge compounded the situation.

I am just very aware of my lack of complete knowledge of FPBX and frustrated by being unable to satisfy my ignorance with available documentation. I don't think one can ever underestimate one's own lack of knowledge.

VirtualBox - It isn't the best platform and it has many ISSUES, one of which is stopping me use UPnP in FreeSWITCH.

I'll flatten the vmachine and reinstall FS & FPBX form scratch. I was using the FusionPBX install instructions at https://docs.fusionpbx.com/en/latest/getting_started/quick_install.html

(I'm still not sure what the random pipe is for in step 1 of the instructions after executing the script, though.)

I had a quick read of the Fusion PBX training schemes. The language use does disturb me somewhat, especially the penalty for missing a payment. It actually comes across as somewhat spiteful lol. The 12 months at the most basic level, $100 (~£80), isn't that feasible for me, unless I can just get the one month. Is this the same prices for "Someone that loves to tinker with technology" as it is for "Major companies all around the world"? I enjoy tinkering as much as the next person, but that's relatively serious cash for a little tinkerer IMHO lol

I did find the words "Think you know FusionPBX already without training you will be surprised what you didn't know" quite funny in a way - I know full well that I can't find out enough and would not be surprised by anyone's lack of knowledge if they had solely relied on the available materials. I guess I would have a different approach to this sort of thing, but then maybe that's why I don't own two Bentleys.

As I understand good coding practice (uncommon terms today) , the code should be mostly structured comments, pseudo-code, DFDs and documentation, which further confuses me as to the lack of documentation. Or am I too old-fashioned? lol

Sorry, I do moan a lot at the self-proclaimed 'programmers' I've seen over the last decade - "They don't make 'em like they used to, Sonny-Jim." lol

I really do appreciate you taking the time out of your day to answer my cry for assistance, Adrian.

Thank you.

Best regards,

James

 

[vespaman]

Hi James
I wasted a lot of time and money on hosting my own server. Ive used both vultr and digital ocean and some others. They are ideal for what you want to do and imho a better platform to learn/play with fpbx or any multitenant pbx. I have run a small 4 domain fpbx on a Vultr VPS in Sydney for paying customers here in NZ for over a year now. I started like you and then just carried on. Yes the fpbx documentation could be better and the $100 monthly membership is hard to justify if you just want to play but perservere as fppx is stable and reliable once you sort out the little things. Some good, experienced help is available on this forum.
Good luck with it.

 

[KitchM]

@stin
Wow, what a great example of a struggling fellow user! I totally sympathize and want you to know that you have not done any faux-pas at all. I also understand the problem with having long posts and not wanting to miss anything. We should all feel welcome to vent our frustration among sympathetic ears. We feel your pain.

Yes, the documentation leaves a lot to be desired. I wish to help with that.

In any case, my strong suggestion is to start very simply. Forget about FreeSwitch and just focus on FusionPBX. The reason I say this is because it is the very reason for its existence. You should not need to work at the switch level. Stick with the GUI!

As Adrian stated, change the router. You are on the right track to see if your ISP has a better router for you or you need to find a new ISP.

Start there first and then come back here with the results. I'm interested in what you find out.

Good luck!