Freeswitch and endpoint NAT detection

[DudeInMyrtleBeach]

It's been a while since I installed a new version of Fusion, and I don't recall having to do anything when Fusion itself had a public internet IP and the endpoints were behind a NAT.

NAT isn't working. I can have 2 extensions registered sitting side by side, and one can't call the other.

It seems I've missed something, and I've read everything I can get my hands on... On all of my production systems, when you go to the Registration page, you can see that endpoints have a status of 'Registered(UDP-NAT)(unknown)'. That's not what I see with the current version of Fusion -- What is see is 'Registered UDP' with no mention of NAT.

Ok... Just set DEBUG in the SIP profile to 1 and found this: (replaced the actual IP's first few octets with 111...)

2023-04-08 19:41:31.011432 97.20% [DEBUG] sofia_glue.c:3422 IP 111.111.111.83 is on local network, not seting NAT mode.
2023-04-08 19:41:31.311356 97.20% [DEBUG] sofia_glue.c:3422 IP 111.111.111.84 is on local network, not seting NAT mode.

The FusionPBX sits on 111.111.111.85 -- subnet is /29

The other 2 subnets have networks behind them, and SHOULD have NAT enabled.

How do I force NAT?

 

[falk]

its firewall setting

 

[DudeInMyrtleBeach]

its firewall setting

I wish it was. I could fix that.

localnet.auto is being dynamically built by Fusion, and there doesn't appear to be any way to (easily) change that. Fusion examines it's own IP address, then sees that the 2 other IP's are on the same subnet (/29) and 'assumes' that they are local networks and refuses to apply a NAT policy for the endpoints. In fact, they are separate businesses with their own firewalls sharing a common internet service into the building.

 

[falk]

try to config manual NAT in the endpoint

 

[DudeInMyrtleBeach]

I did. No matter what I do, Fusion sees endpoints behind these 2 IP's as local and does not apply a NAT policy.

 

[Davesworld]

Do you get incoming calls to these extensions from the outside?

I have dealt with this as well but it results in more than just extensions not being able to call each other. It also prevents the extensions from ringing when calls from outside come in. My FusionPBX sits on its own external static IP with my fiber optic services. My firewall/router also has its wan as a static IP in the same subnet and the sip phones are behind THAT firewall. Incoming calls do not ring the extensions, the extensions cannot call each other and yes, nat does not show up in the registrations. The only workarounds that have worked since I am using OPNsense which allows me to put all my sip phone ip addresses under an alias and route those out a different gateway, is either routing them over a VPN which is not ideal or what I presently am doing, running those out a microwave internet link that I still have from before fiber was available.

Yours is the first in a long time where I have seen someone having a lack of nat in the registrations like I have seen.

 

[hfoster]

rfc1918.auto - RFC1918 Space
nat.auto - RFC1918 Excluding your local lan.
localnet.auto - ACL for your local lan.
loopback.auto - ACL for your local lan.

Your external profile probably has localnet.auto in for the local-network-acl, and from your subnetting design that is on the local network technically speaking. I would probably change it to rfc1918.auto or nat.auto

 

[hfoster]

The author of this thread and the error messages:

2023-04-08 19:41:31.011432 97.20% [DEBUG] sofia_glue.c:3422 IP 111.111.111.83 is on local network, not seting NAT mode.
2023-04-08 19:41:31.311356 97.20% [DEBUG] sofia_glue.c:3422 IP 111.111.111.84 is on local network, not seting NAT mode.

 

[DudeInMyrtleBeach]

rfc1918.auto - RFC1918 Space
nat.auto - RFC1918 Excluding your local lan.
localnet.auto - ACL for your local lan.
loopback.auto - ACL for your local lan.

Your external profile probably has localnet.auto in for the local-network-acl, and from your subnetting design that is on the local network technically speaking. I would probably change it to rfc1918.auto or nat.auto

I tried rfc1918.auto and nat.auto. It didn't matter. What I ended up doing was moving the PBX instance to Vultr. That fixed the initial issue.

Now, unfortunately - I have another.

The ISP is doing what they call 'Enterprise NAT' or 'Carrier NAT' - which means that they're too cheap to shovel out an IP address to all their customers. They also apparently appear to be using some kind of SIP 'helper' on this godawful mess that is telling Fusion to not enable NAT for the endpoints in question. Here's the layout

Phone ->Calix fiber 'gateway'/'router'/'thing' that is doing NAT -> local ISP that is doing yet another 'enterprise' nat -> INTERNET -> MY FUSION PBX

Ip addresses look like this:
Phone/Calix internal network is 192.168.50.x -> ISP 'Carrier NAT' conglomeration IP is 100.100.123.234 -> ISP Firewall presents itself as '67.209.30.20' to the world.

It might be a double-nat scenario. It might be that the local ISP has some kind of ALG screwing things up.

Believe it or not, audio seems to be working. But - If I call from one extension to another and immediately hang up, the other extension will ring *forever*.

Here's a sample from the Fusion log...

2023-04-11 20:52:08.724191 99.10% [DEBUG] switch_core_state_machine.c:581 (sofia/internal/7345@xxpbx.mydomain.net) Running State Change CS_NEW (Cur 1 Tot 125)
2023-04-11 20:52:08.724191 99.10% [INFO] sofia.c:10453 sofia/internal/7345@xxpbx.mydomain.net receiving invite from 67.209.30.20:51616 version: 1.10.9 -release 64bit call-id: 1821768575-5060-3439@BJC.BGI.FA.DA
2023-04-11 20:52:08.724191 99.10% [DEBUG] sofia.c:10547 verifying acl "providers" for ip/port 67.209.30.20:0.
2023-04-11 20:52:08.724191 99.10% [DEBUG] switch_core_state_machine.c:600 (sofia/internal/7345@xxpbx.mydomain.net) State NEW
2023-04-11 20:52:08.724191 99.10% [DEBUG] sofia.c:2419 detaching session bc42708a-d1fb-4648-881f-2937a6feffe9
2023-04-11 20:52:09.044206 99.10% [DEBUG] sofia.c:2532 Re-attaching to session bc42708a-d1fb-4648-881f-2937a6feffe9
2023-04-11 20:52:09.044206 99.10% [INFO] sofia.c:10453 sofia/internal/7345@xxpbx.mydomain.net receiving invite from 67.209.30.20:51616 version: 1.10.9 -release 64bit call-id: 1821768575-5060-3439@BJC.BGI.FA.DA
2023-04-11 20:52:09.044206 99.10% [DEBUG] sofia.c:10547 verifying acl "providers" for ip/port 67.209.30.20:0.
2023-04-11 20:52:09.044206 99.10% [DEBUG] sofia.c:11668 Setting NAT mode based on via received
2023-04-11 20:52:09.044206 99.10% [DEBUG] sofia.c:7487 Channel sofia/internal/7345@xxpbx.mydomain.net entering state [received][100]
2023-04-11 20:52:09.044206 99.10% [DEBUG] sofia.c:7497 Remote SDP:
v=0
o=7345 8000 8000 IN IP4 192.168.50.30
s=SIP Call
c=IN IP4 192.168.50.30
t=0 0
m=audio 5004 RTP/AVP 0 8 4 18 9 97 2 123 101
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:4 G723/8000
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=no
a=rtpmap:9 G722/8000
a=rtpmap:97 iLBC/8000
a=fmtp:97 mode=30
a=rtpmap:2 G726-32/8000
a=rtpmap:123 opus/48000/2
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=ptime:20

Everything endpoint on this list should be using NAT. Only those behind a static IP - 216.99.xx.xx are working. Those other IP's are part of the ISP's 'Enterprise/Carrier' NAT.



If anyone has any ideas on how to deal with this, I'll buy you a case of beer and dinner. And free golf if you ever come to the Myrtle Beach area.

 

[Davesworld]

I'm in the same situation as I indicated and I have experienced it in three similar situations. First, I was on assignment in Moses Lake, Wa, and had fiber and static IPs, the sip phones were on a LAN, and the firewall/router controlling that LAN used one of the static IPs on WAN. My FusionPBX used another static IP but all 5 static IPs were in the same subnet. Wound up just using a Vultr instance.

Bought a house and ended the assignment, at the new house only Microwave internet was available which worked as good or better than any DSL I ever had, I got 5 static IPs on the same subnet, and of course, the same issue.

Fiber Optic became available 5 months later, got 5 static IPs but all were assigned in the same subnet, same problem, so I had the Microwave internet, tiered down to the lowest bandwidth but they let me keep the static IPs because there wasn't much demand for them in my area so I wound up setting up a second WAN over the Microwave link and pointing the sip phones to that WAN and it solved the problem but it is not an acceptable solution since I am using two different Internet providers to do it. I have also sent the sip phones out a VPN and had that work but commercial VPNs are unreliable, you cannot count on one server to work consistently and yes, they too cost money.

If only co-location was as inexpensive as a Vultr instance with 1GB of ram and a single core.

As far as changing the external ACL, external is for the SBC or provider to communicate with the server, the sip phones are registered to the local profile. On external, NAT is usually not involved.

 

[hfoster]

Other than the NDLB settings for FreeSwitch. You might be looking at one of those scenarios where you have to deploy a VPN on the FusionPBX server and creating a SIP profile for VPN connecting phones. No experience with Grandstream phones here, but OpenVPN has saved our bacon with Yealinks when a customer refused to get one of our own circuits.

 

[DudeInMyrtleBeach]

Got this squared away with Mark's help. I created a new SIP profile and changed the port. Apparently, the local ISP is messing with 5060. Once I did that, NAT worked.

 

[Davesworld]

Got this squared away with Mark's help. I created a new SIP profile and changed the port. Apparently, the local ISP is messing with 5060. Once I did that, NAT worked.

Interesting. May I ask what ISP? In my case it does not matter whether TLS over 5061 is used or udp over 5060.

 

[DudeInMyrtleBeach]

Focus Broadband in Brunswick County, NC.

I'm not using 5060 or 5061 anymore on this PBX.

 

[hfoster]

Classic.

ISPs, Router Manufacturers and Firewall manufacturers all to often think they need to 'fix' SIP by messing with 5060 UDP packets without realising phone systems fixed it years ago.