Advice on SMS gateway...

[micah.quinn]

Hello,

I've been working part-time for several weeks trying to connect SMS messages from our carriers (Bandwidth and Telnyx) to a FusionPBX instance in an attempt to accomplish a few things:

* Allow users to send/receive text messages using softphones
* Allow users to send/receive text messages using a web interface
* Allow users to receive text messages via e-mail

While I have part of this working, it required an incredible amount of debugging and fiddling with the Fusion SMS app. I now have it receiving SMS messages and e-mailing users, but something is broken when trying to deliver to endpoints. After exerting so much energy troubleshooting the simple part of things, I'm second guessing my approach and wondering if I need to have a sanity check on the whole attempt. Either I'm really doing this the hard way or this is simply the current state of affairs with SMS and FusionPBX.

Has anyone successfully setup SMS messaging, send/receive, with Telnyx and Bandwidth such that messages can be both sent and received by endpoints AND delivered via e-mail AND sent/received via a web interface? (i.e. FusionPBX app "messages")

 

[voipBull]

Hello @micah.quinn, sorry I have no help to offer here but have a question for you if I may. Are you using both the SMS app and the Messages app? or just the Messages app alone to accomplish all the tasks you've listed above?

 

[micah.quinn]

I'm using both SMS and Messages. Messages appears to be a way to simply receive/send via the web interface and SMS is gateway that routes messages.

 

[voipBull]

hmmm, for some reason I was under the impression that Messages app is a replacement for the SMS app, and should be able to be used to send/receive SMS's via the FusionPBX WebUI Messages application and also via extensions registered on SIP softphones that are capable of sending/receiving messages.

@markjcrane apologies for tagging you directly into this thread, but if you have a minute, can you please clear this doubt for me.

 

[micah.quinn]

Apologies. Bull, you are correct that messages does both inbound/outbound and UI. This means the SMS is not necessary when using Messages.

I've managed to get inbound SMS with messages (The documentation is not clear that the callback URL is supposed to be .../app/messages/index.php) And I'm seeing that a new message is received in the UI. I also hear the notification in the browser. However, I do not see the message contents (body) in the web UI. I've verified that the database (v_messages) does indeed populate correctly. I'm also getting an Invalid Profile when messages attempts to deliver to endpoints. I believe that is because the sofia profile name is not being prepended correctly to the destination. (i.e. internal/1000@sub.domain.tld)

I'll keep working on it.

(P.S. The messages app seems much more up to date with current providers. Much appreciated!)

hmmm, for some reason I was under the impression that Messages app is a replacement for the SMS app, and should be able to be used to send/receive SMS's via the FusionPBX WebUI Messages application and also via extensions registered on SIP softphones that are capable of sending/receiving messages.

@markjcrane apologies for tagging you directly into this thread, but if you have a minute, can you please clear this doubt for me.

 

[voipBull]

Maybe a selfish ask on my end, but when you get this to work, will you please share the steps/instructions you followed that are not very well covered in the documentation. It'll be a good kick start for me or anyone else who may want to setup the Messages app in the future.

 

[micah.quinn]

I can provide a summary of the steps here once I have everything working. Here's a brief update on my current status:

• Messages is delivering to endpoints correctly; Invalid Profile was from a different test.
• It looks like the SQL query for numbers in "message_contact.php" is stripping out the "+" from numbers which was causing issues when trying to display the messages. I modfied the line to not strip out the + and the messages started showing up. I'm not sure why the SQL query is arbitrarily stripping out pluses, but this seems to be incorrect:

  //      $sql .= "               select distinct(REPLACE(message_from, '+', '')) as number from v_messages \n";
          $sql .= "               select distinct message_from as number from v_messages \n";

• Eventhough I hear the new message notification in the browser and the latest message under the contact number correctly updates, the main conversation thread goes blank. If I click on the contact again, the thread frame updates and shows the latest messages. This seems related to the "+" character issue as the hidden input value for "message_to" has the + stripped out. I'm assuming that value is used on state change to update the messages.

Perhaps there is something in my configuration that is just not set right to handle the leading plus. It feels like the plus should be getting stripped out on inbound message receipt, but I'm not sure how to affect this.

UPDATE: If I instead strip out the + in index.php, everything else seems to work properly:

    $message_from = get_value($message, $setting['message_from']);
    $message_from = preg_replace('/\+/', '', $message_from);

 

[micah.quinn]

I'm going to post a quasi-final update on this here to close out this thread. I was able to get Messages to work sufficiently enough for me to test/evaulate if it will work for our needs.

Currently I have the following working:

* Send/receive from web UI
* Send/receive from softphone endpoint
* Send/receive from hardphone endpoint

In addition to the the changes in my previous post I added some code to be able to support multiple inbound carriers based on user agent rather than IP address. Since our server exists behind a reverse-proxy, all originating IPs appear to come from the reverse proxy. I can then control access via the reverse proxy. A less disruptive change would probably be to configure the webserver to use the "real IP" header.

Sending messages through the web UI works, but is buggy. You cannot simply reply to a text message using the "Enter Response..." text field. Instead, you have to click "+ NEW MESSAGE" and enter all the associated information, then hit send. I'm sure it's just a bug. The code does not appear to be populating the reply phone number correctly in a hidden form value. There also seems to be no way to select from contacts when composing a new message. There appears to be no delete option for messages/threads.

My experience leads me to believe that, while a vast improvement over SMS, Messages still has some issues that need to be addressed:

• The documentation does not mention that index.php is the endpoint for carrier webhooks
• Handling of + in a routing a number needs some work. (There's a bunch of adhoc code for doing various things with plus or %2B.) It's not clear anywhere in the UI if I need to use a country code or not when addressing numbers.
• Telnyx V2 messaging API doesn't seem to work properly with Messages. Messages sent using this API store the JSON data in the text field of the database and nothing else. This leads me to believe it's a parsing issue, but I didn't take the time to debug it.
• Fix the in-thread reply form in the web UI
• No way to delete messages/threads
• No integration with contacts for new messages

Feel free to request any details that might be helpful.

 

[johnnyorange]

I realize this is a tremendous ask @micah_quinn, but if you would be so kind as to post the steps and code you utilized in order to get the messages app working for telnyx to deliver sms to endpoints?

On a totally new 5.3.3 fusionpbx install with tls and telnyx voice inbound/outbound working beautifully.

In the messages app, I see inbound messages there in the log, but they never get delivered to endpoints (groundwire in my case).

To your point, I agree that the json response is not being parsed by the messages/index.php and thusly not delivered to endpoints.

the issue that is a showstopper for me now is outbound sms through telnyx, and I believe my issue is in chatplan.xml in that it is not formatted set up to work.

that said, for the messages app, am I correct in assuming that freeswitch and chatplan.xml are being called by the messages app?

or am i barking up the wrong tree?

 

[datafanatics]

Oh how I would love to send/receive SMS using a softphone like Groundwire. I have the SMS app installed in FusionPBX presently and can successfully get inbound SMS sent via email, but beyond that I just haven't found the appropriate documentation that would allow me to tie in Groundwire (or any other softphone) to do both inbound/outbound.

 

[micah.quinn]

@datafanatics I would imagine that Groundwire would work just like any other softphone using the Messages app. Like I said, I gave up on the SMS app.

@johnnyorange I believe the Messages app does tie into the chatplan, but I made no modification to it in order to make this work. The only code modification I made, with the exception of the user_agent -> provider ID change, was the one mentioned above in index.php. If you're running Fusion on a public IP or otherwise passing through the public ID you shouldn't need to use user_agent identification.

From there, just make sure at least one extension has your SMS number set as it's outbound caller id number. Messages should then be delivered to any connected endpoints. If it isn't working, I'd recommend using postman or curl to send test inbound SMS messages and debug statements in the code to see where the breakdown is. You can also monitor the v_messages table in the database to see if they are getting correctly added. Again, this only worked for me with the Bandwidth.com provider; the Telnyx provider is not properly parsing the JSON.

 

[datafanatics]

@micah.quinn Funny enough at the time of my last posting, I didn't know there was a Messages app. I've since installed it. I'm just not able to find documentation on how to interface with it.

 

[micah.quinn]

What exactly do you mean by "interface with it"? If you mean connecting your carrier to the app, the webhook is ".../app/messages/index.php". There is logic in there that matches the from IP address in the request header with the provider. (Make sure you setup the provider correctly in FusionPBX).

 

[datafanatics]

What exactly do you mean by "interface with it"? If you mean connecting your carrier to the app, the webhook is ".../apps/messages/index.php". There is logic in there that matches the from IP address in the request header with the provider. (Make sure you setup the provider correctly in FusionPBX).

You nailed it! Thanks! I'm about to add things into Twilio to see if I can start receiving.

 

[adrnandes]

Hi guys,

I'm also having trouble getting this to work with my Fusionpbx 5.3, on Debian 12, + Telnyx sms provider. Has anyone had any success?
I'm not sure what credentials I should be putting in the "Providers" file in the "http_auth_password" and "http_auth_username" fields.
I'm also not sure if I can change the +1xxxxxxxxx fields to my local UK number +44xxxxxxxx

I'll leave the logs, and would appreciate if anyone could give me some guidance.

I successfully followed the steps in the link below.
https://docs.fusionpbx.com/en/latest/applications/messages.html

In the "Accounts > Providers" file, I made the following changes.
1) I inserted the Telnyx Key in the "Authorization > Bearer KEY...." field
2) I edited all the numbers +1xxxxxxxxx to my local UK number, +44xxxxxxxx.
3) I left the "http_auth_username" and "http_auth_password" fields blank.

In my Telnyx account's Dashboard,
1) I configured the number for SMS
2) I added the link below to my server's Web-hook, which I believe is the correct one.
File Web hook: "/var/www/fusionpbx/app/messages/index.php", correct?

https://my-fusionpbx-domain/app/messages/index.php

Could you please give me some suggestions?
Maybe I should create a new web hook file?

LOGS
To: <sip:+44xxxxxxxx@xxxx.xxxx.com>
From: <sip:202@xxxxxxxx@xxxx.xxxx.com>
NS: imdn <urn:ietfarams:imdn>
imdn.Message-ID: JFu87.7zDa61
DateTime: 2024-11-22T16:51:56.616Z
imdn.Disposition-Notification: positive-delivery, display
Content-type: text/plain
Content-length: 4

Test
2024-11-22 16:51:56.908793 96.93% [INFO] mod_sms.c:368 Processing text message 202->+44xxxxxxxx in context public
Chatplan: +44xxxxxxxx parsing [public->demo] continue=false
Chatplan: +44xxxxxxxx@xxxxxxxx@xxxx.xxxx.com> Regex (PASS) [demo] to(+44xxxxxxxx@xxxxxxxx@xxxx.xxxx.com>) =~ /^(.*)$/ break=on-false
2024-11-22 16:51:56.908793 96.93% [DEBUG] mod_sms.c:477 SMS chatplan no actions found

2024-11-22 16:51:57.008790 96.93% [WARNING] sofia_reg.c:2843 Cannot locate any authentication credentials to complete an authentication request for realm '"xxxxxxxx@xxxx.xxxx.com"'


Thanks

Ad

 

[micah.quinn]

Hi guys,

I'm also having trouble getting this to work with my Fusionpbx 5.3, on Debian 12, + Telnyx sms provider. Has anyone had any success?
I'm not sure what credentials I should be putting in the "Providers" file in the "http_auth_password" and "http_auth_username" fields.
I'm also not sure if I can change the +1xxxxxxxxx fields to my local UK number +44xxxxxxxx

I'll leave the logs, and would appreciate if anyone could give me some guidance.

I successfully followed the steps in the link below.
https://docs.fusionpbx.com/en/latest/applications/messages.html

In the "Accounts > Providers" file, I made the following changes.
1) I inserted the Telnyx Key in the "Authorization > Bearer KEY...." field
2) I edited all the numbers +1xxxxxxxxx to my local UK number, +44xxxxxxxx.
3) I left the "http_auth_username" and "http_auth_password" fields blank.

In my Telnyx account's Dashboard,
1) I configured the number for SMS
2) I added the link below to my server's Web-hook, which I believe is the correct one.
File Web hook: "/var/www/fusionpbx/app/messages/index.php", correct?

https://my-fusionpbx-domain/app/messages/index.php

Could you please give me some suggestions?
Maybe I should create a new web hook file?

LOGS
To: <sip:+44xxxxxxxx@xxxx.xxxx.com>
From: <sip:202@xxxxxxxx@xxxx.xxxx.com>
NS: imdn <urn:ietfarams:imdn>
imdn.Message-ID: JFu87.7zDa61
DateTime: 2024-11-22T16:51:56.616Z
imdn.Disposition-Notification: positive-delivery, display
Content-type: text/plain
Content-length: 4

Test
2024-11-22 16:51:56.908793 96.93% [INFO] mod_sms.c:368 Processing text message 202->+44xxxxxxxx in context public
Chatplan: +44xxxxxxxx parsing [public->demo] continue=false
Chatplan: +44xxxxxxxx@xxxxxxxx@xxxx.xxxx.com> Regex (PASS) [demo] to(+44xxxxxxxx@xxxxxxxx@xxxx.xxxx.com>) =~ /^(.*)$/ break=on-false
2024-11-22 16:51:56.908793 96.93% [DEBUG] mod_sms.c:477 SMS chatplan no actions found

2024-11-22 16:51:57.008790 96.93% [WARNING] sofia_reg.c:2843 Cannot locate any authentication credentials to complete an authentication request for realm '"xxxxxxxx@xxxx.xxxx.com"'


Thanks

Ad

Ad, It looks like you are close. Did you assign the " Outbound Caller ID Number" to an extension? Did you assign a user to the extension?I believe that sets up the chatplan so that the inbound SMS then has a destination. I'd recommend setting the DID and the deliver email and get inbound working first. Outbound tends to be much more difficult.

 

[adrnandes]

Hi Micah,

Thanks for your help!

I adjusted the settings, and now I'm able to receive SMS Telnyx, but it seems like they are arriving in all extensions.
I have not found any documentation on how to create a specific incoming route for SMS. I have tried several ways, but I have not been successful.
Does anyone know how to do this? Please?

I would also like to know if anyone has had success receiving SMS with the Groundwire or PortSip app.
I enabled "Simple SIP" on Groundwire, but it doesn't receive anything.
PortSip receives the message notification, but the message doesn't arrive, however, it arrives normally on my Zoiper Desktop on the Mac.


Thank you

Ad

 

[micah.quinn]

Hi Micah,

Thanks for your help!

I adjusted the settings, and now I'm able to receive SMS Telnyx, but it seems like they are arriving in all extensions.
I have not found any documentation on how to create a specific incoming route for SMS. I have tried several ways, but I have not been successful.
Does anyone know how to do this? Please?

I would also like to know if anyone has had success receiving SMS with the Groundwire or PortSip app.
I enabled "Simple SIP" on Groundwire, but it doesn't receive anything.
PortSip receives the message notification, but the message doesn't arrive, however, it arrives normally on my Zoiper Desktop on the Mac.


Thank you

Ad

In "index.php" there is a debug flag that causes the code to write out some extra information along the way. Critically this includes things like the user and extensions that the sip message is bound for. At this point, I'd recommend setting $debug to true at the top of the script and then sending a few messages. The log file is located in '/tmp/messages.log'.

 

[pbxguru]

Earlier this year, we got inbound working with Telnyx on 5.2 with Messages, and Groundwire App. Then, it all mysteriously stopped working and I do believe Telnyx JSON method changed.

In any case, we learned that to deliver to an extension, you need the following: +1 DID >> Extension >>User >> Contact - The key was the contact record where you enabled Text and make sure the +1 DID is set. This is how we can inbound finally working..then it broke. You guys are inspiring me to try again.

 

[adrnandes]

In "index.php" there is a debug flag that causes the code to write out some extra information along the way. Critically this includes things like the user and extensions that the sip message is bound for. At this point, I'd recommend setting $debug to true at the top of the script and then sending a few messages. The log file is located in '/tmp/messages.log'.

Hi Micah,

Thanks for the suggestion, I'm seeing some logs.