Let's Encrypt installation multitenat

[charneval]

Hi,

i'm new to FusionPBX, i need to install a multitenant setup with different 3 domains. example: fusionone.domainone.com, fusiontwo.domaintwo.com and fusionthree.domainthree.com

i'm following this guide: https://docs.fusionpbx.com/en/latest/getting_started/lets_encrypt.html?highlight=ssl

but i can't found any tips for my case. i don't need a wildcard setup because i have three different domains and the wildcars is only for the same main domain.

tell me if there is some new guide for my case.

Thanks.
Andrea

 

[hfoster]

It's not really anything unique to FusionPBX. Here's a bash script I used. Certbot needs to be in the executing user's $PATH, and the template.conf is practically the 'fusionpbx' config that comes enabled, only with placeholder.example.com to replace the certificate location:

#!/bin/bash
# Script to generate domain configurations for nginx
set -o errexit;

function generatepbx {

    DOMAIN=$1

    certbot certonly --preferred-chain "ISRG Root X1" --webroot -w /var/www/fusionpbx -d $DOMAIN
    cp /etc/nginx/sites-available/template.conf /etc/nginx/sites-available/$DOMAIN
    sed -i "s/placeholder.example.com/$DOMAIN/g" /etc/nginx/sites-available/$DOMAIN
    ln -s /etc/nginx/sites-available/$DOMAIN /etc/nginx/sites-enabled/$DOMAIN

}

if [ -z "$1" ]; then
  echo -e "Domain name is required.\n"
  exit 1
fi


for var in "$@"
do
  generatepbx $var
done

 

[viiiwonder]

I was able to follow the docs.

Let’s Encrypt — FusionPBX Docs documentation

docs.fusionpbx.com

One tip for your situation (as I was in the same): when you do your first request, make sure you do it with multiple domains, otherwise the initial and subsequent certs don't have alternate subject names if you add second domains later.

When you add a domain, just repeat the process with the full list of domains + new one.

Still trying to assess if this renews automatically...

 

[BelluX]

how to renew for the wildcard case?

dehydrated -c seems to work but only in manual mode.

how i can schedule a crontab for this? thank you.

 

[koolandrew]

Hi Everyone, i am having issues with my domains as i am pretty sure that i used dehydrated three months ago, but i can assure you that there is no renewal as mine didnt renew. I believe i have a wildcard as well as a specific domain and even though i renewed the domain using cd /usr/src/fusionpbx-install.sh/debian/resources and then ./letsencrypt.sh, the certificates didnt renew, even though it says they did according to the results.

I went to the dehydrated website and searched it some more, and created a cron job which someone asked about but i dont know if it works. I did crontab -e then 14 3 * * 5 /usr/bin/dehydrated -c. I have no idea if it works, as i have limited linux skills...and the docs do not cover how to add new domains with dehydrated just certbot.

Could someone please point me in the right direction

1. Continue using dehydrated (very limited documentation) vs certbot, and if i should change how do i get rid of dehydrated.
2. Adding domains properly -it appears that even if you use a wildcard, you still need to add domains for each pbx instance that you create.
3. How to add it properly so it gets updated for the purpose of tls.

Thanks for your help.

 

[viiiwonder]

This is a contradiction to the best of my knowledge:

I believe i have a wildcard as well as a specific domain

It would help to know what the contents of /etc/dehydrated/domains.txt; you can get it with:

cat /etc/dehydrated/domains.txt

Does that file have a wildcard, one, or multiple domains? Post at your own discretion - it will contain your domains.

The crontab you configured translates to:
Run

/usr/bin/dehydrated -c

at 3:14 every Friday.

Just by editing the crontab, you haven't run that command - it will get run on Friday at 3:14. You can just run that command manually to test though. Does that command renew your certificates if you run it manually? (I have no idea if it should or not... this whole subject is actually on my list of things to evaluate ("How to ensure certificates are automatically renewed?")

Wildcard certificates will be very difficult to renew without significant skill set - they require DNS based validation with dehydrated, so your cron/script/task has to be able to request the new cert, get the new TXT verification value, update your DNS (like... in godaddy, etc), and then complete the signing. Not an impossible series of scripted events, but fairly complex.

 

[koolandrew]

Hi thanks for the quick reply. It turns out that it is the wildcard, which is mydomain.com...i did try using
dehydrated -x -c
and you are correct, i am using cloudflare so everything got rejected...as i can see in the response. i can send the logs

Here is what i would like to do as this will never work with a wildcard, which is strange as i could renew wildcards with other installations...

1. Should i switch to certbot with individual domains...if so how do i get rid of dehydrated...
2. Can i follow the instructions to auto-renew
3. Can i use this for tls..

Here are the logs:

root@fusionpbx:~# dehydrated -x -c
# INFO: Using main config file /etc/dehydrated/config
Processing mydomain.com
+ Checking expire date of existing cert...
+ Valid till Oct 17 16:26:32 2022 GMT (Less than 30 days). Renewing!
+ Signing domains...
+ Generating private key...
+ Generating signing request...
+ Requesting new certificate order from CA...
+ Received 1 authorizations URLs from the CA
+ Handling authorization for mydomain.com
+ 1 pending challenge(s)
+ Deploying challenge tokens...
+ Responding to challenge for mydomain.com authorization...
+ Cleaning challenge tokens...
+ Challenge validation has failed
ERROR: Challenge is invalid! (returned: invalid) (result: ["type"] "http-01"
["status"] "invalid"
["error","type"] "urn:ietfarams:acme:error:unauthorized"
["error","detail"] "2606:4700:20::681a:8b0: Invalid response from http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8: 404"
["error","status"] 403
["error"] {"type":"urn:ietfarams:acme:error:unauthorized","detail":"2606:4700:20::681a:8b0: Invalid response from http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8: 404","status":403}
["url"] "https://acme-v02.api.letsencrypt.org/acme/chall-v3/165677163826/WPZazQ"
["token"] "_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8"
["validationRecord",0,"url"] "http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8"
["validationRecord",0,"hostname"] "mydomain.com"
["validationRecord",0,"port"] "80"
["validationRecord",0,"addressesResolved",0] "104.26.8.176"
["validationRecord",0,"addressesResolved",1] "104.26.9.176"
["validationRecord",0,"addressesResolved",2] "172.67.71.240"
["validationRecord",0,"addressesResolved",3] "2606:4700:20::681a:8b0"
["validationRecord",0,"addressesResolved",4] "2606:4700:20::ac43:47f0"
["validationRecord",0,"addressesResolved",5] "2606:4700:20::681a:9b0"
["validationRecord",0,"addressesResolved"] ["104.26.8.176","104.26.9.176","172.67.71.240","2606:4700:20::681a:8b0","2606:4700:20::ac43:47f0","2606:4700:20::681a:9b0"]
["validationRecord",0,"addressUsed"] "2606:4700:20::681a:8b0"
["validationRecord",0] {"url":"http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8","hostname":"mydomain.com","port":"80","addressesResolved":["104.26.8.176","104.26.9.176","172.67.71.240","2606:4700:20::681a:8b0","2606:4700:20::ac43:47f0","2606:4700:20::681a:9b0"],"addressUsed":"2606:4700:20::681a:8b0"}
["validationRecord"] [{"url":"http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8","hostname":"mydomain.com","port":"80","addressesResolved":["104.26.8.176","104.26.9.176","172.67.71.240","2606:4700:20::681a:8b0","2606:4700:20::ac43:47f0","2606:4700:20::681a:9b0"],"addressUsed":"2606:4700:20::681a:8b0"}]
["validated"] "2022-10-17T19:36:52Z")

 

[viiiwonder]

We're using dehydrated with multiple domains. I haven't set up a cron to auto-renew, so can't comment if that works, but I expect that it will, and the script sets everything up for TLS as well.

I'm skeptical of going too far afield of what the devs have in mind for cert management, it starts to limit the supportability via community. I'll be testing renewal this week. You seem very fixated on certbot...

I would advise removing all certs (I had issues adding additional subdomains if the initial request didn't include multiples - it's like if the cert isn't issued with an alternative subject name, it can never have one), then re-running the ./letsencrypt.sh script and setting a couple subdomains and then testing renewal via cron.

 

[koolandrew]

I am not psyched on certbot, but it is everywhere and the dehydrated website hasn't been updated for a year.

The issue is that there is no documentation for multiple domains, only for certbot, so i dont know what to do there.

Could you please tell me how to properly remove everything and i will start with one domain (not a wildcard) and test the cron job as well.

Thanks

 

[viiiwonder]

IIWM, I would clear out

/etc/dehydrated/certs

and then rerun the letsencrypt.sh script as instructed here.

As stated above, my advice is that if you're going to start with multiple domains, start with at least two so that your first cert gets written with an alternate subject name- I had issues adding my first alternate subject name after doing a single domain cert first; had to clear out the existing cert to get it to re-write it with the asn. YMMV.

For the record, I definitely share your concerns about the code commit frequency on dehydrated. I just got my Fusion membership, so hopefully in some of the training there's some justification for using Dehydrated over certbot that will put me at ease or at least make me understand why the decision was made.

 

[hfoster]

I'm going to guess it's because... dehydrated only uses bash which means it doesn't pull in more dependencies. I've used it once when I was doing a different project on Amazon Linux. Certbot is in one of those dumbass Ubuntu snaps. Why? Who knows? I'm pretty sure it's just a python3 script. It seems very vogue to create your own installer/updater instead of letting apt/yum do the lifting.

Either way, I try to use Certbot where I can because of the documentation and sheer momentum of the project.

As for the error log, looks less like it's dehydrated's fault and more like the nginx config:

["error","detail"] "2606:4700:20::681a:8b0: Invalid response from http://mydomain.com/.well-known/acme-challenge/_cm_SIyT7AvA6_H6zNV3Yd7wJ6N-50pRSip07uxLMp8: 404"
["error","status"] 403

Check out the nginx entries, you may need to modify the acme-challenge locations, ensure the right directory is being used, I made a slight change for my own purposes:

        #redirect letsencrypt to dehydrated
        location ^~ /.well-known/acme-challenge {
                default_type "text/plain";
                auth_basic "off";
                root /var/www/letsencrypt;
        }

 

[koolandrew]

This is becoming more pressing than i thought as my calls now go to voicemail. i dont understand why this is happening, as i am not sure what a certificate has to do with it, but it started occurring ever since my certificate expired.

I am fairly novice when it comes to linux etc so if you could please explain:
From viiiwonder:

IIWM, I would clear out

/etc/dehydrated/certs

1. What is IIWM?
2. Clear it out=delete..is that same to assume....do i need to stop nginx during this?

3. Another acronym..re-write it with the asn. YMMV.---You might make ???????..Maybe i am too old..lol

4.Regarding using the online instructions as reference---doesnt dehydrated do this below...as these are generic instructions...is there something else that i need to do first

• Request an SSL certificate from Let’s Encrypt.
• Configure NGINX to use the SSL certificate.

5. If requested multiple domains, is there are specific way to do so:

Is it one per line or separate by space or commas??

6. Certificate renewal. i realize i will have 90 days once i get this sorted, but i had no problems for the first 90 and dont want to keep going through this.

You mentioned the bash script, and i tried to excecute a cron job, but i really dont know what i am doing. Unless the instructions are very specific, i cannot fill in the gaps...

I appreciate everyone's input on this. I try to do the same on topics that i am capable of replying to, but when things stop working, i am really lost.

Domain Name: domain.tld
Email Address: support@fusionpbx.com

 

[viiiwonder]

IIWM = If it were me
YMMV = Your mileage may vary
ASN = alternative subject name (a certificate construct that allows a single certificate to be valid for multiple domains)

Not sure about the voicemail thing - be careful about assuming correlation. If you want to know why that's happening, review the logs. Regardless, fix your cert. Possible that if you somehow managed to get TLS on calls in place, that is now failing.

Have you reviewed the public documentation on this topic?
https://docs.fusionpbx.com/en/latest/getting_started/lets_encrypt.html

1. Delete the contents of /etc/dehydrated/certs
2. Rerun ./letsencrypt.sh as detailed in the above article
3. When asked, pass it either one or two FQDNs separated by a space
4. Done

I can confirm that there is documentation regarding renewals in the membership documentation section. As such, I won't comment on a procedure for that here.

As for Linux, I would recommend you review the concept of Cron in general - it's just a task scheduler.

 

[koolandrew]

Thanks again for the quick reply, i am familiar with ASN...the others, not so much IMHO..lol

Regarding the renewal, i thought that was for certbot only, thats why i ignored it. My bad.

Thanks again.

 

[koolandrew]

I deleted the contents, and noticed that nginx or at the least web page stopped working. I rebooted the server, no change. i could not log in using the ip, no web page.

It appears that dehydrated is trying to add my wildcard domain as that is what i initially did 3 months ago?

When i issue the command
cat /etc/dehydrated/domains.txt

I get the names of the two subdomains that i added, and they are both pointed to this ip address in cloudflare and added in the web interface, prior to doing this.

/etc/nginx/nginx.conf ----------this is now empty?
/etc/dehydrated/certs-----this is now empty?

The nginx logs are below, they support the same issue.

The results are below including the errors.

root@fusionpbx:~# cd /usr/src/fusionpbx-install.sh/debian/resources/
root@fusionpbx:/usr/src/fusionpbx-install.sh/debian/resources# ./letsencrypt.sh
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
curl is already the newest version (7.74.0-1.3+deb11u2).
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Domain Name: fusionpbx.mydomain.com server4.mydomain.com
Email Address: andrew.wells@mydomain.com
Cloning into 'dehydrated'...
remote: Enumerating objects: 2341, done.
remote: Counting objects: 100% (194/194), done.
remote: Compressing objects: 100% (86/86), done.
remote: Total 2341 (delta 118), reused 175 (delta 108), pack-reused 2147
Receiving objects: 100% (2341/2341), 901.73 KiB | 5.93 MiB/s, done.
Resolving deltas: 100% (1471/1471), done.
# INFO: Using main config file /etc/dehydrated/config
+ Account already registered!
# INFO: Using main config file /etc/dehydrated/config
Processing fusionpbx.mydomain.com with alternative names: server4.mydomain.com
+ Creating new directory /etc/dehydrated/certs/fusionpbx.mydomain.com ...
+ Signing domains...
+ Generating private key...
+ Generating signing request...
+ Requesting new certificate order from CA...
+ Received 2 authorizations URLs from the CA
+ Handling authorization for fusionpbx.mydomain.com
+ Found valid authorization for fusionpbx.mydomain.com
+ Handling authorization for server4.mydomain.com
+ Found valid authorization for server4.mydomain.com
+ 0 pending challenge(s)
+ Requesting certificate...
+ Using preferred chain with CN = ISRG Root X1
+ Checking certificate...
+ Done!
+ Creating fullchain.pem...
+ Done!
nginx: [emerg] cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /etc/nginx/nginx.conf test failed

****************Nginx logs************************
2022/10/19 10:54:35 [emerg] 825804#825804: cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
2022/10/19 10:55:22 [emerg] 825838#825838: cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
2022/10/19 10:57:54 [emerg] 825878#825878: cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
2022/10/19 11:00:40 [emerg] 575#575: cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
2022/10/19 11:03:20 [emerg] 4764#4764: cannot load certificate "/etc/dehydrated/certs/mydomain.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/dehydrated/certs/mydomain.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

 

[viiiwonder]

/etc/nginx/nginx.conf

Might have some references to '/etc/dehydrated/certs/mydomain.com/fullchain.pem', which may no longer be correct, since you have now moved from doing a wildcard to multiple explicit domains. Might have to clean up that nginx.conf file and rerun the letsencrypt.sh script to have it populate it again with the valid cert paths. (or just go manually update it)

 

[ronthompson]

If you using Dehydrate
Run the command for your first cert using your Host Domain,
cd /usr/src/fusionpbx-install.sh/debian/resources/
./letsencrypt.sh

Once complete edit the domains.txt file, add all other domain names to be included in the SAN. Separate each domain by a space.
1.domainname.com 2.domainname.com 3.domainname.com
nano /etc/dehydrated/domains.txt
Then run
cd /usr/src/dehydrated
./dehydrated -c -f /etc/dehydrated/config

You can then include the certs for wss once complete.
cd /etc/dehydrated/certs/your.domainname.com
cat cert.pem privkey.pem > /etc/freeswitch/tls/wss.pem

 

[koolandrew]

/etc/nginx/nginx.conf

Might have some references to '/etc/dehydrated/certs/mydomain.com/fullchain.pem', which may no longer be correct, since you have now moved from doing a wildcard to multiple explicit domains. Might have to clean up that nginx.conf file and rerun the letsencrypt.sh script to have it populate it again with the valid cert paths. (or just go manually update it)

thanks again for following up, there are only a bunch of files in /etc/dehydrated/certs/fusionpbx.mydomain.com and many of the filelinks are 0 bytes , priv.key, cert.pem, chain.pem, fullchain.pem, cert.csr...the same names except have -1666191791 have are not 0 bytes.

When i look at nginx.conf file it appears the only changes are as follows:

include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;

when i go sites-enabled i see this which stands out

ssl_certificate /etc/dehydrated/certs/mydomain.com/fullchain.pem;
ssl_certificate_key /etc/dehydrated/certs/mydomain.com/privkey.pem;

Should i remove these two lines are they obviously dont exist or edit them to say fusionpbx.mydomain.com and/or server4.mydomain.com

 

[koolandrew]

If you using Dehydrate
Run the command for your first cert using your Host Domain,
cd /usr/src/fusionpbx-install.sh/debian/resources/
./letsencrypt.sh

Once complete edit the domains.txt file, add all other domain names to be included in the SAN. Separate each domain by a space.
1.domainname.com 2.domainname.com 3.domainname.com
nano /etc/dehydrated/domains.txt
Then run
cd /usr/src/dehydrated
./dehydrated -c -f /etc/dehydrated/config

You can then include the certs for wss once complete.
cd /etc/dehydrated/certs/your.domainname.com
cat cert.pem privkey.pem > /etc/freeswitch/tls/wss.pem

Thanks very much for pitching in. My only concern that the nano /etc/dehydrated/domains.txt is correct right now...so should i remove them and then do as you say or simply leave it and go ahead.

The web interface is still down at this point, notwithstanding the ssl.

Thx

 

[viiiwonder]

Should i remove these two lines are they obviously dont exist or edit them to say fusionpbx.mydomain.com and/or server4.mydomain.com

You can certainly just try to edit them - if they point to files that exist, then nginx will probably start.

Otherwise, if you remove them, nginx's conf validation might complain that you have NO cert specified... but that'd be the next step (remove the lines altogether).

If you just look at the letsencrypt.sh script, you can see what it does to the nginx.conf and emulate it/understand it.

there are only a bunch of files in /etc/dehydrated/certs/fusionpbx.mydomain.com and many of the filelinks are 0 bytes , priv.key, cert.pem, chain.pem, fullchain.pem, cert.csr...the same names except have -1666191791 have are not 0 bytes

Sounds right... they're all just dated files and symlinked; refer/use the non-numbered files when you're editing nginx.conf

The web interface is still down at this point, notwithstanding the ssl

This is all because nginx is failing to find the certs ref'd in nginx.conf and won't start because of that error.