Ubuntu, apache2 and SSL (443)

[foxp3]

Hi Team Linus,

 

first off, I'm quite a newbie when it comes to servers. Actually this is my first project.

 

I bought a 100GB v-server and it comes with Ubuntu 16.04 including an apache2 server. It also comes with a fixed IP, lets say 123.123.123.123 and I have a subdomain pointing with an A record directly at the IP. The apache2 default site works perfectly both ways, either typing in cloud.mysuperdomainname.at or the IP directly via HTTP. But I have serious problems installing an SSL certificate (letsencrypt, https://certbot.eff.org/#ubuntuxenial-apache).

 

 

My ports.conf file:

Listen 80
Listen 443 // I added this extra, the original file came without this line

<IfModule ssl_module>
        Listen 443
</IfModule>

<IfModule mod_gnutls.c>
        Listen 443
</IfModule>

My sites-enabled file: 000-default.conf

<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        ServerName cloud.mysuperdomainname.at
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}access.log combined
</VirtualHost>

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        Servername cloud.mysuperdomainname.at
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

 

Trying to install the letsencrypt certificate gives the following error:

Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for cloud.mysuperdomainname.at
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. cloud.mysuperdomainname.at (tls-sni-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Error getting validation data

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: cloud.mysuperdomainname.at
   Type:   connection
   Detail: Error getting validation data

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

I tried quite alot and searched google for hours but I don't have any solution yet. Can you guys help me? If you need logfiles or anything, just let me know. That would be awesome.

 

[KuJoe]

The domain "mysuperdomainname.at" doesn't appear to be registered. Check to make sure you didn't mispell it and if you did in fact register it contact your registrar to find out why the DNS isn't setup.

[foxp3]

Just now, KuJoe said:

The domain "mysuperdomainname.at" doesn't appear to be registered. Check to make sure you didn't mispell it and if you did in fact register it contact your registrar to find out why the DNS isn't setup.

yes, its a mock-up.

[KuJoe]

1 minute ago, foxp3 said:

yes, its a mock-up.

Then please provide some real information for us to assist you with. Giving us fake domains and IPs will not help us troubleshoot a DNS problem.

[foxp3]

xxx pointing at xxx

 

Thank you!

[jj9987]

27 minutes ago, KuJoe said:

Then please provide some real information for us to assist you with. Giving us fake domains and IPs will not help us troubleshoot a DNS problem.

It's not a DNS problem, if the website is accessible via both IP and the domain as said by OP:

Quote

The apache2 default site works perfectly both ways, either typing in cloud.mysuperdomainname.at or the IP directly via HTTP. But I have serious problems installing an SSL certificate

 

First, you don't need to add Listen 443 to the ports.conf file. If you load the SSL module, it is configured automatically.

Secondly, remove the VirtualHost configuration for port 443. It is failing as it does not have SSL configuration. If I am not mistaken, then Certbot tries to verify the acme-challenge(s) via HTTPS (port 443) connection using it's own certificates, therefor resulting in an error.

 

Certbot will create the necessary VirtualHost configuration for HTTPS for you.

[KuJoe]

2 minutes ago, jj9987 said:

It's not a DNS problem

Correct, and now I'm sure of it.

[foxp3]

1 minute ago, KuJoe said:

Correct, and now I'm sure of it.

yeah sorry, paranoid european here. i always try to use fake data. (e.g. foxp3 is not my real name, but i bet you would have guessed that by now )

[KuJoe]

Just now, foxp3 said:

yeah sorry, paranoid european here. i always try to use fake data. (e.g. foxp3 is not my real name, but i bet you would have guessed that by now )

No problem. I just wanted to make sure it wasn't a caching issue anywhere. I see some weirdness with ccTLDs and LetsEncrypt quite a bit.

[foxp3]

7 minutes ago, jj9987 said:

It's not a DNS problem, if the website is accessible via both IP and the domain as said by OP:

 

First, you don't need to add Listen 443 to the ports.conf file. If you load the SSL module, it is configured automatically.

Secondly, remove the VirtualHost configuration for port 443. It is failing as it does not have SSL configuration. If I am not mistaken, then Certbot tries to verify the acme-challenge(s) via HTTPS (port 443) connection using it's own certificates, therefor resulting in an error.

 

Certbot will create the necessary VirtualHost configuration for HTTPS for you.

Thank you. I removed the things you mentioned, unfortunately still not working

[Jarsky]

Seems your port 443 isn't open or isn't listening.

Check your firewall rules in iptables / ufw / firewalld in Ubuntu and if your host has any sort of firewall config on your hosting panel like with Amazon EC2

 

also make sure you've enabled the SSL module

 

sudo a2enmod ssl

 

P.S generally your SSL should be a seperate conf file. So if your main site is  000-default.conf then you should be putting your SSL config into  000-default-ssl.conf 

 

You then use the a2ensite command to enable it.

sudo a2ensite 000-default-ssl

 

[foxp3]

On 16.7.2017 at 8:20 PM, Jarsky said:

Seems your port 443 isn't open or isn't listening.

Check your firewall rules in iptables / ufw / firewalld in Ubuntu and if your host has any sort of firewall config on your hosting panel like with Amazon EC2

Also make sure you've enabled the SSL module

 

sudo a2enmod ssl

 

P.S generally your SSL should be a seperate conf file. So if your main site is  000-default.conf then you should be putting your SSL config into  000-default-ssl.conf 

 

You then use the a2ensite command to enable it.

sudo a2ensite 000-default-ssl

 

That's why I initially added the "Listen 443" part to my ports.conf file in apache.

 

netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      274/sshd        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      2770/apache2  


That was before I added the "Listen 443" to ports.conf - with it, it looks like this.

netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      274/sshd        
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      2770/apache2    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      2770/apache2


iptables -L gives the following:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     icmp --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     tcp  --  anywhere             anywhere             state NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http state NEW,ESTABLISHED
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

Does this help?

[Jarsky]

Can you telnet the 443 port from the localhost? i.e

 

telnet 127.0.0.1 443

 

 

[foxp3]

12 hours ago, Jarsky said:

Can you telnet the 443 port from the localhost? i.e

 

telnet 127.0.0.1 443

 

 

telnet 127.0.0.1 443

Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Without a problem. Could it be that my iptables only have an http entry and not an https entry?

[Jarsky]

1 hour ago, foxp3 said:

telnet 127.0.0.1 443

Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Without a problem. Could it be that my iptables only have an http entry and not an https entry?

 

Yup, you only have a rule for the http

 

ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http state NEW,ESTABLISHED

 

You're probably best to also use conntrack, so I would remove the existing entry and create a new one.

 

Run this command to find the chain number of the http entry above - its probably #5

sudo iptables -L --line-numbers

 

Then you should run this to remove the rule:

sudo iptables -D INPUT 5   (where 5 is the chain number of the rule)

Then create your new rules:

    sudo iptables -I 5 INPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

 

Check that you can load the site on both port 80 & 443.

If all is good then you should be able to create your certificate.

[foxp3]

32 minutes ago, Jarsky said:

 

Yup, you only have a rule for the http

 

ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http state NEW,ESTABLISHED

 

You're probably best to also use conntrack, so I would remove the existing entry and create a new one.

 

Run this command to find the chain number of the http entry above - its probably #5

sudo iptables -L --line-numbers

 

Then you should run this to remove the rule:

sudo iptables -D INPUT 5   (where 5 is the chain number of the rule)

Then create your new rules:

    sudo iptables -I 5 INPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
    sudo iptables -A OUTPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

 

Check that you can load the site on both port 80 & 443.

If all is good then you should be able to create your certificate.

 

 

Sir, you are my absolute hero. It works (I had to leave out the "5" at the first rule though, because it gave me an error). What does conntrack do to make it better?

Thanks to all of you for your awesome help!

[Jarsky]

No problem My iptables is a bit rusty since I typically use ufw or firewalld these days (different management tools for the same Linux firewall) - but I was just trying to input the rule in the same position in your firewall chain. Firewalls are a hierarchy, so it follows through the rules in order - it shouldn't matter though since you don't really have anything in there.

 

Conntrack is an abbreviation for connection tracking, it allows you to do some cool management rules based on statistics if you start taking high load on those ports, which can happen with webservers that are public facing.

 

You might also want to look into security such as fail2ban for security your SSH - and possibly even SHA256 authentication for SSH. I do that for all my public facing linux servers. Heres a really good hardening guide: https://www.thefanclub.co.za/how-to/how-secure-ubuntu-1604-lts-server-part-1-basics

[foxp3]

2 minutes ago, Jarsky said:

No problem My iptables is a bit rusty since I typically use ufw or firewalld these days (different management tools for the same Linux firewall) - but I was just trying to input the rule in the same position in your firewall chain. Firewalls are a hierarchy, so it follows through the rules in order - it shouldn't matter though since you don't really have anything in there.

 

Conntrack is an abbreviation for connection tracking, it allows you to do some cool management rules based on statistics if you start taking high load on those ports, which can happen with webservers that are public facing.

Is there a way to disable iptables and just use UFW? Ive used that one and it seemed supereasy to use.

 

[Jarsky]

UFW should already be installed on the latest Ubuntu. Just run:

 

sudo service ufw status

Should get an output like:

 

● ufw.service - Uncomplicated firewall
   Loaded: loaded (/lib/systemd/system/ufw.service; enabled; vendor preset: enabled)
   Active: active (exited) since Thu 2017-07-06 22:10:04 NZST; 1 weeks 5 days ago
     Docs: man:ufw(8)
 Main PID: 331 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 19660)
   CGroup: /system.slice/ufw.service

 

If it's installed but not running then just

 

sudo service ufw start

 

If it's not installed you can install it with

 

sudo apt-get install ufw