Jump to content

Proxy HTTPS to HTTP

Poet129
By Poet129
in Networking
 Share
Go to solution Solved by WereCatf,
1 minute ago, Poet129 said:

Okay so do I need a cert for every website or just one?

Presumably only one. That said, I don't use Squid so I can't help you more than that. You should google on how to use Squid for HTTPS/SSL-caching or read the link I gave and just give it a try.

Posted

Is there a proxy I can run on Linux or Windows that converts HTTPS to HTTP traffic like this diagram.

 

Client<--HTTP-->Proxy<--HTTPS-->Internet

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

That's also known as an HTTPS terminator. We use Nginx for that. It receives the HTTPS traffic and proxies it to our server application which is running on port 8080.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author

I saw that but wasn't sure what config I should use.

2 minutes ago, Eigenvektor said:

That's also known as an HTTPS terminator. We use Nginx for that. It receives the HTTPS traffic and proxies it to our server application which is running on port 8080.

 

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
6 minutes ago, Poet129 said:

I saw that but wasn't sure what config I should use.

Here's an example of the configuration our app uses. Simplified a bit and names removed:

Spoiler 

server {
    listen 0.0.0.0:80;
    listen [::]:80;
    server_name ${HOSTNAME};
    server_tokens off; ## Don't show the nginx version number, a security best practice
    # Redirect to HTTPS
    return 301 https://$host$request_uri;
    access_log  /var/log/nginx/${APP}_access.log ${APP};
    error_log   /var/log/nginx/${APP}_error.log;
}

server {
    listen 0.0.0.0:443 ssl;
    listen [::]:443 ssl;
    server_name ${HOSTNAME};
    server_tokens off; ## Don't show the nginx version number, a security best practice

    access_log  /var/log/nginx/${APP}_access.log ${APP};
    error_log   /var/log/nginx/${APP}_error.log;

    ssl_dhparam          /etc/nginx/dhparams.pem;
    ssl_certificate      /etc/letsencrypt/live/${HOSTNAME}/cert.pem;
    ssl_certificate_key  /etc/letsencrypt/live/${HOSTNAME}/privkey.pem;

    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/letsencrypt/live/${HOSTNAME}/fullchain.pem;

    ssl_prefer_server_ciphers   on;
    ssl_ciphers '...lot's of ciphers...';
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    client_max_body_size 1024M;
    large_client_header_buffers 8 32k;

    proxy_http_version 1.1;
    proxy_read_timeout 90;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    location / {
		gzip on;
		gzip_proxied any;
		gzip_min_length  1100;
		gzip_buffers 4 32k;
		gzip_types text/plain application/javascript text/xml text/css;
		gzip_vary on;

		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto $scheme;
		proxy_pass http://127.0.0.1:8080;
		proxy_read_timeout 300;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection $connection_upgrade;
    }


    error_page 502 /502.html;
    error_page 503 /502.html;
    error_page 504 /502.html;
    location /502.html {
        root /etc/nginx/errors;
    }
}

 

Essentially nginx listens on port 80 (HTTP) and 443 (HTTPS). Connections to HTTP are simply forwarded to HTTPS. Connections to HTTPS are proxied to localhost:8080.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Eigenvektor said:

Here's an example of the configuration our app uses. Simplified a bit and names removed:

  Reveal hidden contents 


server {
    listen 0.0.0.0:80;
    listen [::]:80;
    server_name ${HOSTNAME};
    server_tokens off; ## Don't show the nginx version number, a security best practice
    # Redirect to HTTPS
    return 301 https://$host$request_uri;
    access_log  /var/log/nginx/${APP}_access.log ${APP};
    error_log   /var/log/nginx/${APP}_error.log;
}

server {
    listen 0.0.0.0:443 ssl;
    listen [::]:443 ssl;
    server_name ${HOSTNAME};
    server_tokens off; ## Don't show the nginx version number, a security best practice

    access_log  /var/log/nginx/${APP}_access.log ${APP};
    error_log   /var/log/nginx/${APP}_error.log;

    ssl_dhparam          /etc/nginx/dhparams.pem;
    ssl_certificate      /etc/letsencrypt/live/${HOSTNAME}/cert.pem;
    ssl_certificate_key  /etc/letsencrypt/live/${HOSTNAME}/privkey.pem;

    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/letsencrypt/live/${HOSTNAME}/fullchain.pem;

    ssl_prefer_server_ciphers   on;
    ssl_ciphers '...lot's of ciphers...';
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    client_max_body_size 1024M;
    large_client_header_buffers 8 32k;

    proxy_http_version 1.1;
    proxy_read_timeout 90;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    location / {
		gzip on;
		gzip_proxied any;
		gzip_min_length  1100;
		gzip_buffers 4 32k;
		gzip_types text/plain application/javascript text/xml text/css;
		gzip_vary on;

		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto $scheme;
		proxy_pass http://127.0.0.1:8080;
		proxy_read_timeout 300;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection $connection_upgrade;
    }


    error_page 502 /502.html;
    error_page 503 /502.html;
    error_page 504 /502.html;
    location /502.html {
        root /etc/nginx/errors;
    }
}

 

 

Looks great, but looking at the code it looks like I would need the certificates private key to decrypt or is that for re-encryption if so is there a way to leave it disabled?

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
3 minutes ago, Poet129 said:

Looks great, but looking at the code it looks like I would need the certificates private key to decrypt or is that for re-encryption if so is there a way to leave it disabled?

That's simply the SSL certificate for HTTPS, the one needed for browsers to show it as valid. You should absolutely use one.

 

There is no encryption or re-encryption between the proxy and our application. Nginx terminates the encrypted SSL connection. The example is using a Let's Encrypt SSL certificate, which anyone can get, free of charge.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Eigenvektor said:

That's simply the SSL certificate for HTTPS, the one needed for browsers to show it as valid. You should absolutely use one.

 

There is no encryption or re-encryption between the proxy and our application. Nginx terminates the encrypted SSL connection. The example is using a Let's Encrypt SSL certificate, which anyone can get, free of charge.

I'm looking to redirect all traffic to HTTP so if for example I went to www.google.com it would show up as HTTP and unencrypted.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
6 minutes ago, Poet129 said:

I'm looking to redirect all traffic to HTTP so if for example I went to www.google.com it would show up as HTTP and unencrypted.

That's not what I got from your initial question. You were talking about proxying HTTPS connections, not redirecting them. Why would you want to redirect users from e.g. "https://example.com" to "http://example.com"? If you need a simply redirect, look at the HTTP section of the configuration, this redirects users to HTTPS. That should also work in reverse, e.g. make the HTTPS portion of the configuration redirect users to HTTP.

 

The purpose of this configuration is to have a valid HTTPS connection for your users that terminates at the web server. The web server proxies the (unencrypted) traffic to our application which doesn't need to know anything about encryption and is only reachable on the internal network.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Eigenvektor said:

That's not what I got from your initial question. You were talking about proxying HTTPS connections, not redirecting them. Why would you want to redirect users from e.g. "https://example.com" to "http://example.com"? If you need a simply redirect, look at the HTTP section of the configuration, this redirects users to HTTPS. That should also work in reverse, e.g. make the HTTPS portion of the configuration redirect users to HTTP.

That would be a simple redirect If you use http://www.google.com it will just redirect you again because of hsts, and some sites only use HTTPS. Some what I'm looking for is a proxy that coverts HTTPS traffic to HTTP.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
Just now, Poet129 said:

That would be a simple redirect If you use http://www.google.com it will just redirect you again because of hsts, and some sites only use HTTPS. Some what I'm looking for is a proxy that coverts HTTPS traffic to HTTP.

Sorry, I'm not sure what you're trying to say. Are you saying you want to force outgoing connections to connect to HTTP instead of HTTPS? I'm talking about incoming connection to a web server.

 

This configuration does exactly this: It accepts HTTPS connections, decrypts them and forwards the decrypted traffic to port 8080 on the same machine. The connection between the browser and the web server (nginx) is encrypted. Which is why you need a valid certificate. The traffic between nginx and the app running on port 8080 is not encrypted. This is proxying.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Eigenvektor said:

Sorry, I'm not sure what you're trying to say. Are you saying you want to force outgoing connections to connect to HTTP instead of HTTPS? I'm talking about incoming connection to a web server.

 

This configuration does exactly this: It accepts HTTPS connections, decrypts them and forwards the decrypted traffic to port 8080 on the same machine. The connection between the browser and the web server (nginx) is encrypted. Which is why you need a valid certificate. The traffic between nginx and the app running on port 8080 is not encrypted. This is proxying.

Exactly that but without encryption between nginx and the web browser.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
4 minutes ago, Poet129 said:

Exactly that but without encryption between nginx and the web browser.

It looks like we're talking about different things in this case. This is what I have in mind: 

Browser <--(https)--> [Internet] <--(https)--> Web server/Reverse proxy/SSL terminator <--(http)--> Web application

The purpose of this is to have encrypted traffic between the browser and the web server over the internet.

 

It looks like what you want is more like this: 

Browser <--(http)--> Proxy <--(https)--> [Internet] <--(https)--> Web server

This is not what nginx is made for. Nginx is a web server/reverse proxy. It is not made to break up SSL traffic (a security worst practice)

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Eigenvektor said:

It looks like we're talking about different things in this case. This is what I have in mind: 


Browser <--(https)--> [Internet] <--(https)--> Web server/Reverse proxy/SSL terminator <--(http)--> Web application

The purpose of this is to have encrypted traffic between the browser and the web server over the internet.

 

It looks like what you want is more like this: 


Browser <--(http)--> Proxy <--(https)--> [Internet] <--(https)--> Web server

This is not what nginx is made for. Nginx is a web server/reverse proxy. It is not made to break up SSL traffic (a security worst practice)

This is exactly what I want, but I don't know how to set it up.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
15 minutes ago, Poet129 said:

One last question if I set it up on a device for the whole device not just the browser everything will be HTTP on the client side right?

No. If an application (e.g. browser) connects to HTTPS it expects an encrypted response with a valid certificate. You can't simply decrypt the response and return that to the client. If the site is using HSTS the browser should also refuse to be redirected to an unencrypted site.

 

Basically you need to configure your browser to use e.g. Fiddler as a proxy and you also need to install Fiddler's certificate in your browser so that it is seen as valid. Fiddler will decrypt the response, log it, and then re-encrypt it before sending it to the browser, who needs to accept the response as valid (e.g. valid certificate).

 

If the site is using DNS CAA the browser may still refuse to connect, because the certificate hasn't been issued by the expected CA.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
10 minutes ago, Eigenvektor said:

No. If an application (e.g. browser) connects to HTTPS it expects an encrypted response with a valid certificate. You can't simply decrypt the response and return that to the client. If the site is using HSTS the browser should also refuse to be redirected to an unencrypted site.

 

Basically you need to configure your browser to use e.g. Fiddler as a proxy and you also need to install Fiddler's certificate in your browser so that it is seen as valid. Fiddler will decrypt the response, log it, and then re-encrypt it before sending it to the browser, who needs to accept the response as valid (e.g. valid certificate).

 

If the site is using DNS CAA the browser may still refuse to connect, because the certificate hasn't been issued by the expected CA.

My Idea was for everything to HTTP after proxy no HTTPS. This did work as you explained but not quite what I'm looking for. I want the device to see something like www.google.com under HTTP instead of HTTPS.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
17 minutes ago, Poet129 said:

My Idea was for everything to HTTP after proxy no HTTPS. This did work as you explained but not quite what I'm looking for. I want the device to see something like www.google.com under HTTP instead of HTTPS.

Thankfully it isn't that easy, for security reasons. Google is using a HSTS header with a max-age of 31,536,000 seconds (365 days), so browsers will know to only ever access it via HTTPS. Your browser will automatically redirect you to HTTPS for one year after it has last seen that header. So you'd have to wait for one year with no connection to Google in between before your browser would even let you attempt to connect to HTTP without an automatic redirect (and Google would immediately redirect you and the browser would see the header again...)

 

~edit: Also, other software (e.g. mobile apps) may be hard coded to connect to their server with HTTPS and may not even honor/care about redirects. For additional security they may also be using certificate pinning, so they would always refuse Fiddler's certificate.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
1 hour ago, Poet129 said:

My Idea was for everything to HTTP after proxy no HTTPS. This did work as you explained but not quite what I'm looking for. I want the device to see something like www.google.com under HTTP instead of HTTPS.

Why would you want to remove all security like that in the first place?

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
7 hours ago, WereCatf said:

Why would you want to remove all security like that in the first place?

A lancache between the client and the proxy... I tried it with fiddler the idea worked but everything was still HTTPS so It wouldn't cache. The reason was so that lancache could be basically a entire Internet cache instead of just limited by the part of the Internet that is just HTTP

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
8 hours ago, Eigenvektor said:

Thankfully it isn't that easy, for security reasons. Google is using a HSTS header with a max-age of 31,536,000 seconds (365 days), so browsers will know to only ever access it via HTTPS. Your browser will automatically redirect you to HTTPS for one year after it has last seen that header. So you'd have to wait for one year with no connection to Google in between before your browser would even let you attempt to connect to HTTP without an automatic redirect (and Google would immediately redirect you and the browser would see the header again...)

 

~edit: Also, other software (e.g. mobile apps) may be hard coded to connect to their server with HTTPS and may not even honor/care about redirects. For additional security they may also be using certificate pinning, so they would always refuse Fiddler's certificate.

Couldn't you tell the proxy to not let the header though.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
7 minutes ago, Poet129 said:

A lancache between the client and the proxy... I tried it with fiddler the idea worked but everything was still HTTPS so It wouldn't cache. The reason was so that lancache could be basically a entire Internet cache instead of just limited by the part of the Internet that is just HTTP

You're kinda trying to climb up a tree ass first. Squid is a caching proxy and can also be made to cache SSL-traffic, like e.g. https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
6 minutes ago, Poet129 said:

Couldn't you tell the proxy to not let the header though.

Sure, and after one year the browser would no longer try to force HTTPS. It will force HTTPS for one year after it has last seen that header.

Remember to either quote or @mention others, so they are notified of your reply

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
Just now, Poet129 said:

Okay but does it run under ubuntu?

Yes, you can run Squid under Linux.

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to comment
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Networking

×