Setting up web server

[QChoumont]

I'm working on a project for school where we need to create a chat application for Android that communicates a website.

 

We have never done anything that communicates over a web server and I'm having trouble understanding webserver communication.

 

So we are creating an android application and will likely be using HttpURLConnect to send our Posts to the application server which will be written in Java using javalin( we've also never touched javalin) the application server will communicate with the database through a translation layer and then send back the requested information as a json object, the android application will interpret the json object locally to display the nessesary info. The same will be done through the web interface.

 

Most of our communication will be updating and requesting chat logs from the database.

 

That's sort of what I understand what's confusing me is that I was told that on-top of the application server we need a separate webserver for the web interface, and what I don't get is why do we need that and why can't the web interface simply communicate directly with the application server?

 

Also I need to figure out how to set up an application server for testing locally.

 

Any advice or book recommendations that cover these topics would be appreciated.

[Eigenvektor]

You typically use a web server in front of the actual application server for added security. For example our application server at work can only speak HTTP. It operates on port 8080 and is located somewhere on the internal network. The web server is accessible from the Internet and provides the normal ports (80, 443).

 

We could change our application to use HTTPS, but it's far simpler to use a product that is specifically designed for this use case. If someone where to hack into the web server, they're still not able to access the application directly and still one step further removed from accessing the database.

[Aljaxus]

Hey @QChoumont

 

Alrighty so you're basically developing a simple chat webapp with an android client - but the client doesn't really play any significant role in this play. It's just a "side character".

 

I'll try to write a short tutorial on how I would personally approach this task. I'll try to explain most of the things that I think you would not know about - if there is anything that you'll still find too complicated ping me, or even better send me an email (google my nickhandle and you'll find it) because I really, really rarely come here.

 

The whole thing

 

Imagine this; you have three services₁; the database₂, a reverse proxy₃ and the API₄.

 

(edit 1) These small numbers after these terms are references to the thingy at the very bottom of the post. Read that shit

 

 

database

Let's start with the most obvious stuff - the database - you'll use it to store data. You need to make sure that your API can connect to it. I don't know what you're planning to use (or what you have to use) and it really doesn't matter. The only thing that you need to make sure is that if possible, only the API service can access it. Do not expose it to the internet, do not expose it to LAN, it should be only accessible to the API service. And of course there are exceptions - when you have clusters, distributed systems, your services running "in the cloud" (eg. Kubernetes or similar solutions) but we're not here for that.

 

 

reverse proxy

 

If I were you - and I suppose you're just starting out - I would google for "how to setup a reverse proxy for web server" and I would most likely find a blog post or some answer on stackexchange talking about Apache (or httpd) and I'd use that. Don't. Just use NGINX. It's better, it's faster, it's easier to configure especially when you get used to it and the configuration uses an actually readable format (.conf) unlike Apache...

 

 

Quote

This is just my opinion though, you can use whatever you're more comfy with - do your own research, install both services (on a VM or a docker container, not on your PC or laptop...) and see what's better for you. The same thing goes for everything else I wrote in this post. do your own research. at least google the darn thing and take a look at the documentation.

 

Now all you have to do it tell your reverse proxy to serve static files by default and when the request is made to the /api endpoint, you simple proxy it to your API service. You can proxy requests to UNIX socket files or to a TCP/IP socket.

 

Quote

A UNIX socket is an inter-process communication mechanism that allows bidirectional data exchange between processes running on the same machine.

 

IP sockets (especially TCP/IP sockets) are a mechanism allowing communication between processes over the network. In some cases, you can use TCP/IP sockets to talk with processes running on the same computer (by using the loopback interface).

 

UNIX domain sockets know that they’re executing on the same system, so they can avoid some checks and operations (like routing); which makes them faster and lighter than IP sockets. So if you plan to communicate with processes on the same host, this is a better option than IP sockets.

 

Edit: As per Nils Toedtmann's comment: UNIX domain sockets are subject to file system permissions, while TCP sockets can be controlled only on the packet filter level.

  - Copied from https://serverfault.com/a/124518

 

TL;DR and a simple version of that thing above is as follows:

When you have a server₅ it has to "listen" to a socket, so clients can actually talk to it and request stuff. When a service is bound (when it "listens") to IP sockets - also known as "ports" - it's reachable over the network. When it's bound to a UNIX socket it communicates with its clients over a file on your filesystem (it's more complicated than that, but for you at the moment... ...well that's what it looks like. It's a file, when on linux you usually find them in /var/run/SERVICE/SERVICE.sock or /var/run/SERVICE.sock where "SERVICE" is replaced with the service's name  )

 

Your reverse proxy should bind to IP socket (port) 80/tcp (aka "port 80") which is the default port for HTTP traffic. In case you'll want to setup HTTPS too, you'll have to tell it to bind to port 443/tcp too. And you'll need to setup certificates for your domain. But this is beyond the scope of this "project".

 

Quote

TCP is a connection oriented stream over an IP network. It guarantees that all sent packets will reach the destination in the correct order. This imply the use of acknowledgement packets sent back to the sender, and automatic retransmission, causing additional delays and a general less efficient transmission than UDP.

 

UDP is a connection-less protocol. Communication is datagram oriented. The integrity is guaranteed only on the single datagram. Datagrams reach destination and can arrive out of order or don't arrive at all. It is more efficient than TCP because it uses non ACK. It's generally used for real time communication, where a little percentage of packet loss rate is preferable to the overhead of a TCP connection.

 

In certain situations UDP is used because it allows broadcast packet transmission. This is sometimes fundamental in cases like DHCP protocol, because the client machine hasn't still received an IP address (this is the DHCP negotiaton protocol purpose) and there won't be any way to establish a TCP stream without the IP address itself.

    - Copied from https://stackoverflow.com/a/5970545

 

 

the API

 

This is your - if I remember correctly - java-based server. Since we're running the reverse proxy on the same system as the API, you should really just bind this server to a UNIX socket. If you want you can bind it to a IP socket too, but it's useless, because all your clients should access it through the proxy anyway.

 

The API is the part of the backend which will **actually** process the requests. When a client posts a new message it will send you a request which will be telling you what the content it, who the client is and in what channel they are posting. now all you have to do is verify that the client is actually logged in (if you'll have authentication), parse the content to make sure you don't have an XSS attack in the content itself (or any other "malicious stuff") - you can do that simply by only allowing ASCII characters, it will make your chat app restrictive as hell, but at least it'll be "safer" and easy enough to implement - then you save the record and notify all clients about the new message.

 

About notifying clients ... this might be a bit more complicated - so you should really just stick to "pooling" instead of "web sockets". Google for more info, but TL;DR is - with websockets you keep the client<>server connection alive so not only the client can "start the convo" with the server, but also the server can just tell the client "hey, take this new data and process it". In comparison to this, "pooling" is the method of having the client ask the server "is there anything new" every {{ insert your short interval here | one second? }}. Instead of having an open connection between the server and the client where the client can reach out to the server and the server can reach out to the client too, you just have the client spamming the server every second for new events. Websocket are the more "elegant" (and better) approach, but pooling is much easier to setup.

If I were you I would go with pooling in this case. It's a simple app that should never to go production, it'll be served over plain HTTP, so everyone on the network can see what you're seeing... Don't bother.

 

static files

 

Well these are the "website" files that your reverse proxy will serve to the browsers when they visit your website. They don't have the chat client, the UI, like the android app will have, so you need to serve the UI to them. They will, just like the android clients, communicate with the API, but they first need the logic that allows them to even know how to communicate and this is what these files are here to do.

 

All you need is a simple html file with basic structure, a css file for styling and a JS file for some basic logic. You probably know this already for some time now, so Imma stop here

 

- - - -

1) Service - I am referring to an application / program which does not have an UI and is run on a server (non-client machine). An example would be a webserver, database, cron etc. It does not have to be a server that serves data. It's just a piece of software that runs on your machine in the background doing stuff that it's made to do.

2) database - a database server₅. Presumably mariadb, postgresql, mongodb, firebase, firebird, sqlite etc.

3) reverse proxy - usually an HTTP server₅ which is used to proxy traffic to services that are not to be access directly. Eg. use NGINX to proxy all requests that are posted to /api/v1 to the old API service, proxy all /api/v2 requests to the new API service and for all other requests serve static files from /var/com.myapp/static folder.

4) API - application programming interface. This is basically the "brain" of your application. Your clients should NEVER communicate directly with the database, and this is what makes sure that they don't.

5) Server - it's a service that serves stuff. Be it to clients (users), eg. a web server serving web content to your browser, or to other services, like a database that serves data to your API₄.

 

 

I'm sure I have not explained some stuff that is obvious to me, but isn't to you. As said above - email me (well you can probs also just tag me and I should get the mail notification... I dunno, try it I guess, we'll see if I'll reply in less than an hour )

 

Also, if you need a server to test stuff - feel free to ask - we'll hop in DMs and I can give you a VM for free. Can't give you a public ip, because I don't have any left, but you will still be able to access the server directly via a VPN :thonk:

[QChoumont]

@Aljaxus Thank you so much for your reply, i've been chewing on it for a few weeks now and it really put things into perspective for me. I've got my server up and running, I decided on WAMP because its pretty well documented and easy to set up. I've got a static public ip and its working.

I loved the idea of using the web-server as a proxy server but my team opted to have the clients communicated directly to the application server. Frankly when you responded I didn't have the slightest clue what a proxy server was, or even what a web-server was.

Again thank you so much I wouldn't have been able to even start this project without this post.

[Aljaxus]

On 10/6/2020 at 5:17 PM, QChoumont said:

@Aljaxus Thank you so much for your reply, i've been chewing on it for a few weeks now and it really put things into perspective for me. I've got my server up and running, I decided on WAMP because its pretty well documented and easy to set up. I've got a static public ip and its working.

I loved the idea of using the web-server as a proxy server but my team opted to have the clients communicated directly to the application server. Frankly when you responded I didn't have the slightest clue what a proxy server was, or even what a web-server was.

Again thank you so much I wouldn't have been able to even start this project without this post.

I have no clue about what your project looks like, how it works, not how it will be used. But if there's one advice that I can give you it's that you should really, really move away from application hosting on Windows as soon as possible. Sure, xampp or (wampp) are very easy to setup - but running things on Apache is not the best way of doing things by far (a very simple attack is slow loris and there's quite a few more...).

 

Welp, I "tossed" my two cents, have a good one