Create a website using Python - server side development

[panther420]

Quick disclaimer - I'm quite new to server side web development and I've just recently started doing research so my apologies if anything I'm saying doesn't make any sense. I'm familiar with HTML, CSS, JavaScript, and Python for offline programming, but I'm finding that I'm way over my head making this confusing advance from client side to server side.

 

So some time ago I decided I want to make a personal website with various features, like a place to host some of  my coding projects , maybe a small blog or something, and a URL shortener. I've already made the decision between buying a server to run at home and purchasing from a provider, but now I'm looking at how I want to accomplish the features my website will have and how I want to build it. Specifically, I'm looking at which web server to use and which frameworks and server-side languages to use. I figured that since I already know Python, a framework like Django or Flask would be a good choice instead of having to learn PHP. But for some reason, the resources I'm finding online for Django and Flask (or really any alternative to a simple LAMP stack) are talking about dynamic web pages and REST apis and reverse proxies and all this complicated crap - do I need to learn all of these things if all I want to do is run the equivalent of a LAMP stack with Python in place of PHP? I'm completely over my head - I want to make a simple website, probably just a set of static pages with some scripts and a database to do the URL shortener thing. Is there maybe an Apache plugin for Python like there is for PHP? How do I set this up?

 

I would ask on a site like StackOverflow for this, but I'm worried that it will get removed for being too opinion-based or for being a duplicate of an unhelpful question from 5 years ago. I find that the folks here are much less elitist when it comes to helping computer noobs.

 

tl:dr; Looking for a way to run Python for a simple, static website without having to learn complicated web paradigms that appear to be geared towards much larger scale websites.

Sorry if it didn't make any sense or the answer was obvious and I didn't do my research (didn't seem like that was the case to me).

[geo3]

use the http.server package to serve up your html files.

[panther420]

3 hours ago, geo3 said:

use the http.server package to serve up your html files.

That simple? How does this differ or relate to using Apache or Nginx? Why one method over the other?

[geo3]

6 minutes ago, panther420 said:

That simple? How does this differ or relate to using Apache or Nginx? Why one method over the other?

If you just want a static site, yes, it's pretty simple. Apache is a full framework with tons of features, also Java based, and is completely overkill for what you're doing. Never used Nginx. 

[bowrilla]

Did you have a look at Django or Flask documentation? Basically all the web frameworks for Python, Ruby and Javascript work in very similar ways: create a server and make it listen (this can be an external server like Apache with Rack middleware or the built in one like in node.js, the internal ones of Django, Rails and Sinatra are more dev servers than deployment ones), handle your routes in your main app, answer with the page you want it to show. You might need to have a look at the specific ways your framework handles templates and answer structures. 

 

Personally speaking, even though Python is a lovely language, it is not (my) first choice for development of web apps. It can be done and there are several examples out there in the wild but the Ruby community is bigger and still no match for node.js. With node.js and Javascript you save yourself one language in your project.

[panther420]

4 hours ago, bowrilla said:

Did you have a look at Django or Flask documentation? Basically all the web frameworks for Python, Ruby and Javascript work in very similar ways: create a server and make it listen (this can be an external server like Apache with Rack middleware or the built in one like in node.js, the internal ones of Django, Rails and Sinatra are more dev servers than deployment ones), handle your routes in your main app, answer with the page you want it to show. You might need to have a look at the specific ways your framework handles templates and answer structures. 

 

Personally speaking, even though Python is a lovely language, it is not (my) first choice for development of web apps. It can be done and there are several examples out there in the wild but the Ruby community is bigger and still no match for node.js. With node.js and Javascript you save yourself one language in your project.

That was exactly what I was taking about in my original post, none of the terms you just used made any sense to me. For example, what do you mean "handle your routes in my main app?" What's my main app in this case? Are you talking about a specific python script that represents my main app? All of this just seems way too technical for what I'm trying to accomplish.

 

addendum: to be clear, I would say that I'm also not making a web app. I'm not entirely sure what constitutes a web app, if it's just a website with some purpose or function or what it is, but from the looks of it all this web app architecture is completely unecessary for what I'm trying to accomplish and is just irrelevant. In fact, it's getting quite annoying that everything has to be about web apps and dynamic page loading nowadays.. 

[bowrilla]

On 13.5.2018 at 4:30 PM, panther420 said:

[…]

 

Well, your urls are adresses. For example http://something.test/ is something else than http://soemthing.test/abc and so on. Those are routes. With an Apache webserver those routes are resolved automatically by the structure of your file system. In a Flask, Sinatra, Rails, etc application you define those routes manually and tell your web framework how to react when a specific route is requested. Logicwise it's basically a lot of if clauses or switch case constructions. For the very lightweight frameworks (like Sinatra) that's basically it to get it running. Some frameworks/languages require you to setup a webserver that listens on a specific port (i.e. node) others work together with Apache or nginx via a rack middleware like phusion passenger (i.e. ruby). Some frameworks offer a development server (like Ruby on Rails).

When using a more complex framework with a higher degree of structure you need to follow the basic concept. For Ruby on Rails it's MVC: Model View Controller. You have to split up logic, templates and data. Other frameworks allow you to choose a different approach and just answer a client's request with single documents or statements.

 

Before you begin, you should think about what you're trying to acchieve and choose your tools accordingly. Python is not the best choice though since its market share for web apps isn't that big compared to like PHP, Javascript or Ruby. It's a good choice though if you're web app requires a lot of complex math. Performancewise it's not a good choice though compared to Javascript/node for example.

For a very simple, static website, going for server side scripting lagnuages is not neccessary. If every client will get the same response on its requests there's no real logic running on the server, that Apache couldn't handle easily. Of course, if you just want to learn a thing or two than just give it a try. I recommend some online tutorials after making a decision for a language and framework.

P.S.:

And back to your original plan: making a blog surely requires some serverside scripting and potentially client-side dynamic loading of content with AJAX. Hosting your web apps isn't really relevant here imho, it's more a question on what kind of server you get. A url shortener … well, google has a working API you can just implement with some AJAX calls.

So you'll need two things, if you really want to write your own blog: some server side framework of your choice and some frontend framework for dynamic calls (probably either Vue, React or Angular, those are the big three atm). Personally I'd choose node as a server side platform and express as a lean framework. Saves you from switching between languages for front- and backend.

 

Codecademy has some pretty good introduction crash courses for node and react. Those should explain all you need, give you some practice and after that you're basically good to go. And best of it all: the free courses are all you need. They don't offer anything for Python and Flask though.

[burnttoastnice]

On 12/05/2018 at 10:53 PM, panther420 said:

[snip]

I currently do this on my own webserver (lighttpd). I opted not to use a pre-made framework (like Django etc) and designed my own instead because I wanted web application code to be easy to port between PHP and Python, and to be able to interchangeably use both languages in a single web application.

 

When you set this up on your webserver, make sure you add ".py" files as an exclusion to the static file extensions list. The configuration I use on my webserver:

$HTTP["url"] =~ "^(.*)\/cgi-bin(.*)" {
        cgi.assign = ( ".py" => "/usr/bin/python3", ".cgi" => "" )
}

This allows .py files to be executed by python3 as long as they are in a folder called cgi-bin, (so if I was to host a .py file for downloading it would download as intended, and not return a webpage with the code output). Files with the .cgi extension are executed as binary applications (you won't need that, I only have that included for testing purposes).

 

You might be wondering how form data submitted online gets to the .py script. This data is sent into your script through its standard input (stdin) and placed into environment variables, which can be accessed by the python script. Working with the data directly isn't very feasible, hence why I made a library to extract, interpret and place this data into workable python lists and dictionaries. Here's what the logout code for one of my python web apps look like:

Spoiler

 

import sys;
sys.path.append("/path/to/other/python/libraries");
import pywebtk.v1r2p2 as cgi;
import json;

environment = cgi.environment();
session = cgi.session('application_name');
api_response = {};

cgi.headers.send('Cache-Control: no-cache, no-store, must-revalidate');

################################################################################################################################
if not environment.https_on():
	cgi.headers.flush();
	print(json.dumps({'result':'error','description':'https-not-enabled-on-active-connection'}));
	exit(0);
################################################################################################################################

if session.debug()['cookie needed'] is False:
	session.start();
	if 'loggedin' in session.data and session.data['loggedin'] == True:
		session.data = {};
		session.close();
		session.delete();

		api_response['result'] = 'success';
		api_response['description'] = 'logged out'; # test passed

	else:
		api_response['result'] = 'error';
		api_response['description'] = 'Already logged out, please log in first'; # test passed

else:
	api_response['result'] = 'error';
	api_response['description'] = 'you do not meet the requirements to log out'; # test passed

cgi.headers.flush();
print(json.dumps(api_response));

 

Using a premade library such as django is miles easier than writing your own, as you likely wouldn't have to worry about creating secure sessions etc, managing logins/cookies and the like.

[vorticalbox]

As I work in node the only thing I can recommend is express, there is an npm package that will build a basic website structure for you.

 

npm i -g express-generator

express --pug myapp

CD myapp 

npm i 

npm start

[panther420]

14 hours ago, bowrilla said:

 

Well, your urls are adresses. For example http://something.test/ is something else than http://soemthing.test/abc and so on. Those are routes. With an Apache webserver those routes are resolved automatically by the structure of your file system. In a Flask, Sinatra, Rails, etc application you define those routes manually and tell your web framework how to react when a specific route is requested. Logicwise it's basically a lot of if clauses or switch case constructions. For the very lightweight frameworks (like Sinatra) that's basically it to get it running. Some frameworks/languages require you to setup a webserver that listens on a specific port (i.e. node) others work together with Apache or nginx via a rack middleware like phusion passenger (i.e. ruby). Some frameworks offer a development server (like Ruby on Rails).

When using a more complex framework with a higher degree of structure you need to follow the basic concept. For Ruby on Rails it's MVC: Model View Controller. You have to split up logic, templates and data. Other frameworks allow you to choose a different approach and just answer a client's request with single documents or statements.

 

Before you begin, you should think about what you're trying to acchieve and choose your tools accordingly. Python is not the best choice though since its market share for web apps isn't that big compared to like PHP, Javascript or Ruby. It's a good choice though if you're web app requires a lot of complex math. Performancewise it's not a good choice though compared to Javascript/node for example.

For a very simple, static website, going for server side scripting lagnuages is not neccessary. If every client will get the same response on its requests there's no real logic running on the server, that Apache couldn't handle easily. Of course, if you just want to learn a thing or two than just give it a try. I recommend some online tutorials after making a decision for a language and framework.

P.S.:

And back to your original plan: making a blog surely requires some serverside scripting and potentially client-side dynamic loading of content with AJAX. Hosting your web apps isn't really relevant here imho, it's more a question on what kind of server you get. A url shortener … well, google has a working API you can just implement with some AJAX calls.

So you'll need two things, if you really want to write your own blog: some server side framework of your choice and some frontend framework for dynamic calls (probably either Vue, React or Angular, those are the big three atm). Personally I'd choose node as a server side platform and express as a lean framework. Saves you from switching between languages for front- and backend.

 

Codecademy has some pretty good introduction crash courses for node and react. Those should explain all you need, give you some practice and after that you're basically good to go. And best of it all: the free courses are all you need. They don't offer anything for Python and Flask though.

Thanks a bunch for the explanation and clarification. I've given the website some thought and I've had a little bit of a shift in the website's focus that will hopefully allow for me to more succinctly explain the goal I wanted from the beginning, and that way you can point me in the right direction. I've also shifted in my opinions on what languages and frameworks I'm willing to use to accomplish it.

 

"URL shortener" wasn't really what I was after. Instead, I wanted some kind of file uploading service private to me, where I could upload a file to the website and create a link to that file that anyone can download (or preview in the browser if possible) once they have the link. Sort of like a super simplified Google Drive, and where I'm the only one that can upload to it (so I'd need to setup a little login page for it). I probably don't even need to access the files I've previously uploaded, and I would probably set it so the file automatically expires and gets deleted in a certain amount of time. I'd also like it to be able to interface with my NAS and store it there instead of on the webserver's local storage, as I don't imagine I'll have much to play with. I also don't think I'll even need the other features I proposed for the website, like the blog. I would really like to focus on this one feature, but for someone who's never done server side development, it's quite a daunting task.

 

So what I'm wondering is if you could point me to some guides or free online classes that allow me to get comfortable with server side development, as I imagine that even if I find a tutorial online telling me how to do exactly this, I would become lost quickly. I think a quickstart guide for a language and framework would be helpful, showing me how to get a server set up and what not. This time I'm open to just about any language, including PHP. I would rather use something like Node if possible but if PHP is the easy way to go then I'd be happy to do so. I'll therefore let you recommend me a language after you hopefully know more clearly what I'm trying to accomplish and what would suit my needs best. And any other resources you think would be helpful would be great too. 

 

Thanks again for all your help.

[panther420]

8 hours ago, vorticalbox said:

As I work in node the only thing I can recommend is express, there is an npm package that will build a basic website structure for you.

 

 

npm i -g express-generator

express --pug myapp

CD myapp 

npm i 

npm start

 

Cool. I'm thinking of using Express, seems pretty nice.

[panther420]

9 hours ago, burnttoastnice said:

I currently do this on my own webserver (lighttpd). I opted not to use a pre-made framework (like Django etc) and designed my own instead because I wanted web application code to be easy to port between PHP and Python, and to be able to interchangeably use both languages in a single web application.

 

When you set this up on your webserver, make sure you add ".py" files as an exclusion to the static file extensions list. The configuration I use on my webserver:

$HTTP["url"] =~ "^(.*)\/cgi-bin(.*)" {
        cgi.assign = ( ".py" => "/usr/bin/python3", ".cgi" => "" )
}

This allows .py files to be executed by python3 as long as they are in a folder called cgi-bin, (so if I was to host a .py file for downloading it would download as intended, and not return a webpage with the code output). Files with the .cgi extension are executed as binary applications (you won't need that, I only have that included for testing purposes).

 

You might be wondering how form data submitted online gets to the .py script. This data is sent into your script through its standard input (stdin) and placed into environment variables, which can be accessed by the python script. Working with the data directly isn't very feasible, hence why I made a library to extract, interpret and place this data into workable python lists and dictionaries. Here's what the logout code for one of my python web apps look like:

  Reveal hidden contents

 

import sys;
sys.path.append("/path/to/other/python/libraries");
import pywebtk.v1r2p2 as cgi;
import json;

environment = cgi.environment();
session = cgi.session('application_name');
api_response = {};

cgi.headers.send('Cache-Control: no-cache, no-store, must-revalidate');

################################################################################################################################
if not environment.https_on():
	cgi.headers.flush();
	print(json.dumps({'result':'error','description':'https-not-enabled-on-active-connection'}));
	exit(0);
################################################################################################################################

if session.debug()['cookie needed'] is False:
	session.start();
	if 'loggedin' in session.data and session.data['loggedin'] == True:
		session.data = {};
		session.close();
		session.delete();

		api_response['result'] = 'success';
		api_response['description'] = 'logged out'; # test passed

	else:
		api_response['result'] = 'error';
		api_response['description'] = 'Already logged out, please log in first'; # test passed

else:
	api_response['result'] = 'error';
	api_response['description'] = 'you do not meet the requirements to log out'; # test passed

cgi.headers.flush();
print(json.dumps(api_response));

 

Using a premade library such as django is miles easier than writing your own, as you likely wouldn't have to worry about creating secure sessions etc, managing logins/cookies and the like.

Check out my newest post, I've decided that I'm not necessarily using python anymore. That's a little technical for me as well. But thanks anyway for the information.

[vorticalbox]

3 hours ago, panther420 said:

Cool. I'm thinking of using Express, seems pretty nice.

If you need any help send me a pm or tag me in a forum post and I'll be happy to help when I have a little free time

[Fire Yoshi]

I like CherryPy for simple projects : https://cherrypy.org/

[bowrilla]

10 hours ago, panther420 said:

"URL shortener" wasn't really what I was after. Instead, I wanted some kind of file uploading service private to me, where I could upload a file to the website and create a link to that file that anyone can download (or preview in the browser if possible) once they have the link. Sort of like a super simplified Google Drive, and where I'm the only one that can upload to it (so I'd need to setup a little login page for it). I probably don't even need to access the files I've previously uploaded, and I would probably set it so the file automatically expires and gets deleted in a certain amount of time. I'd also like it to be able to interface with my NAS and store it there instead of on the webserver's local storage, as I don't imagine I'll have much to play with. I also don't think I'll even need the other features I proposed for the website, like the blog. I would really like to focus on this one feature, but for someone who's never done server side development, it's quite a daunting task.

Well, basically every server side scripting language is able to get something like that going. Since you're using it on your own, performance isn't a big issue as well (I assume there won't be millions of requests to your server each day).

 

At the end it's a matter of preference. 

PHP can do that job. You just have to deal with PHP and that might be an issue since PHP isn't really know to be the most elegant language. It's super easy though to get a server running – Apache with PHP modules is all you need and there are tons of resources out there to answer your questions. PHP is the classic way of doing it since the mid/end 90s. It evolved and became more powerful but that's what makes PHP also kind of "ugly" (personal opinion).

You can go with Javascript/node. You save yourself one scripting language since you can use Javascript for front- and backend, there are a lot of resources out there, node is pretty powerful and the community is pretty big. Big companies like eBay, Netflix, etc chose it and it's really powerful. You can also use Typescript or Coffeescript to get a more beautiful syntax if you want. Node gives you a server built in so no need for rack middleware.

Ruby with Ruby on Rails. It was all the rage a few years back. The community is still strong and there are a lot of resources. Ruby is a special kind of language as it offers you many ways to achieve your goal but that's also what's putting off some people. Companies like AirBnB and GitHub use it – or used to use it. You need a middleware for production use cases and afaik Phusion Passenger isn't running on Windows machines. 

Python with Django. It's pretty similar to Ruby languagewise. It offers a lot of science related modules and libraries and is pretty popular with scientists. The community is a bit a smaller though (might change though) and therefore there are a little less ressources around. Never used it but if I'm not totally confused it needs rack middleware as well.

Personal choice: node. I just like it the most and if your want to be able to use cutting edge web technologies at some point node and Javascript are the way to go. WebGL and WebAssembly were made for Javascript for example and only got ported later on (if at all, though both are frontend technologies). 

Have a look at codecademy for their crash courses. They offer a node and sqlite course and a full express course. That should get you started. They offer a RoR course as well. You can also find some React and Angular courses – the biggest frontend frameworks around.