Search for valid URL in python 3.7

[boey]

So I have a task, where I have to find a valid URL. 

 

Let’s say the URL is www.123.com/<random 4 digit number>

we have to find that 4 digit number to access the correct page. How would I go about doing this?

 

Importing webbrowser and opening every page from 0000 to 9999 is not a good idea because I’m on 16GB of RAM.

[Mephi00]

WIth Java youd just check if the host responses, thats all I can tell you. Maybe you find a way to do that in python

[duncannah]

5 hours ago, _d0nut said:

Importing webbrowser and opening every page from 0000 to 9999 is not a good idea because I’m on 16GB of RAM.

That’s the only way you can go through this, unfortunately. RAM shouldn’t matter unless you got a memory leak

[vorticalbox]

 

2 hours ago, duncannah said:

That’s the only way you can go through this, unfortunately. RAM shouldn’t matter unless you got a memory leak

Not true you do not need to open any web browser you can make HTTP requests inside python

 

 

Assuming the website returns an error code E.G 404, for invalid pages this will work

import requests
for number in range(10000):
    r = requests.get('http://www.123.com/%d' % number)
    if r.status_code == requests.codes.ok:
      print('Valid')
    else:
      print('not valid')

[boey]

6 minutes ago, vorticalbox said:

 

Not true you do not need to open any web browser you can make HTTP requests inside python

 

 

Assuming the website returns an error code E.G 404, for invalid pages this will work

import requests
for number in range(10000):
    r = requests.get('http://www.123.com/%d' % number)
    if r.status_code == requests.codes.ok:
      print('Valid')
    else:
      print('not valid')

I managed to solve it. It was a time pressure exercise, but I ended up using webbrowser and opening 5000 tabs in windows.

[vorticalbox]

21 minutes ago, _d0nut said:

I managed to solve it. It was a time pressure exercise, but I ended up using webbrowser and opening 5000 tabs in windows.

yeah you should try my solution out will use basically use no ram, only problem is it has to do one at a time perfect use case to learn threading

[MladenM]

On 6/21/2019 at 3:46 PM, _d0nut said:

I managed to solve it. It was a time pressure exercise, but I ended up using webbrowser and opening 5000 tabs in windows.

You should try @vorticalbox solution and also you should understand why that soltuion works and why is better. 
Also would not hurt to learn about HTTP protocol and get request while you on the task. Could be useful.

Cheers. 

[boey]

8 hours ago, MladenM said:

You should try @vorticalbox solution and also you should understand why that soltuion works and why is better. 
Also would not hurt to learn about HTTP protocol and get request while you on the task. Could be useful.

Cheers. 

I assume requests is a module i have to install with pip? i tried to import requests on the school computer and it said it was an invalid module. couldn't install pip either since it was a school computer

[MladenM]

On 6/28/2019 at 8:27 AM, _d0nut said:

I assume requests is a module i have to install with pip? i tried to import requests on the school computer and it said it was an invalid module. couldn't install pip either since it was a school computer

yes thats right.
Pip install requests

or pip3 install requests.

Here is documentation for requests

https://2.python-requests.org/en/master/

[vorticalbox]

On 6/28/2019 at 7:27 AM, _d0nut said:

I assume requests is a module i have to install with pip? i tried to import requests on the school computer and it said it was an invalid module. couldn't install pip either since it was a school computer

while you're lookign into things look at virtualenv too

 

https://virtualenv.pypa.io/en/stable/