[Official][Update 8/15] Linus Tech Tips API

[loganryan99]

With the way current LTT notifiers work, they constantly have to reload the index page of the forum and parse the messy HTML and stuff, and download all of the images just to get the notification count. I figured I would make a simple way to fix this.

 

API URL: https://linustechtips.com/main/page/api.php

 

This page returns a JSON array with some data about the member currently logged in. Any future LTT addons should use this as it increases speed and reduces bandwidth

Functionality will be expanded in the future.

 

Note: When there is no user logged in the display_name value will be set to Guest.

 

If you're developing an extension outside of a browser, you'll have to figure out a way to send a POST request to the server to log the user in. I'm not even sure this is possible at this point because IPB might have some kind of security for that.

 

Update 8/14:

Use this URL: http://linustechtips.com/main/?app=core&module=global&section=login?cm_api=true to process sign in from external apps. You should be able to just display it to the user then save the cookies it gives you somehow. I'll publish my implementation of it when it's ready.

 

Update 8/15:

Using this URL: http://linustechtips.com/main/index.php?app=core&module=usercp&tab=core&area=notificationlog&cm_notif_api=true you can get user notifications encoded in JSON.

Edited February 11, 2016 by colonel_mortis
Struck out unsupported features

[MattP123]

It don't wanna work today. 

[Askew]

It don't wanna work today. 

 

You mean you didn't understand it today.

[MattP123]

You mean you didn't understand it today.

[Levent]

Why not let the user create their API and give an unique ID to each API created ,this would give user an ability to choose if they want API or not and that would make integration MUCH more easier for us

@

[loganryan99]

Why not let the user create their API and give an unique ID to each API created ,this would give user an ability to choose if they want API or not and that would make integration MUCH more easier for us

@

 

We're going to have a better system once we upgrade IPB versions. This is just something I threw together in a few minutes to make things slightly easier

 

EDIT: I should mention that this future API is mainly for internal use and I can't guarantee that LMG will let me open it up to everyone.

[colonel_mortis]

Looks great; I can't wait to implement it. Perhaps this should be in forum news and info? (it would be more official and essentially sticky)

[linusforsell]

Can't wait to start calling this API instead of scraping the site. It'll be so much easier.  :lol:

 

Can we expect query access to PM and notification details (when the active user has access, of course) to land at some point? Such as the message body, or events from a notification and what not.

[TizzyT]

Is this JSON(looks like it)? and how would one go about logging in?

[fizzlesticks]

Is this JSON(looks like it)? and how would one go about logging in?

Yes it is JSON and here's a quick Python script for logging in and calling the new API, it doesn't do any error checking and I don't know if the "auth_key" ever changes.

import urllib, urllib.request, http.cookiejar, time, jsonuser = '<your username>'password = '<your password>'loginURL = 'https://linustechtips.com/main/index.php?app=core&module=global&section=login&do=process'apiURL = 'http://linustechtips.com/main/page/api.php'cj = http.cookiejar.MozillaCookieJar()opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))opener.addheaders = [('User-agent', 'Opera/9.80 (Windows NT 6.1; WOW64; U; de) Presto/2.10.289 Version/12.01')]login_data = urllib.parse.urlencode({'ips_username' : user,                               'ips_password' : password,                               'auth_key' : '880ea6a14ea49e853634fbdc5015a024',                               'referer' : 'http://linustechtips.com/main/',                               'rememberMe' : '1'})resp = opener.open(loginURL,login_data.encode('utf-8'))while True:        resp = opener.open(apiURL)    text = resp.read().decode()    resp.close()    js = json.loads(text)    print('User name: ' + js['display_name'])    print('New notes: ' + js['new_notifications'])    print('New pms: ' + js['new_pms'])    time.sleep(5)

[loganryan99]

Can't wait to start calling this API instead of scraping the site. It'll be so much easier.  :lol:

 

Can we expect query access to PM and notification details (when the active user has access, of course) to land at some point? Such as the message body, or events from a notification and what not.

 

Yes, should come sometime after the upgrade to IPB 4

[colonel_mortis]

Do you think you would be able to add a notification and a message etag so that the addon can see whether the resources have changed since the last time it was checked? It would mean that my ff addon would only ever need to load the actual site when there is new content (to prevent no change being visible due to notifications being cleared before the addon sees them (if you get what I mean (It's late and I'm tired so it probably makes no sense))).

[Cheesebaron]

A couple of observations and requests for the current API. I get the following payload right now when I access the API:

{   "display_name" : "Cheesebaron",  "new_notifications" : "1",  "new_pms" : "0",  "profile_img" : "http:\/\/linustechtips.com\/main\/uploads\/profile\/photo-thumb-122.png?_r=0"}

Firstly the profile_img field's value is strangely escaped, even though we have quotes around the string. That is really unnecessary in JSON.

 

It would be really useful if there was a date telling when a pm/notification was last received, such that you can compare with a local value.

 

Maybe expand the payload such that it included the description of the latest notifications and titles and maybe a line of text from a PM:

{  "display_name" : "Cheesebaron",  "profile_img" : "http:\/\/linustechtips.com\/main\/uploads\/profile\/photo-thumb-122.png?_r=0",  "new_pms" : "2",  "new_notifications" : "1",  "latest_pms" : [    {      "url" : "xxxx",      "received_date" : "2014-08-5T19:43:37+0200",      "title" : "hurrdurr",      "body" : "durr durr durr"    },    {      "url" : "xxx",      "received_date" : "2014-08-5T19:44:37+0200",      "title" : "hurrdurr",      "body" : "durr durr durr"    }  ],  "latest_notifications" : [    {      "type" : "quote",      "thread_url" : "xxx",      "time" : "2014-08-5T19:44:37+0200",      "description" : "xxx"    }  ]}

Surely this is just a draft of what it could look like, but there might be some other information that would be cool to have in the API. Or even multiple URLs for different kind of data.

[colonel_mortis]

@ Is the response header always 200 if the forum is up (even if the user is logged out)? Because I'm just installing the sanity checks for the ltt notifier.

[linusforsell]

@ Is the response header always 200 if the forum is up (even if the user is logged out)? Because I'm just installing the sanity checks for the ltt notifier.

 

Yep, as long as the server manages to respond it'll return a 200 header

[colonel_mortis]

Yep, as long as the server manages to respond it'll return a 200 header

Thanks! That makes sanity checks a whole load easier (shorter anyway)

[fizzlesticks]

@ Is the response header always 200 if the forum is up (even if the user is logged out)? Because I'm just installing the sanity checks for the ltt notifier.

If the user isn't logged in it returns a guest user.

{"display_name":"Guest","profile_img":"http:\/\/linustechtips.com\/main\/public\/style_images\/day_theme_images_new\/profile\/default_large.png?_r=0","new_pms":null,"new_notifications":null}

[colonel_mortis]

If the user isn't logged in it returns a guest user.

{"display_name":"Guest","profile_img":"http:\/\/linustechtips.com\/main\/public\/style_images\/day_theme_images_new\/profile\/default_large.png?_r=0","new_pms":null,"new_notifications":null}

Yeah, I was just checking that I didn't have lots of different HTTP codes all indicating that the server was up because I will use it as an indication that it's offline.

[colonel_mortis]

The API doesn't seem to work over HTTPS - I get a 404 error. It's not necessary, but I would prefer to be requesting to an API for a product that I plan using an encrypted connection.

[fizzlesticks]

The API doesn't seem to work over HTTPS - I get a 404 error. It's not necessary, but I would prefer to be requesting to an API for a product that I plan using an encrypted connection.

It works for me using 'https://www.linustechtips.com/main/page/api.php' I had to add the www.

[colonel_mortis]

It works for me using 'https://www.linustechtips.com/main/page/api.php' I had to add the www.

Yeah, but that only works because it redirects to the http base domain. It's not encrypted at all unfortunately.

[loganryan99]

Yeah, but that only works because it redirects to the http base domain. It's not encrypted at all unfortunately.

 

This is out of my control as we are at the mercy of IP.Content but I don't see why it would matter. No sensitive data is being transferred via the API, and most of this site isn't SSL anyways.

[colonel_mortis]

This is out of my control as we are at the mercy of IP.Content but I don't see why it would matter. No sensitive data is being transferred via the API, and most of this site isn't SSL anyways.

Yeah I guess it doesn't really matter at this stage, but when the API is expanded to include stuff like PMs, it's more important IMO. I just like to know that my users will be using the safest connection available.

[loganryan99]

Yeah I guess it doesn't really matter at this stage, but when the API is expanded to include stuff like PMs, it's more important IMO. I just like to know that my users will be using the safest connection available.

 

When we upgrade to 4.0 it will be outside of IP.Content which will let us use SSL

[colonel_mortis]

When we upgrade to 4.0 it will be outside of IP.Content which will let us use SSL

Will that break the current URLs then? Because I wouldn't want to rely on them if they will break at some point in the fairly near future, even if it just means adding code to ignore it if the link is broken.