Squid Proxy Server for Caching

[Jackpot17]

Recently I've become more and more interested in home networking and looking for projects I can try etc.

I know only a small amount about proxy server caching and it has interested me.

I have a Chromebox (NUC style PC) and a 320GB external hard drive connected with decent read/write speeds and my network is gigabit. (I know the Chromebox setup isn't ideal but it works)

I have 40mbps down so speeds would be increased greatly I assume.

I've heard about Squid on Linux but can't find much information on how to configure it for what I would like to do.

How would I go about making a server for caching on my network to make previously visited websites and steam library updates quicker? (My brother and I both have large steam libraries)

 

ps. I'm also interested in blocking ads, could this be done using Squid?

 

Help appreciated.

 

 

[brwainer]

40 minutes ago, Jackpot17 said:

Recently I've become more and more interested in home networking and looking for projects I can try etc.

I know only a small amount about proxy server caching and it has interested me.

I have a Chromebox (NUC style PC) and a 320GB external hard drive connected with decent read/write speeds and my network is gigabit. (I know the Chromebox setup isn't ideal but it works)

I have 40mbps down so speeds would be increased greatly I assume.

I've heard about Squid on Linux but can't find much information on how to configure it for what I would like to do.

How would I go about making a server for caching on my network to make previously visited websites and steam library updates quicker? (My brother and I both have large steam libraries)

 

ps. I'm also interested in blocking ads, could this be done using Squid?

 

Help appreciated.

 

 

This is something I've thought about trying before but never really started on. My reason is that without a lot of users hitting the same sites, I figure the amount of speedup would be minimal. Your browser already does its own caching of things like images on websites. Steam and other updates rarely have the same data in them, so unless you and your brother have lots of the same games I wouldn't expect much usable data to get cached. But if you try it and report back on any improvements seen, maybe I'll give it a go.

 

Blocking ads is something I think you can do with a Squid proxy, but I honestly don't know for sure.

[leadeater]

As @brwainer said proxy servers really come in to their own for networks with many users rather than 1 or just a few. I have setup Squid for home use before and configured it for some very aggressive caching and forcing some things to cache explicitly so it does give a decent speed up but only when you rewatch the same youtube video or go to the same website very often.

 

I used to play a browser based game that involved a lot of page loads so for that Squid gave a huge improvement, I could do a click every second rather than every 3-10 seconds.

 

I would say do this project anyway, it's worth it just for the experience and knowledge. Also have a look at using DansGuardian for web filtering/ad blocking, it's extremely advanced in comparison to what Squid can do by itself.

[Jackpot17]

18 minutes ago, leadeater said:

As @brwainer said proxy servers really come in to their own for networks with many users rather than 1 or just a few. I have setup Squid for home use before and configured it for some very aggressive caching and forcing some things to cache explicitly so it does give a decent speed up but only when you rewatch the same youtube video or go to the same website very often.

 

I used to play a browser based game that involved a lot of page loads so for that Squid gave a huge improvement, I could do a click every second rather than every 3-10 seconds.

 

I would say do this project anyway, it's worth it just for the experience and knowledge. Also have a look at using DansGuardian for web filtering/ad blocking, it's extremely advanced in comparison to what Squid can do by itself.

Very informative, also nice to see someone who supports doing projects for experience even though they may be pointless.

How would I be able to test the benefits of this if I were to set it up?

What size disk did you use and what size would you recommend?

Does DansGurdian run along side Squid on Linux for example? 

[Jackpot17]

1 hour ago, brwainer said:

This is something I've thought about trying before but never really started on. My reason is that without a lot of users hitting the same sites, I figure the amount of speedup would be minimal. Your browser already does its own caching of things like images on websites. Steam and other updates rarely have the same data in them, so unless you and your brother have lots of the same games I wouldn't expect much usable data to get cached. But if you try it and report back on any improvements seen, maybe I'll give it a go.

 

Blocking ads is something I think you can do with a Squid proxy, but I honestly don't know for sure.

Very interesting and thank you for the reply. My brother and I do share many Steam games. Arma updates, for example, are constant. I know they're not that large considering my download speed, but it would be nice for it to be more efficient. It also would just give a project to try out considering I have the hardware being unused at the moment.

[leadeater]

Just now, Jackpot17 said:

Very informative, also nice to see someone who supports doing projects for experience even though they may be pointless.

What size disk did you use and what size would you recommend?

Does DansGurdian run along side Squid on Linux for example? 

I don't remember the disk cache size I used but it doesn't need to be large, 10GB-100GB is plenty even for a large network. A lot of stuff doesn't cache very well or the web server sends a no-cache header to tell things like Squid not to cache that content, which you can override if you wish.

 

DansGuardian requires Squid to work, it's basically a pluigin/extension to Squid that only does filtering and scanning.

[Jackpot17]

36 minutes ago, leadeater said:

I don't remember the disk cache size I used but it doesn't need to be large, 10GB-100GB is plenty even for a large network. A lot of stuff doesn't cache very well or the web server sends a no-cache header to tell things like Squid not to cache that content, which you can override if you wish.

 

DansGuardian requires Squid to work, it's basically a pluigin/extension to Squid that only does filtering and scanning.

Oh okay, the Chromebox I would use itself has a 15GB m.2 SSD inside or I could just use the external drive which would be almost capable of saturating gigabit speeds anyway.

 

[Jackpot17]

8 hours ago, brwainer said:

This is something I've thought about trying before but never really started on. My reason is that without a lot of users hitting the same sites, I figure the amount of speedup would be minimal. Your browser already does its own caching of things like images on websites. Steam and other updates rarely have the same data in them, so unless you and your brother have lots of the same games I wouldn't expect much usable data to get cached. But if you try it and report back on any improvements seen, maybe I'll give it a go.

 

Blocking ads is something I think you can do with a Squid proxy, but I honestly don't know for sure.

7 hours ago, leadeater said:

As @brwainer said proxy servers really come in to their own for networks with many users rather than 1 or just a few. I have setup Squid for home use before and configured it for some very aggressive caching and forcing some things to cache explicitly so it does give a decent speed up but only when you rewatch the same youtube video or go to the same website very often.

 

I used to play a browser based game that involved a lot of page loads so for that Squid gave a huge improvement, I could do a click every second rather than every 3-10 seconds.

 

I would say do this project anyway, it's worth it just for the experience and knowledge. Also have a look at using DansGuardian for web filtering/ad blocking, it's extremely advanced in comparison to what Squid can do by itself.

I've set up a Squid proxy server using this tutorial: (Really helpful and straight forward if anyone would like to give it a try)

 

I've only noticed improvements during downloads not in web page loading times (guessing that's  because my internet wasn't exactly slow in the first place.

 

I have however just posted this:

If you would like to give it a read it's just about a few queries I have as I'd like it to be able to cache files larger than 100MB and to increase the maximum storage for caching.

 

Thanks for all the replies.

 

[gexacor]

Caching proxy server will affect overall throughput only if you have bunch of users accessing the same websites

If you only have your own home you'll not notice any real difference I believe.