Steam Caching Tutorial

Dred88 · August 26, 2018

5 minutes ago, Androidz said:

@Dred88 - You would point to steamcache-dns IP.

I understand that. I'm asking if I then wanted the windowsudpate cache, would I then change the IP???

Or is that one IP as my DNS server for "Steamchache" going to work for pulling/caching from all the services that were listed below or do I change the DNS IP depending on which i want to pull from?????

-e STEAMCACHE_IP=192.168.1.50 \
  -e ORIGINCACHE_IP=192.168.1.51 \
  -e BLIZZARDCACHE_IP=192.168.1.52 \
  -e RIOTCACHE_IP=192.168.1.53 \
  -e FRONTIERCACHE_IP=192.168.1.54 \
  -e WINDOWSCACHE_IP=192.168.1.55 \
  steamcache/steamcache-dns:latest

Androidz · August 26, 2018

@Sam Katoen - GOD BLESS YOU! This is working!!!

For those of you wanting to cache Windows updates see Sam's reply - https://linustechtips.com/main/topic/962655-steam-caching-tutorial/?page=2&tab=comments#comment-11672873

TL;DR -e WSUSCACHE_IP=192.168.1.55 \

Androidz · August 26, 2018

@Dred88 - No, it's all inclusive you only use steamcache-dns IP and it handles everything (Steam, Windows Updates, Origin, etc)

Dred88 · August 26, 2018

Thanks, ignore my last response if you saw it asking the question again, didn't see your response. Thank you for letting me know that info. T that makes it much more user friendly. I was worried I'd have to change the IP to bounce between cache's

Androidz · August 26, 2018

@Dred88 - That would be awful!

Does anyone know if it possible to forward DNS from pfsense -> steamcache-dns -> out (or back to pfsense)?

Dred88 · August 27, 2018

1 hour ago, Androidz said:

@Dred88 - That would be awful!

Does anyone know if it possible to forward DNS from pfsense -> steamcache-dns -> out (or back to pfsense)?

I've got limited knowledge but have a pfsense router setup. Wouldn't you just go to services/DHCP Server/LAN and put the 192.168.1.xx in the DNS field for your cache server?

I'm sure you've tried that already. I haven't thought my response all the way through. Too many delicious homebrew beers. haha

Oniyakun · August 27, 2018

Will, its not run on ARM or raspberry pi

Sad ;_;

Androidz · August 27, 2018

So I have WSUS caching setup and it works for about 4 hours and then randomly stops working. Dockers are up and running, but I can't see anything obvious in the logs. When I created the docker I set my upstream DNS to my pfsense router, so perhaps that has something to do with it.

Anyone have any ideas?

TheeeStig · August 28, 2018

On 8/21/2018 at 9:30 PM, Unieveth said:

I have it working to a point where the DNS is not crashing, but now the issue is that I see no traffic going through the server when I'm downloading a game from steam. It still seems to be bypassing the server. I have changed the DNS settings on my local PC to test it and it's not going through the server. Here is an image of the settings along with putty with nload running: https://i.imgur.com/mcqwyOA.png I have also flushed the DNS on my system so unless I'm missing something it should be working. I have also checked the stream cache and it has 1 item in it: https://i.imgur.com/zIhwXQq.png

Fixed it

How did you manage to fix this? I have the same problem of no traffic going through the server

jackk556 · August 29, 2018

so I followed step by step and I seem to not be able to get it working.... i'm running the same version of ubuntu a unraid VM im not to sure if it is running correctly or not

I'm a quick learner when it comes to new things but ive caught a bump in the road and I don't know what to do...… can someone help me find a way to check and see if it is working ?

ive seen videos of running this script that shows docker taking the files and sending file vie red and green text, maybe someone could help me out with that …. thanks

maybe this was a problem ?

sudo docker run \
>   --restart unless-stopped -d \
>   --name steam-cache \
>   -v /home/jack/cache/steam/data:/data/cache \
>   -v /home/jack/cache/steam/logs:/data/logs \
>   -p 192.168.0.5:80:80 \
>   steamcache/generic:latest
Unable to find image 'steamcache/generic:latest' locally
latest: Pulling from steamcache/generic
911c6d0c7995: Pull complete
131e13eca73f: Pull complete
95376bf29516: Pull complete
6717402ec973: Pull complete
becc9b76442e: Pull complete
d6e5215e88b8: Pull complete
Digest: sha256:a06abb08cb1f2e0b58bce36f0ebc0d57b319855ea905e76da500f805b0e80d06
Status: Downloaded newer image for steamcache/generic:latest
01fe988426307f404041733e3767f534d0660eb3116f0df870ec209119d3c4d6
docker: Error response from daemon: driver failed programming external connectivity on endpoint steam-cache (a44a70992d603355866a77174386a32feb8143330dd0dc98215162a1c61177f4): Error starting userland proxy: listen tcp 192.168.0.5:80: bind: cannot assign requested address.

but it looked like all docker images were "running" idk maybe im wrong ...

sudo docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4a83acb0714e steamcache/sniproxy:latest "sniproxy -f" 24 seconds ago Up 21 seconds 0.0.0.0:443->443/tcp sniproxy

any and all help would be greatly appreciated

thanks - Jack

N0tnathan · August 29, 2018

Is anyone else experiencing slow downloads from Blizzard? When I point my workstation to a public DNS like 1.1.1.1 or 8.8.8.8, I'll get 16MB/s download. When I point DNS to my steamcache server, it runs at maybe 1~2MB/s. Once the data has been written to the cache, re-running the install works very fast (around ~100MB/s). Is there anything I can do to help move this along faster or do I need this to just run its course? Also League of Legends looks to be downloading at slower speeds too, but does not seem to be pulling from the cache when I try to re-install. Let me know if you have any questions.

Shadowman94 · August 29, 2018

2 hours ago, N0tnathan said:

Is anyone else experiencing slow downloads from Blizzard? When I point my workstation to a public DNS like 1.1.1.1 or 8.8.8.8, I'll get 16MB/s download. When I point DNS to my steamcache server, it runs at maybe 1~2MB/s. Once the data has been written to the cache, re-running the install works very fast (around ~100MB/s). Is there anything I can do to help move this along faster or do I need this to just run its course? Also League of Legends looks to be downloading at slower speeds too, but does not seem to be pulling from the cache when I try to re-install. Let me know if you have any questions.

Same here, but for me, blizzard app cannot complete download.. when the reach last 1 or 2mb, its stuck and spamming cache server with requests and its reach 800-900mbps upload... until the stop battlenet app so its cant complete the download progress...

N0tnathan · August 29, 2018

1 hour ago, Shadowman94 said:

Same here, but for me, blizzard app cannot complete download.. when the reach last 1 or 2mb, its stuck and spamming cache server with requests and its reach 800-900mbps upload... until the stop battlenet app so its cant complete the download progress...

That's strange, cause mine is able to complete the downloads when I'm pulling from the cache. I would just like this download to be moving faster cause I have the ISP capabilities and it should not take over 12 hrs to download World of Warcraft, which right now, that download is still going with 11gb left to download. Anyone have any ideas?

haltatarry · August 30, 2018

Got this all working according to the instructions on 16.04 (its all I had...). Worth noting you will probably want to setup SSH access as you really don't need access to the GUI anyways.

Shadowman94 · August 30, 2018

17 hours ago, N0tnathan said:

That's strange, cause mine is able to complete the downloads when I'm pulling from the cache. I would just like this download to be moving faster cause I have the ISP capabilities and it should not take over 12 hrs to download World of Warcraft, which right now, that download is still going with 11gb left to download. Anyone have any ideas?

Try to change slice 1m; to slice 10m; in config in blizzard container.

nicedevil · August 31, 2018

Hey there,

first, THANK YOU for this awesome guide

I have a little problem now... I run an ubuntu 18.04 server in my home for cloud services for my family, plex aaaaaaaaaaaaaaaand PiHole (maybe a good thing for a new video @Linus)

For everybody who don't know what PiHole is: It is DNS Service that could be installed for blocking Ads.

So Port 53 is already in use for my Homeserver and after adding the additional IP addresses for the steamcaches, pihole starts listing on port 53 on those addresses as well.

Looks like this:

docker@NAS:sudo netstat -tulpn | grep 53
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.2:53             0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.50:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.51:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.52:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.53:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.54:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.55:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.56:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp6       0      0 ::1:53                  :::*                    LISTEN      2337/pihole-FTL
tcp6       0      0 fe80::a60:6eff:fe74::53 :::*                    LISTEN      2337/pihole-FTL
udp        0      0 127.0.0.1:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.2:53             0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.50:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.51:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.52:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.53:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.54:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.55:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.56:53            0.0.0.0:*                           2337/pihole-FTL

Can somebody help me out with this error?

Error starting userland proxy: listen udp 10.0.4.50:53: bind: address already in use.

Maybe someone got PiHole installed and steamcache as well on the same machine.

My Pihole is set as default DNS on my router. So the only DNS Entry for that is 10.0.4.2

Is that even possibile to get it run on one machine? If not, I'm going to buy a rapsberry Pi 3 for my pihole

Mhhh now I got an idea:

While running this command:

sudo docker run \
  --restart unless-stopped -d \
  --name steamcache-dns \
  -p 10.0.4.50:53:53/udp \
  -e UPSTREAM_DNS=10.0.4.X \
  -e STEAMCACHE_IP=10.0.4.50 \
  -e ORIGINCACHE_IP=10.0.4.51 \
  -e BLIZZARDCACHE_IP=10.0.4.52 \
  -e RIOTCACHE_IP=10.0.4.53 \
  -e FRONTIERCACHE_IP=10.0.4.54 \
  -e UPLAYCACHE_IP=10.0.4.55 \
  -e WINDOWSCACHE_IP=10.0.4.56 \
  steamcache/steamcache-dns:latest

And thinking about having my PiHole on an other machine (i.e. raspberrypi3) so should it run while adding this ip address in line 4?

Edited August 31, 2018 by nicedevil
An Idea to fix :D

Shadowman94 · August 31, 2018

@nicedevil u need to set in pi-hole/FTL config to bind service only to primary ip address, i think it's 10.0.4.2. After u can bind steamcache-dns to all other ip.

bashbro · August 31, 2018

Dear lord,

grace me with the presence of help. I have been at work at this 4 freaking days...no work...no going out nothing. Please can someone proof my work on this ubuntu desktop 18 version. I have teamviewer. please. I am about to break out in tears lol. Everything is working steampowered querys to the machine...but it doesnt download the games to the machine.

I am looking at my post every hour lol..please someone!

VastNorrland · September 1, 2018

Hi,

I am trying to do this all on a Synology NAS, it got a Docker app on it and in that app i could find both containers "steamCache-Generic" & "SteamCache-dns" I start both of them in docker and they seam to be working fine.

Routing my DNS to the NAS IP but it don't seam to do anything at all when i download games from steam. Maybe it is more to it than just loading up the two containers but i am in no way savvy enough to figure out what.

Would be very nice to just have it all running on the NAS as i do not have another PC ro run linux on and so on.

Shadowman94 · September 1, 2018

2 hours ago, VastNorrland said:

Hi,

I am trying to do this all on a Synology NAS, it got a Docker app on it and in that app i could find both containers "steamCache-Generic" & "SteamCache-dns" I start both of them in docker and they seam to be working fine.

Routing my DNS to the NAS IP but it don't seam to do anything at all when i download games from steam. Maybe it is more to it than just loading up the two containers but i am in no way savvy enough to figure out what.

Would be very nice to just have it all running on the NAS as i do not have another PC ro run linux on and so on.

Yeah i got same problem with steamcache... but the main problem is... when i want download a game (e.g. cs:go, dota 2, AOE2:HD ), the steam is start downloading with https protocol.. (what is not able to cache, only http), but with Battlenet app, windows update, uplay is works fine.

bashbro · September 3, 2018

Does anyone know how to get ArenaNet(guild wars 2) to work?

bashbro · September 3, 2018

On 9/1/2018 at 8:15 AM, Shadowman94 said:

Yeah i got same problem with steamcache... but the main problem is... when i want download a game (e.g. cs:go, dota 2, AOE2:HD ), the steam is start downloading with https protocol.. (what is not able to cache, only http), but with Battlenet app, windows update, uplay is works fine.

Try enabling that

Quote


$ sudo apt-get install \     apt-transport-https \     ca-certificates \     curl \     software-properties-common

Edited September 3, 2018 by bashbro
it didn't type it right.

NickBlaze2 · September 3, 2018

I keep getting stuff like this on the dns

but it doesn't cache what have I done wrong ?

these are the commands I runned as they follow

docker run --restart unless-stopped -d --name cache-steam -v /cache/steam/data:/data/cache -v /cache/steam/logs:/data/logs -p 192.168.1.250:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-battlenet -v /cache/battlenet/data:/data/cache -v /cache/battlenet/logs:/data/logs -p 192.168.1.251:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-riot -v /cache/riot/data:/data/cache -v /cache/riot/logs:/data/logs -p 192.168.1.252:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-windows -v /cache/windows/data:/data/cache -v /cache/windows/logs:/data/logs -p 192.168.1.253:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --name steamcache-dns -p 192.168.1.250:53:53/udp -e UPSTREAM_DNS=1.1.1.1 -e STEAMCACHE_IP=192.168.1.250 -e BLIZZARDCACHE_IP=192.168.1.251 -e RIOTCACHE_IP=192.168.1.252 -e WSUSCACHE_IP=192.168.1.253 steamcache/steamcache-dns:latest

docker run --restart unless-stopped -d --name sniproxy -p 443:443 steamcache/sniproxy:latest

docker exec -it cache-steam tail -f /data/logs/access.log

Extra info:

I run docker on windows server 2016 these are the server IP settings

These are the settings for the others PC's on the gaming network

These are the the docker running "tasks" the result of the command "docker ps" and "docker ps -a"

tmvkrpxl0 · September 3, 2018

is it able to run steamcache linux on virtual machine, and use window and connect to dns and download some games?

bashbro · September 3, 2018

This does not look like the way Linus did it on the video. Preferred DNS Server was the server itself 192 blah blah blah.

Sign In

Steam Caching Tutorial

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites

Link to post

Share on other sites

Link to comment

Share on other sites