Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
jakkuh_t

Steam Caching Tutorial

Recommended Posts

5 minutes ago, Androidz said:

@Dred88 - You would point to steamcache-dns IP.

I understand that.  I'm asking if I then wanted the windowsudpate cache, would I then change the IP???

 

Or is that one IP as my DNS server for "Steamchache" going to work for pulling/caching from all the services that were listed below or do I change the DNS IP depending on which i want to pull from?????

 

-e STEAMCACHE_IP=192.168.1.50 \
  -e ORIGINCACHE_IP=192.168.1.51 \
  -e BLIZZARDCACHE_IP=192.168.1.52 \
  -e RIOTCACHE_IP=192.168.1.53 \
  -e FRONTIERCACHE_IP=192.168.1.54 \
  -e WINDOWSCACHE_IP=192.168.1.55 \
  steamcache/steamcache-dns:latest
Link to post
Share on other sites

@Sam Katoen - GOD BLESS YOU! This is working!!!

For those of you wanting to cache Windows updates see Sam's reply - https://linustechtips.com/main/topic/962655-steam-caching-tutorial/?page=2&tab=comments#comment-11672873

TL;DR -e WSUSCACHE_IP=192.168.1.55 \

Link to post
Share on other sites

Thanks, ignore my last response if you saw it asking the question again, didn't see your response.  Thank you for letting me know that info. T that makes it much more user friendly.   I was worried I'd have to change the IP to bounce between cache's

Link to post
Share on other sites
1 hour ago, Androidz said:

@Dred88 - That would be awful!

Does anyone know if it possible to forward DNS from pfsense -> steamcache-dns -> out (or back to pfsense)?

I've got limited knowledge but have a pfsense router setup.  Wouldn't you just go to services/DHCP Server/LAN and put the 192.168.1.xx in  the DNS field for your cache server?

 

I'm sure you've tried that already.  I haven't thought my response all the way through.  Too many delicious homebrew beers. haha

Link to post
Share on other sites

So I have WSUS caching setup and it works for about 4 hours and then randomly stops working. Dockers are up and running, but I can't see anything obvious in the logs. When I created the docker I set my upstream DNS to my pfsense router, so perhaps that has something to do with it.

Anyone have any ideas?

Link to post
Share on other sites
On 8/21/2018 at 9:30 PM, Unieveth said:

I have it working to a point where the DNS is not crashing, but now the issue is that I see no traffic going through the server when I'm downloading a game from steam.  It still seems to be bypassing the server.  I have changed the DNS settings on my local PC to test it and it's not going through the server.  Here is an image of the settings along with putty with nload running: https://i.imgur.com/mcqwyOA.png I have also flushed the DNS on my system so unless I'm missing something it should be working.  I have also checked the stream cache and it has 1 item in it: https://i.imgur.com/zIhwXQq.png

 

Fixed it

How did you manage to fix this? I have the same problem of no traffic going through the server

Link to post
Share on other sites

so I followed step  by step and I seem to not be able to get it working.... i'm running the same version of ubuntu a unraid VM im not to sure if it is running correctly or not

 

 

 

I'm a quick learner when it comes to new things but ive caught a bump in the road and I don't know what to do...… can someone help me find a way to check and see if it is working ?

 

 

ive seen videos of running this script that shows docker taking the files and sending file vie red and green text, maybe someone could help me out with that …. thanks 

 

maybe this was a problem ?

 

sudo docker run \
>   --restart unless-stopped -d \
>   --name steam-cache \
>   -v /home/jack/cache/steam/data:/data/cache \
>   -v /home/jack/cache/steam/logs:/data/logs \
>   -p 192.168.0.5:80:80 \
>   steamcache/generic:latest
Unable to find image 'steamcache/generic:latest' locally
latest: Pulling from steamcache/generic
911c6d0c7995: Pull complete
131e13eca73f: Pull complete
95376bf29516: Pull complete
6717402ec973: Pull complete
becc9b76442e: Pull complete
d6e5215e88b8: Pull complete
Digest: sha256:a06abb08cb1f2e0b58bce36f0ebc0d57b319855ea905e76da500f805b0e80d06
Status: Downloaded newer image for steamcache/generic:latest
01fe988426307f404041733e3767f534d0660eb3116f0df870ec209119d3c4d6
docker: Error response from daemon: driver failed programming external connectivity on endpoint steam-cache (a44a70992d603355866a77174386a32feb8143330dd0dc98215162a1c61177f4): Error starting userland proxy: listen tcp 192.168.0.5:80: bind: cannot assign requested address.

 

 

but it looked like all docker images were "running"  idk maybe im wrong ...

 

 

sudo docker container ls
CONTAINER ID        IMAGE                        COMMAND             CREATED             STATUS              PORTS                  NAMES
4a83acb0714e        steamcache/sniproxy:latest   "sniproxy -f"       24 seconds ago      Up 21 seconds       0.0.0.0:443->443/tcp   sniproxy

 

 

 

any and all help would be greatly appreciated

 

thanks - Jack

Link to post
Share on other sites

Is anyone else experiencing slow downloads from Blizzard? When I point my workstation to a public DNS like 1.1.1.1 or 8.8.8.8, I'll get 16MB/s download. When I point DNS to my steamcache server, it runs at maybe 1~2MB/s. Once the data has been written to the cache, re-running the install works very fast (around ~100MB/s). Is there anything I can do to help move this along faster or do I need this to just run its course? Also League of Legends looks to be downloading at slower speeds too, but does not seem to be pulling from the cache when I try to re-install. Let me know if you have any questions. 

Link to post
Share on other sites
2 hours ago, N0tnathan said:

Is anyone else experiencing slow downloads from Blizzard? When I point my workstation to a public DNS like 1.1.1.1 or 8.8.8.8, I'll get 16MB/s download. When I point DNS to my steamcache server, it runs at maybe 1~2MB/s. Once the data has been written to the cache, re-running the install works very fast (around ~100MB/s). Is there anything I can do to help move this along faster or do I need this to just run its course? Also League of Legends looks to be downloading at slower speeds too, but does not seem to be pulling from the cache when I try to re-install. Let me know if you have any questions. 

Same here, but for me, blizzard app cannot complete download.. when the reach last 1 or 2mb, its stuck and spamming cache server with requests and its reach 800-900mbps upload... until the stop battlenet app :/ so its cant complete the download progress...

Screenshot_12.png

Link to post
Share on other sites
1 hour ago, Shadowman94 said:

Same here, but for me, blizzard app cannot complete download.. when the reach last 1 or 2mb, its stuck and spamming cache server with requests and its reach 800-900mbps upload... until the stop battlenet app :/ so its cant complete the download progress...

Screenshot_12.png

That's strange, cause mine is able to complete the downloads when I'm pulling from the cache. I would just like this download to be moving faster cause I have the ISP capabilities and it should not take over 12 hrs to download World of Warcraft, which right now, that download is still going with 11gb left to download. Anyone have any ideas?

Link to post
Share on other sites

Got this all working according to the instructions on 16.04 (its all I had...).  Worth noting you will probably want to setup SSH access as you really don't need access to the GUI anyways.

Link to post
Share on other sites
17 hours ago, N0tnathan said:

That's strange, cause mine is able to complete the downloads when I'm pulling from the cache. I would just like this download to be moving faster cause I have the ISP capabilities and it should not take over 12 hrs to download World of Warcraft, which right now, that download is still going with 11gb left to download. Anyone have any ideas?

Try to change slice 1m; to slice 10m; in config in blizzard container.

Link to post
Share on other sites

Hey there,

 

first, THANK YOU for this awesome guide :)

 

I have a little problem now... I run an ubuntu 18.04 server in my home for cloud services for my family, plex aaaaaaaaaaaaaaaand PiHole (maybe a good thing for a new video @Linus)

For everybody who don't know what PiHole is: It is DNS Service that could be installed for blocking Ads.

So Port 53 is already in use for my Homeserver and after adding the additional IP addresses for the steamcaches, pihole starts listing on port 53 on those addresses as well.

 

Looks like this:

docker@NAS:sudo netstat -tulpn | grep 53
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.2:53             0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.50:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.51:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.52:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.53:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.54:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.55:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp        0      0 10.0.4.56:53            0.0.0.0:*               LISTEN      2337/pihole-FTL
tcp6       0      0 ::1:53                  :::*                    LISTEN      2337/pihole-FTL
tcp6       0      0 fe80::a60:6eff:fe74::53 :::*                    LISTEN      2337/pihole-FTL
udp        0      0 127.0.0.1:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.2:53             0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.50:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.51:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.52:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.53:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.54:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.55:53            0.0.0.0:*                           2337/pihole-FTL
udp        0      0 10.0.4.56:53            0.0.0.0:*                           2337/pihole-FTL

Can somebody help me out with this error?

Error starting userland proxy: listen udp 10.0.4.50:53: bind: address already in use.

Maybe someone got PiHole installed and steamcache as well on the same machine.

My Pihole is set as default DNS on my router. So the only DNS Entry for that is 10.0.4.2

 

Is that even possibile to get it run on one machine? If not, I'm going to buy a rapsberry Pi 3 for my pihole :)

 

Mhhh now I got an idea:

 

While running this command:

sudo docker run \
  --restart unless-stopped -d \
  --name steamcache-dns \
  -p 10.0.4.50:53:53/udp \
  -e UPSTREAM_DNS=10.0.4.X \
  -e STEAMCACHE_IP=10.0.4.50 \
  -e ORIGINCACHE_IP=10.0.4.51 \
  -e BLIZZARDCACHE_IP=10.0.4.52 \
  -e RIOTCACHE_IP=10.0.4.53 \
  -e FRONTIERCACHE_IP=10.0.4.54 \
  -e UPLAYCACHE_IP=10.0.4.55 \
  -e WINDOWSCACHE_IP=10.0.4.56 \
  steamcache/steamcache-dns:latest

And thinking about having my PiHole on an other machine (i.e. raspberrypi3) so should it run while adding this ip address in line 4?

Edited by nicedevil
An Idea to fix :D
Link to post
Share on other sites

Dear lord,

 

grace me with the presence of help. I have been at work at this 4 freaking days...no work...no going out nothing. Please can someone proof my work on this ubuntu desktop 18 version. I have teamviewer. please. I am about to break out in tears lol. Everything is working steampowered querys to the machine...but it doesnt download the games to the machine.

 

I am looking at my post  every hour lol..please someone!

Link to post
Share on other sites

Hi,

 

I am trying to do this all on a Synology NAS, it got a Docker app on it and in that app i could find both containers "steamCache-Generic" & "SteamCache-dns" I start both of them in docker and they seam to be working fine. 
 

Routing my DNS to the NAS IP but it don't seam to do anything at all when i download games from steam. Maybe it is more to it than just loading up the two containers but i am in no way savvy enough to figure out what. 

Would be very nice to just have it all running on the NAS as i do not have another PC ro run linux on and so on. 

Link to post
Share on other sites
2 hours ago, VastNorrland said:

Hi,

 

I am trying to do this all on a Synology NAS, it got a Docker app on it and in that app i could find both containers "steamCache-Generic" & "SteamCache-dns" I start both of them in docker and they seam to be working fine. 
 

Routing my DNS to the NAS IP but it don't seam to do anything at all when i download games from steam. Maybe it is more to it than just loading up the two containers but i am in no way savvy enough to figure out what. 

Would be very nice to just have it all running on the NAS as i do not have another PC ro run linux on and so on. 

Yeah i got same problem with steamcache... but the main problem is... when i want download a game (e.g. cs:go, dota 2, AOE2:HD ), the steam is start downloading with https protocol.. (what is not able to cache, only http), but with Battlenet app, windows update, uplay is works fine.

Link to post
Share on other sites
On 9/1/2018 at 8:15 AM, Shadowman94 said:

Yeah i got same problem with steamcache... but the main problem is... when i want download a game (e.g. cs:go, dota 2, AOE2:HD ), the steam is start downloading with https protocol.. (what is not able to cache, only http), but with Battlenet app, windows update, uplay is works fine.

Try enabling that 
"
Quote

$ sudo apt-get install \     apt-transport-https \     ca-certificates \     curl \     software-properties-common

 

Edited by bashbro
it didn't type it right.
Link to post
Share on other sites

I keep getting image.thumb.png.1e3142f02ae57fafbfabd58c6d67abb4.pngstuff like this on the dns

image.thumb.png.9bde81070cae5c4d9cc7ee640b429e67.png

but it doesn't cache what have I done wrong ?image.thumb.png.6211e3ca7e57d9c99c9bb1970c28ef7c.png

 

these are the commands I runned as they follow

docker run --restart unless-stopped -d --name cache-steam -v /cache/steam/data:/data/cache -v /cache/steam/logs:/data/logs -p 192.168.1.250:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-battlenet -v /cache/battlenet/data:/data/cache -v /cache/battlenet/logs:/data/logs -p 192.168.1.251:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-riot -v /cache/riot/data:/data/cache -v /cache/riot/logs:/data/logs -p 192.168.1.252:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest

docker run --restart unless-stopped -d --name cache-windows -v /cache/windows/data:/data/cache -v /cache/windows/logs:/data/logs -p 192.168.1.253:80:80 -e CACHE_DISK_SIZE=614400m -e CACHE_MAX_AGE=100d steamcache/generic:latest
docker run --name steamcache-dns -p 192.168.1.250:53:53/udp -e UPSTREAM_DNS=1.1.1.1 -e STEAMCACHE_IP=192.168.1.250 -e BLIZZARDCACHE_IP=192.168.1.251 -e RIOTCACHE_IP=192.168.1.252 -e WSUSCACHE_IP=192.168.1.253 steamcache/steamcache-dns:latest

docker run --restart unless-stopped -d --name sniproxy -p 443:443 steamcache/sniproxy:latest

docker exec -it cache-steam tail -f /data/logs/access.log

Extra info:

I run docker on windows server 2016 these are the server IP settingsimage.png.91d39c2b64c94c21c3730796915356ef.png

These are the settings for the others PC's on the gaming networkimage.png.8050b905ab6fe1c3e12a5cd3d424161c.png

These are the the docker running "tasks" the result of the command "docker ps" and "docker ps -a"

image.thumb.png.439d8af1237efdef64f87efadf740506.png

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×