Twitch is testing peer-to-peer streaming in Korea

[Not_A_Spider]

Summary

 

On July 29th, Twitch rolled out p2p for source quality streams in Korea due to "rising operating costs" in that country. Meaning that everyone in Korea watching a stream above 720p will be sharing their bandwidth and compute power with other people watching.

 

Quotes

Quote

What is Peer-to-peer (P2P) technology?

P2P is used to effectively deliver HD content by leveraging the distributed computing power of viewers. With the use of P2P, viewers can share video content from each other if they are watching the same stream.

Quote

Can I disable the use of P2P?

For viewers, the use of P2P means that they view a stream in the highest possible quality. However, using P2P for viewers is optional, and you can choose to continue watching Twitch at 720p or below.

Quote

How does peer-to-peer impact my privacy?

In order to create a peer-to-peer (P2P) connection, it is necessary for Twitch to make the IP address of each participant available to the other. It may be possible for a highly motivated and technically proficient person to discover participants’ IP addresses, which could potentially be used to approximate location. Viewers who have privacy concerns with P2P can watch streams in 720p to avoid any IP sharing risk.

My thoughts

It's honestly only a matter of time before this gets rolled out everywhere. We're getting closer and closer to a 1TB data cap not being enough for basic use, then ISPs can start charging extra for packages where services like streaming won't count against your limit.

 

Sources

https://www.sportskeeda.com/esports/news-twitch-testing-peer-to-peer-technology-korea-despite-potential-privacy-concerns

https://help.twitch.tv/s/article/p2p-faq?language=en_US

[jaslion]

Yeah so watching twitch in higher resolutions in gonna start costing people a chunk of money in terms of energy costs.

 

Also the ip thing they mention is quite concerning and gives off a lack of security features vibe

[pzspah]

Several countries are moving towards banning made up caps for internet use because they are just that, made up and serve no other purpose than greed.

[porina]

I wonder how this will work. I presume a streamer will still stream directly to Twitch, and Twitch then redistributes either the 720p as normal, or 1080p in p2p. So streamer still gets hidden IP as long as they don't run a high res client to monitor the stream.

 

Just trying to understand what the impacts are to Twitch to go this way. Built up areas of South Korea has had great bandwidth for much longer than most of the West. Is that putting more server load on Twitch than they're used to in other countries? Or is it bandwidth? For example, if someone were to stream 720p at 6000k is that different than 1080p at 6000k? Is there a new lower limit for 720p or they just recode it anyway?

 

Edit: Thinking more, as a viewer, would that impact latency? I estimate for most stuff I stream/watch on Twitch the typical latency between streamer doing something and it hitting client is ball park of 5 seconds. So that'll be some latency between streamer and Twitch, possible recoding if they qualify, and resending to viewers. Making it p2p implicitly adds at least one extra stop so delay could get worse.

[MedicTim78]

17 minutes ago, porina said:

I wonder how this will work. I presume a streamer will still stream directly to Twitch, and Twitch then redistributes either the 720p as normal, or 1080p in p2p. So streamer still gets hidden IP as long as they don't run a high res client to monitor the stream.

 

Just trying to understand what the impacts are to Twitch to go this way. Built up areas of South Korea has had great bandwidth for much longer than most of the West. Is that putting more server load on Twitch than they're used to in other countries? Or is it bandwidth? For example, if someone were to stream 720p at 6000k is that different than 1080p at 6000k? Is there a new lower limit for 720p or they just recode it anyway?

 

Edit: Thinking more, as a viewer, would that impact latency? I estimate for most stuff I stream/watch on Twitch the typical latency between streamer doing something and it hitting client is ball park of 5 seconds. So that'll be some latency between streamer and Twitch, possible recoding if they qualify, and resending to viewers. Making it p2p implicitly adds at least one extra stop so delay could get worse.

I don't think this will work well, imagine getting thousands or tens of thousand of viewers peer connecting to your stream, it will inevitably crash it. Also the security concerns are well beyond what I'd be willing to risk, personally.  

[porina]

2 minutes ago, MedicTim78 said:

I don't think this will work well, imagine getting thousands or tens of thousand of viewers peer connecting to your stream, it will inevitably crash it. Also the security concerns are well beyond what I'd be willing to risk, personally.  

My guess on how it works is streamer only streams directly to Twitch, so they retain their IP anonymity. Twitch then acts like the seed in bittorrent and the high res stream gets around that way. The risk is that other watchers can get your IP, unless you go through a VPN I suppose. The other part I guess is that the p2p network, on average, would have to have each node uploading as much as they download to make it scale. However to make latency not totally suck and also have redundancy as nodes appear and disappear, some nodes will have to reupload more than they consume. It will be interesting to see how it works in practice.

 

Wonder if it would be simpler if they just limited to 720p unless you're a subscriber?

[Ydfhlx]

So it's just like torrenting, but in real time. Wonder how they going to fix latency problem, and also problem of loosing connection to peers.

[TetraSky]

Everything to cut cost. Can't wait for some vulnerability to be found when using P2P video streaming, allowing attackers to get access to your computer just by watching the same video stream as you.

 

As for latency... Honestly they could just add a 30s delay and nobody would really care except streamers who want real time chat for some reason. Would also be better to reduce sniping and other things viewers are known to do to streamers.

[Kisai]

4 hours ago, porina said:

I wonder how this will work. I presume a streamer will still stream directly to Twitch, and Twitch then redistributes either the 720p as normal, or 1080p in p2p. So streamer still gets hidden IP as long as they don't run a high res client to monitor the stream.

 

Just trying to understand what the impacts are to Twitch to go this way. Built up areas of South Korea has had great bandwidth for much longer than most of the West. Is that putting more server load on Twitch than they're used to in other countries? Or is it bandwidth? For example, if someone were to stream 720p at 6000k is that different than 1080p at 6000k? Is there a new lower limit for 720p or they just recode it anyway?

 

Edit: Thinking more, as a viewer, would that impact latency? I estimate for most stuff I stream/watch on Twitch the typical latency between streamer doing something and it hitting client is ball park of 5 seconds. So that'll be some latency between streamer and Twitch, possible recoding if they qualify, and resending to viewers. Making it p2p implicitly adds at least one extra stop so delay could get worse.

 

This is just an armchair thought, because I've been thinking about this a lot lately.

 

Twitch, or even Youtube, can not "serve everyone equally", the trans-Atlantic and trans-Pacific pipes will simply never allow that to happen.

 

My thought process here, DRM-excepted, is that Ingress remains the same as now, but instead of having like 10000 korean viewers (who are avid esports viewers) trying to pull 50-odd streams over the US link, Twitch's Korean data center operates as a P2P head node, so only one stream is ever pulled from the trans-pacific links, and other people in Korea can get "source quality" video by connecting to the head node and it having auto-determine how to unfold the packet sorting.

 

eg, If a Korean streamer is streaming to twitch, instead of it going all the way to the US and back, the ingress is done on the korean side, but the stream sent to the US is handled as per normal, but everyone in Korea connects to the head node inside Korea and thus the video packets aren't transcoded again.

 

Then again, it may simply be Amazon/Twitch is unable to get peering arrangements(eg zero cost access) with Korean ISP's, and thus making those ISP's pay for it by having their users generate a lot of P2P traffic instead. Which is fine, ultimately the same amount of data is consumed by the ISP, it's just not consumed by Twitch.

 

To be real though, P2P would reduce latency to zero in Korea, their networks don't suck like North American ones do for latency.

[Arika]

This sounds like a bad idea...

[Donut417]

8 hours ago, Not_A_Spider said:

We're getting closer and closer to a 1TB data cap not being enough for basic use,

We are over a TB this month. We get 1.2 TB from Comcast. Every month it seems we are running close. I started to spreadsheet our usage. My dad doesnt understand was a cap is and he walks away when he has streaming content on or falls asleep streaming and we therefore are chewing thru data like its candy.

[williamcll]

P2P works over an internet browser?

[HomeBoi]

 

14 hours ago, Not_A_Spider said:

We're getting closer and closer to a 1TB data cap not being enough for basic use, then ISPs can start charging extra for packages where services like streaming won't count against your limit.

What, data caps on normal ISP internet, what backwater country do you live in? We are starting to close in on removing data caps on mobile phone connectivity here even so this seems just incredibly greedy and anti-consumer.

[Rauten]

Don't use Twitch much, but I guess I'll never watch a 1080p or above stream.

 

This is a disaster waiting to happen.

[WereCat]

15 hours ago, porina said:

Just trying to understand what the impacts are to Twitch to go this way. Built up areas of South Korea has had great bandwidth for much longer than most of the West. Is that putting more server load on Twitch than they're used to in other countries? Or is it bandwidth? For example, if someone were to stream 720p at 6000k is that different than 1080p at 6000k? Is there a new lower limit for 720p or they just recode it anyway?

This is what makes no sense to me.

6000 bitrate is 6000 bitrate regardless of resolution.

Arguably a 6000 bitrate is not enough for 1080p60FPS with x264 so people tend to stream at 8000 bitrate (which works fine even if its not officialy supported by Twitch).

 

So why would this be tied to resolution if I can stream in 720p 8000bitrate and somebody else can stream in 1080p 6000 bitrate? I would still achieve better overal quality at 720p depending on if I stream a fast moving content while a more static content would look better at 1080p even at a bit lower bitrate.

 

Yes, latency will most likely sux a lot over P2P for viewers that live far enough from the streamer. Security concerns are valid as well.

I mean, as a choice this would be fine. Make it OPT IN not OPT OUT for streamers and viewers so that if the streamer decides that he wants to stream at 20000 bitrate then P2P makes sense as there is no reason for imposed Twitch limits.

[porina]

3 minutes ago, WereCat said:

Arguably a 6000 bitrate is not enough for 1080p60FPS with x264 so people tend to stream at 8000 bitrate (which works fine even if its not officialy supported by Twitch).

I didn't know that was a thing. I had in the past set 6000k for video, but with audio on top it is slightly above that overall. Had complaints my stream was too strong for people in areas with weaker internet, and I wasn't big enough to get reprocessing from Twitch (still far from partner). As a balance of quality and bitrate I think I do 1080p30 around 4500k. Most of the games I play are not fast action so lower fps is not important.

 

3 minutes ago, WereCat said:

Yes, latency will most likely sux a lot over P2P for viewers that live far enough from the streamer.

Not sure that will matter. I'd guess the streamer doesn't directly seed the p2p part, and will still upload to Twitch in the first instance. @Kisaiearlier post raises some interesting points about how Twitch's infrastructure might work worldwide, and I don't know enough about that to guess further. Where do they have servers to take in data? Do they have something like a CDN to better optimise regional streaming output?

[WereCat]

32 minutes ago, porina said:

I didn't know that was a thing. I had in the past set 6000k for video, but with audio on top it is slightly above that overall. Had complaints my stream was too strong for people in areas with weaker internet, and I wasn't big enough to get reprocessing from Twitch (still far from partner). As a balance of quality and bitrate I think I do 1080p30 around 4500k. Most of the games I play are not fast action so lower fps is not important.

Yes thats an issue from the viewers side if they don't have fast enough internet but from streamer and twitch side its fine. I've been streaming at 8000 bitrate constantly.

 

33 minutes ago, porina said:

Not sure that will matter. I'd guess the streamer doesn't directly seed the p2p part, and will still upload to Twitch in the first instance. @Kisaiearlier post raises some interesting points about how Twitch's infrastructure might work worldwide, and I don't know enough about that to guess further. Where do they have servers to take in data? Do they have something like a CDN to better optimise regional streaming output?

I don't understand how this makes sense. If it goes to Twitch and then Twitch has to redistribute it then whats the point of P2P? Isn't it the same as right now?

[porina]

25 minutes ago, WereCat said:

I don't understand how this makes sense. If it goes to Twitch and then Twitch has to redistribute it then whats the point of P2P? Isn't it the same as right now?

Now: Streamer > Twitch > full direct connection to all viewers

Possible: Streamer > Twitch > indirectly to multiple viewers via p2p. They wont send it directly and fully to everyone, but seed the p2p cloud. Outgoing bandwidth will be reduced. Twitch still needs to send the 720p stream to those who choose it anyway.

[Sauron]

17 hours ago, Not_A_Spider said:

It may be possible for a highly motivated and technically proficient person to discover participants’ IP addresses

Meaning someone who downloaded wireshark? This would be absolutely trivial. It's not a huge deal to have people know your public IP address since who it's associated with is usually not publically available information, but it can reveal some information about you.

15 hours ago, TetraSky said:

Everything to cut cost. Can't wait for some vulnerability to be found when using P2P video streaming, allowing attackers to get access to your computer just by watching the same video stream as you.

P2P isn't particularly insecure, at least not inherently. I do wonder if you could inject your own content into the stream though.

31 minutes ago, WereCat said:

I don't understand how this makes sense. If it goes to Twitch and then Twitch has to redistribute it then whats the point of P2P? Isn't it the same as right now?

Well no, because twitch would then have to feed that content to fewer users than they do right now. Only one user is uploading to twitch, the rest are all downloading and the fewer are taking the content directly from twitch the lower the bandwidth usage will be on their end.

[Donut417]

4 hours ago, HomeBoi said:

What, data caps on normal ISP internet, what backwater country do you live in?

The United States of America. Comcast, Mediacom, Cox, WOW are some of the biggest ISP's with Data caps. 

[Kisai]

1 hour ago, porina said:

Now: Streamer > Twitch > full direct connection to all viewers

Possible: Streamer > Twitch > indirectly to multiple viewers via p2p. They wont send it directly and fully to everyone, but seed the p2p cloud. Outgoing bandwidth will be reduced. Twitch still needs to send the 720p stream to those who choose it anyway.

Also note that that full direct connection gives them the ability to push ads "in-stream" (literately interrupting the video.) In a P2P environment, the server-end would no longer have control over this, and it could be possible for people to modify the stream to drop/skip the ads, because they won't come over the same P2P head link. Which as far as Twitch is concerned, they're no longer paying for the bandwidth, so the ads would be immaterial.

 

I imagine that ultimately Amazon may be trying to do this due to the mobile bandwidth caps and will bring it to Canada and the US, because both countries subscribe to this mentality of "dialup speed once data cap hit"

 

 

[Doobeedoo]

Yeah this is an interesting move..

Also that privacy part.

 

The data caps. Just what that should not exist on a line. 

Where I am no caps on line. Those who can't get faster speeds can get 5G 200Mbps 2TB 'cap' but still if over that can use 20Mbps down. 

[Spotty]

Quote

How does peer-to-peer impact my privacy?

In order to create a peer-to-peer (P2P) connection, it is necessary for Twitch to make the IP address of each participant available to the other. It may be possible for a highly motivated and technically proficient person to discover participants’ IP addresses, which could potentially be used to approximate location. Viewers who have privacy concerns with P2P can watch streams in 720p to avoid any IP sharing risk.

I wonder if this could also potentially reveal the streamers IP address. If the streamer is included as one of the "seeders" in this situation it would be possible to discover their IP address. Even if the streamer isn't included some streamers will have their own stream open while streaming so they can read & response to chat and monitor the stream, which would include them as a viewer and include them in the P2P list. Considering people already "Swat" streamers live on stream I imagine it won't be long before every time a streamer goes live their internet connection is DoS'd by trolls, bringing down the stream. That's already a problem in some multiplayer games that expose IP addresses, which most streamers typically avoid for this exact reason. Even when a game servers IP address is exposed it can lead to DoS attacks bringing the game server down just to troll the streamers.

This could even allow for a new attack against streamers where people attack their viewers as a way to harass the streamer.

 

I wonder how well this P2P system would work if there are only a small number of viewers. Sure some of the most popular streamers get thousands of viewers at any given time, but there's a significant amount of streams which have less than 10 viewers.

 

Just seems like a bad idea overall just for Twitch to cut costs.

 

18 hours ago, porina said:

Wonder if it would be simpler if they just limited to 720p unless you're a subscriber?

That makes more sense to me and would be easier to implement. There's not that many benefits to subscribing to a channel other than emotes, sub chat mode (which most streamers use sparingly), and removing ads from the stream. Making 1080p or higher a subscriber only option would give more reason to subscribe to a channel, not only offsetting the higher cost of delivering that HD content but also attracting more revenue with more subscriptions sold.

[Kisai]

2 hours ago, Spotty said:

 

I wonder how well this P2P system would work if there are only a small number of viewers. Sure some of the most popular streamers get thousands of viewers at any given time, but there's a significant amount of streams which have less than 10 viewers.

Probably for a small streamer (eg under 100) it would be easy to figure out who the streamer is because they'd be the one with same latency as the chat. I'm assuming that the CHAT is not going through the P2P, because that would bring all the problems of IRC (including netsplits) to it. Which means that someone using pcap/wireshark would be able to just look at all the P2P connections, and even if they don't know who the streamer is, they could DDoS all the peers to make the experience miserable for everyone.

 

 

2 hours ago, Spotty said:

Just seems like a bad idea overall just for Twitch to cut costs.

 

That makes more sense to me and would be easier to implement. There's not that many benefits to subscribing to a channel other than emotes, sub chat mode (which most streamers use sparingly), and removing ads from the stream. Making 1080p or higher a subscriber only option would give more reason to subscribe to a channel, not only offsetting the higher cost of delivering that HD content but also attracting more revenue with more subscriptions sold.

 

I think this is really an end-run around datacaps, or the desire for ISP's to charge amazon more( https://www.reuters.com/business/media-telecom/skorea-broadband-firm-sues-netflix-after-traffic-surge-squid-game-2021-10-01/ ) and they want to see how well it works in Korea because nobody over there would notice the latency. Where as over in North America, the latency would explode if they did it and Peers end up peering with each other on opposing sides of the country. I have a feeling that this is probably going to be a hybrid approach where the head node of the P2P is going to be operated by Amazon, which means whatever data center zone you'd normally hit for Twitch would be where your P2P peers would be.

 

This is where greed meets greed. Like from a reasonable perspective, the ISP's should not be charging anyone for access to anything. Yet they are, they're going after the video sites because the way they're doing things, is excessively bandwidth-wasting.

 

There is no way to have "edge nodes" for live content. It simply doesn't work that way, the best you can do is create edge nodes that add latency for the most bandwidth-intensive live content, and hope people don't undermine it by using google or cloudflare's DNS.

 

So you have (Twitch streamer) ->(Twitch ingest server) -> Twitch Amazon zones -> Twitch Korea (head node) <- Korean peers

 

Like it is a privacy problem in more than one way, one being that rights holders might start going after the individual peers for DMCA claims.

 

[Donut417]

8 hours ago, Kisai said:

I think this is really an end-run around datacaps, or the desire for ISP's to charge amazon more( https://www.reuters.com/business/media-telecom/skorea-broadband-firm-sues-netflix-after-traffic-surge-squid-game-2021-10-01/ )

Comcast already successfully got Netflix to pay the extortion charge. So at least in the US there is precedent. Not sure if Comcast has agreement with any other services. But Comcast also charges data overages on wired broadband service, so they are double dipping.