Internet Data Caps

[skywake]

ISP's themselves typically don't pay for data usage between networks but rather pay for bandwidth (Mbps/Gbps/Tbps) so why they actually charge for data usage seems completely flawed to me.

Except that Gbps directly translates to GB/mo, they're both data rates. Look at it this way, for arguments sake lets say that an ISP buys a 1Gbps link to share amongst 1000 users. That 1Gbps translates to 324TB/mo but it's also divided amongst 1000 users. Which means if each end user hammered their connection 24/7 they could use ~300GB/mo. But because there are some users who hammer it more? Some people would use more and others less. Some may hammer their connections 24/7 slowing down everyone else's link.

 

If that ISP set a quota at 300GB/mo? Nobody could use more than their "full share" of that 1Gbps external connection. Then the heavier users wouldn't be slowing down the speed of the network at the expense of others. Or at the very least they wouldn't be slowing them down quite as much. Maybe at the same time the ISP decides to offer a lower quota product at a lower cost. Purely because that old lady down the road who's only using the connection to google cake recipes and send emails doesn't cost them as much. Is that not fair? Does that not make sense?

 

Because external links in Australia are so constrained we've had quotas since forever. Some ISPs though have offered unlimited plans and often at reasonably attractive prices. But do you know what happens with those ISPs? The main reason that they're cheaper is because they buy less capacity than the other providers. And because ontop of that they have no quotas? Your speeds turn to shit in peak hours. That's why quotas exist. I'd much rather buy from an ISP who has a quota because odds are I'm far less likely to see slower speeds in peak times.

[leadeater]

Except that Gbps directly translates to GB/mo, they're both data rates. Look at it this way, for arguments sake lets say that an ISP buys a 1Gbps link to share amongst 1000 users. That 1Gbps translates to 324TB/mo but it's also divided amongst 1000 users. Which means if each end user hammered their connection 24/7 they could use ~300GB/mo. But because there are some users who hammer it more? Some people would use more and others less. Some may hammer their connections 24/7 slowing down everyone else's link.

 

If that ISP set a quota at 300GB/mo? Nobody could use more than their "full share" of that 1Gbps external connection. Then the heavier users wouldn't be slowing down the speed of the network at the expense of others. Or at the very least they wouldn't be slowing them down quite as much. Maybe at the same time the ISP decides to offer a lower quota product at a lower cost. Purely because that old lady down the road who's only using the connection to google cake recipes and send emails doesn't cost them as much. Is that not fair? Does that not make sense?

 

Because external links in Australia are so constrained we've had quotas since forever. Some ISPs though have offered unlimited plans and often at reasonably attractive prices. But do you know what happens with those ISPs? The main reason that they're cheaper is because they buy less capacity than the other providers. And because ontop of that they have no quotas? Your speeds turn to shit in peak hours. That's why quotas exist. I'd much rather buy from an ISP who has a quota because odds are I'm far less likely to see slower speeds in peak times.

 

It doesn't quite directly translate but they are related. The issue that the ISP is trying to avoid is over utilization or over subscription of the available network bandwidth. A data cap does not prevent this, if every user in Australia decided to start a full speed download all at the same time for a sustained period this would cause congestion, the data caps are put in place in the hopes this will not happen by the discouragement of using what you have,

 

Network bandwidth is not a finite resource and charging in a way that more aligns with that type of resource is inherently less effective. Networking QoS methods don't look at data usage and doing so wouldn't stop uncontrolled packet loss due to more than 100% link utilization trying to be used.

 

Also I live in New Zealand so I know all too well about data caps and congested links out of our country.

[leadeater]

Because external links in Australia are so constrained we've had quotas since forever. Some ISPs though have offered unlimited plans and often at reasonably attractive prices. But do you know what happens with those ISPs? The main reason that they're cheaper is because they buy less capacity than the other providers. And because ontop of that they have no quotas? Your speeds turn to shit in peak hours. That's why quotas exist. I'd much rather buy from an ISP who has a quota because odds are I'm far less likely to see slower speeds in peak times.

 

Typically unlimited plans in Australia and New Zealand come with fair use policies and directly state in the plan terms and conditions that this type of plan has low priority during peak times and if congestion is happening these plans suffer first.

 

This type of practice shows exactly why data caps don't work. If they did work there wouldn't be significant amounts of congestion causing these plans to slow down, since the majority of plans have data caps.

 

There is nothing stopping ISP's providing both fibre or vdsl connections but using connection rate limiting to charge for bandwidth. This method has been used for a long time on business internet connections as they have bandwidth guarantees so need to know exactly who has what.

[Haswell_rasperry_pi]

You know you have issues when in peak hours, you get about .5-2 megabit persecond. We pay comcast for the 150 megabit plan, but haven't gotten over 50 since July.

[GeekJump]

Lucky for me Comcast hasn't put caps in Metro Detroit yet. Im waiting however. Here is why I don't like caps:

 

A: How do I know if the meter is correct? Hell I cant even access the meter on Comcast's site any more. Also, it does not report in real time. 

B: Exempting their online Video service, that is anti competitive.

 

Plus you cant say its a capacity problem. Shit, they are testing 1 Gig service with Docsis 3.1. Even if it was a capacity issue, where the hell did the money go from the Universal Service Fund? Isn't that money giving to the ISP's to Upgrade and maintain their network? On top of charging most people 10 bucks a month for a Modem, which most pay because their to stupid to relies they can buy one. Then 10 bucks a month for every cable box. God help you if you have HD and a DVR. 

 

Yeah, tried to convince my family to buy a modem but they don't trust my exp....I even walked them over to the section in best buy and put the box in front of them.

 

Me: "See! Look, the exact same modem for $90 one-time. We had our last modem for YEARS! This will pay itself off in 9 months, and we'll be saving $10/month!" 

Parents: "Ehhh, well just trust the Comcast guys, they know what they're doing."

Me: -_______-

 

And when I tried to show them a better modem, router combo instead of that crap 2-in-1 unit comcast gives out, forget about it! XO

[GeekJump]

You know you have issues when in peak hours, you get about .5-2 megabit persecond. We pay comcast for the 150 megabit plan, but haven't gotten over 50 since July.

 

Yup had the same problem. Had a rep come over (was told multiple times that it would be a free visit) and the guy's excuse was the modem was old and swapped it out. While he was there we ran a speed test (on comcast's site lol) and we were reaching 120-150 speeds. When he left I ran my own tests and immediately was hovering around 75-100...still good speeds but not as advertised.

 

Then they had the balls to charges us $40 for the visit and $120 for 'rewiring and changing the ports' which he never did! Comcast refused to refund us anything and when a 2nd guy came the next day to review they had even bigger balls to try and charge us again (they refunded that $40)! We told him to get off our property before he came inside.

 

Tl;dr: Comcast on paper looks nice but since they have a monopoly they are doing whatever they want.

[skywake]

@leadeater

TBH I'm not sure what your point is here. Data-caps are a way of trying to avoid slowdown on consumer networks where are generally oversubscribed. And they're oversubscribed by design in an effort to make the prices more palatable. Of course you can get a business connection where they guarantee an amount of bandwidth but that comes at a premium. The sort of premium that an end user isn't willing to pay.

 

Effectively it's kinda like the old "cheap, quick or quality. Pick two" triangle

- You can have unlimited plans at a low cost but you'll get congestion

- You can have unlimited plans with low congestion but it won't be cheap

- You can have a cheap, low congestion plan but you'll have to live with a data cap

 

..... as an end user I'd rather have the third

[Noyu]

I'm on a 3 y/o DSL connection with an ISP here in the Philippines.

 

It's 3Mbps with no cap when we got it.

It's now 7GB a day. Though, when you reach 7GB, they only throttle you down to 1Mbps.

 

We can't upgrade to a better (higher speed) plan since the newer ones have caps.

We can't upgrade to a "premium" unlimited plan since they're not available in our area.

 

I'm stuck with a 3Mbps internet. #sad

 

FML

[leadeater]

@leadeater

TBH I'm not sure what your point is here. Data-caps are a way of trying to avoid slowdown on consumer networks where are generally oversubscribed. And they're oversubscribed by design in an effort to make the prices more palatable. Of course you can get a business connection where they guarantee an amount of bandwidth but that comes at a premium. The sort of premium that an end user isn't willing to pay.

 

Effectively it's kinda like the old "cheap, quick or quality. Pick two" triangle

- You can have unlimited plans at a low cost but you'll get congestion

- You can have unlimited plans with low congestion but it won't be cheap

- You can have a cheap, low congestion plan but you'll have to live with a data cap

 

..... as an end user I'd rather have the third

 

My point is rather than data cap have bandwidth cap, the actual thing they are trying to control. Bandwidth is the issue and using an indirect method to control it is less effective than a direct method. The data cap given to you is an amount not a rate, how quickly you use that amount is an effective rate. I could use the cap in a day, week or the month. Data caps do not stop peak network congestion and never will, there is no possible way it could.

 

Giving everyone with a fibre connection of 30Mbps or 100Mbps regardless of if they need that is silly. Why not have plans that offer 10Mbps, 20Mbps, 40Mbps, 60Mbps, 80Mbps and 100Mbps etc with no data caps. If the majority of users opt for a lower cost low latency fibre connection but with only 20 Mbps the potential peak load is much lower than everyone jumping on 100Mbps who honestly don't need that kind of bandwidth. 

[skywake]

@leadeater

Because that's not how data is consumed, just for a second think about how you consume data as an end-user. Sometimes you'll need to download a game or something and will use every last bit of your connection speed. You'll want that high speed connection. Other times you'll stream a HD video and you'll probably only want a decent amount of bandwidth at the start while it loads the buffer. If you give people a quota and shape them when they go over it? They'll adapt. And it will limit the amount of data people use during peak times. Especially if the reset dates don't align.

 

Also as someone with a generous data cap (1TB/mo) and a slow connection (8Mbps)? I'd gladly give up some of that data for a faster link if it was available. I'd take a 25Mbps link with a 500MB/mo quota in a heartbeat. I'd probably do the same if a a 250GB/mo quota at 50Mbps was on offer. And if they need to put a quota on it to make those high speed links more affordable? I'm all in. Sign me up. Because frankly I'd much rather that game download take <30mins rather than 2hours even if it means I'm "limited" to downloading "25 games a month"

[leadeater]

@leadeater

Because that's not how data is consumed, just for a second think about how you consume data as an end-user. Sometimes you'll need to download a game or something and will use every last bit of your connection speed. You'll want that high speed connection. Other times you'll stream a HD video and you'll probably only want a decent amount of bandwidth at the start while it loads the buffer. If you give people a quota and shape them when they go over it? They'll adapt. And it will limit the amount of data people use during peak times. Especially if the reset dates don't align.

 

Also as someone with a generous data cap (1TB/mo) and a slow connection (8Mbps)? I'd gladly give up some of that data for a faster link if it was available. I'd take a 25Mbps link with a 500MB/mo quota in a heartbeat. I'd probably do the same if a a 250GB/mo quota at 50Mbps was on offer. And if they need to put a quota on it to make those high speed links more affordable? I'm all in. Sign me up. Because frankly I'd much rather that game download take <30mins rather than 2hours even if it means I'm "limited" to downloading "25 games a month"

 

If you often download then pay more for a faster connection, this is already an option on home fibre plans. Typical is 30Mbps or 100Mbps. What I'm saying is have more than 2 possible speeds and not have data caps. Data caps are not how internet infrastructure works period. The whole idea of charging for it is a complete farce. You can either agree or not that's fine but at least acknowledge there is another way to do it which is pay for speed, which is how Australian ISP's are charged for access in to tier 1 internet networks in the first place.

 

A stable 20Mbps connection is by no means slow, if you do for what ever reason find it slow pay for 40 or 60. Simple concept, just as easy to implement and has a greater effect on possible network utilization.

 

I'm no novice at this sort of thing. As part of my job I have setup internet connections for schools and businesses which includes setting up firewalls with QoS rules and bandwidth shaping. What an ISP does is the same thing on a much larger scale. I know this as I have actually talked with network engineers in multiple different ISP's here in NZ. The current University I work at now has around 30,000 students, we don't data cap we bandwidth shape.

[skywake]

@leadeater

Except that data-caps are effectively just a fairer way to shape data. Maybe I do a lot of video streaming and use a lot of data per month but never at that a high rate. Do I cost the ISP less than the guy who wants a high speed connection for fast DL/UL but who doesn't really use anywhere near as much data? I doubt it. Remember that unlike at a University these people are paying for a service directly. If you want to use a lot of data you should be charged more regardless of what speed you subscribe to.

 

Also I think you have kinda ignored the way that data caps work. What happens is that if/when you hit that data-cap you are agressively shaped. With my plan my ISP shapes your connection down to 256k. If I was on a fibre plan (not available in my area) I'd pay the same for the same 1TB/mo quota and be able to get 25Mbps rather than the 8Mbps I had. If I was on that plan? They'd shape me to 8Mbps. I suspect that's not unlike what a University does. When I was at UNI we had a data allowance as well as a bunch of rules. If we went beyond that? We'd get shaped in the same way my ISP would.

 

Seriously, what's the issue here? Do you think it would be better if they didn't restrict the people who hammer the network? At the very least isn't it not fair to charge those people who do want to use more data more? I don't think this whole "just charge for the speed and throttle everyone during peak hours" is better than having data caps. At all.

[leadeater]

@leadeater

Except that data-caps are effectively just a fairer way to shape data. Maybe I do a lot of video streaming and use a lot of data per month but never at that a high rate. Do I cost the ISP less than the guy who wants a high speed connection for fast DL/UL but who doesn't really use anywhere near as much data? I doubt it. Remember that unlike at a University these people are paying for a service directly. If you want to use a lot of data you should be charged more regardless of what speed you subscribe to.

 

You really are missing my point. A video streamer as you pointed out doesn't need a high bandwidth connection to do this task, so yes you do cost the ISP less. You cost the ISP 8Mbps regardless of how much data you use, 1MB or 1TB it's the same. Your potential maximum load on the network is 8Mbps, they must account for this in their network infrastructure regardless if you use it or not simply by the fact that you might. 

 

A sudden peak user is a big problem, much more so than someone who consumes large amounts of data over the month slowly. Peaks cause problems, the potential peak load on a network with 1000 100Mbps users is much greater than 500 with 20Mbps, 200 40Mbps, 150 60Mbps, 100 80Mbps, 50 100Mbps. Do the math, the maximum potential peak load the ISP must account for is vastly different.

 

Controlling the speed of the connections given would more likely mean there will not be congestion and throttling will never happen. Throttling does happen now and will continue to always happen under the current model.

 

Again read this link and understand what it is saying and how costing is actually done, Australia is even talked about. https://blog.cloudflare.com/the-relative-cost-of-bandwidth-around-the-world/

 

Australia is the most expensive region in which we operate, but for an interesting reason. We peer with virtually every ISP in the region except one: Telstra. Telstra, which controls approximately 50% of the market, and was traditionally the monopoly telecom provider, charges some of the highest transit pricing in the world — 20x the benchmark 

 

If Australians wonder why Internet and many other services are more expensive in their country than anywhere else in the world they need only look to Telstra. What's interesting is that Telstra maintains their high pricing even if only delivering traffic inside the country. 

 

What I'm talking about is already starting to be done here in NZ, personally I think there should be something between the 30 to 100 and the 100 to 200. These are all unlimited plans

 

 

Ultra Fast Fibre broadband

 

Naked Fibre 30 Up to 30Mbps Up to 10Mbps $89.99

Naked Fibre 100 Up to 100Mbps Up to 20Mbps $99.99

Naked Fibre 200 Up to 200Mbps Up to 20Mbps $119.99

Naked Fibre 200 Plus Up to 200Mbps Up to 200Mbps $139.99

 

We don't place any artificial speed restrictions on our broadband plans. This means that you will receive the best speed we can provide at any point in time.

 

Yes there are capped options, often only 80GB or unlimited. Data caps are slowly being removed from all NZ plans.

[Donut417]

I see every one is done with caps. The fact is there is another half. How do we as consumers know how accurate those data cap meters are? Comcast has claimed they had their certified by a 3rd party. However, it was not certified by the Government. How much you want to bet, they are screwing you out of bandwidth? Remember back in the data Verizon got caught charging every one 1.99 for data use. These meters need to be certified yearly and have to be real time as well as available to view 24/7/365. Simple as that. I cant even view Comcast's bandwidth meter in my area. If these meters are not right, then I should be able to report them to the Government and they be fined. 

 

I also feel that if a company recedes tax payer dollars, for example from the USF. Then they should not be able to cap their connect. Im sorry, but if you get tax payer funds, then your network is partly owned by the tax payer. Maybe every one needs to start pushing local governments to deploy fiber, that would put these companies in their place. 

[skywake]

@leadeater

Yes peak load across the entire network is a bigger issue, I never said it wasn't. The problem with your argument though is that people don't usually co-ordinate the downloading of large files. The closest you get to that is when a game comes out and even then some people will pre-download it, some will download at midnight, others will wait until they get home from work, others will start it when they get up. Even worse for your argument is the fact that the faster your connection is the smaller the "window of overlap" is. So its not a big issue.

 

Really, the biggest load on consumer networks are streaming sites like Netflix. Because when you sit down to watch a show it goes for hours and is typically in the afternoon. Because of that there is always overlap. Its the same thing with electricity, what's the biggest problem on electric grids? Air-conditioners, by a big margin. Because not only do they pull a fair amount of W but they also run for quite a whine and churn through the kWh. Which means they overlap with other Air-conditioners, increasing the total W.

 

Basically lots of GB/mo? Across a lot of users that means lots of Mbps at the exchange.

[leadeater]

@leadeater

Even worse for your argument is the fact that the faster your connection is the smaller the "window of overlap" is. So its not a big issue.

 

Really, the biggest load on consumer networks are streaming sites like Netflix. Because when you sit down to watch a show it goes for hours and is typically in the afternoon. Because of that there is always overlap. Its the same thing with electricity, what's the biggest problem on electric grids? Air-conditioners, by a big margin. Because not only do they pull a fair amount of W but they also run for quite a whine and churn through the kWh. Which means they overlap with other Air-conditioners, increasing the total W.

 

Basically lots of GB/mo? Across a lot of users that means lots of Mbps at the exchange.

 

No I'm not meaning the higher the connection speed the smaller the window of overlap, I don't even know where you got the impression from. I'm even arguing against giving out high speed connections. I'm saying the higher the connection speed the worse the problem is for peaking loading which causes the most problems. Either you are not actually reading what I am writing, not understanding it or simply don't care what I'm saying. I accept that I could be explaining it badly or in a way that is not clear to you.

 

HD netflix uses 6.83Mbps and Ultra HD Netflix uses 15.83 Mbps. Why does someone who for the majority of usage watches netflix need a 100Mbps connection, they don't. I would say for this theoretical person they should buy either a 20Mbps or 25Mbps connection. This would allow 3 people to watch HD netflix or at the same time do other internet related tasks. What they can actually pick from right now is 30Mbps, 100Mbps or 200Mbps. There is also a fundamental lack of understanding in the PC and gaming community that a higher speed connection is better for gaming or has lower latency, this is not true. So they all go out and buy the biggest and fastest one they can afford for no actual benefit to them, pay for it months on end using barely 10%-30% connection bandwidth and then a major game release comes out where it could help but so many have these connections the content distributor or ISP falls over and no one can play or has issues using the internet in general.

 

In recent times when there has been large scale network outages they have been due to game releases from large groups of people downloading the game on excessively large connection bandwidths. For 99% of the year they are not using the 100Mbps they have so there is no sign of an ever increasing capacity problem as users sign up to these new connections until it is too late. Yes this is poor planning, exactly my point and a data cap will never and can never prevent this type of issue.

 

Your power company example is the exact same scenario, they have a maximum amount of wattage they can deliver to customers, exceed that and you get power cuts. What they can produce is normally finite however so that is why they charge for kWh. They are converting a raw material in to electrical energy and they must pay for the raw good except for solar, wind and hydro. Production of electrical energy even from renewable sources causes wear and requires maintenance and replacement. Pulling data through a computer network does not cause networking devices to fail or wear in a meaningful way compared to the device being on but idle. Providing data does not directly cost money, providing bandwidth does. Bandwidth over time is data so yes it is and acceptable way of charging but should not be used as a method of bandwidth control.

 

Data caps are bandwidth control by means of discouragement from using what you have because it can run out. In reality data can not possibly run out, but bandwidth can. Yes it works but is not a true effective means of bandwidth control and is not a real time method. Offering a greater range of internet plans with different speeds that are all unlimited data caps are a real time bandwidth control and is more effective. This is and only ever has been my point and is what is actually happening. Data caps should never have been a method of charging for internet access.

 

Edit: Also by offering only the higher data cap plans in conjunction with higher speed connections this compounds the problem of picking high speed plans they don't actually need.

[skywake]

No I'm not meaning the higher the connection speed the smaller the window of overlap, I don't even know where you got the impression from. I'm even arguing against giving out high speed connections. I'm saying the higher the connection speed the worse the problem is for peaking loading which causes the most problems. Either you are not actually reading what I am writing, not understanding it or simply don't care what I'm saying. I accept that I could be explaining it badly or in a way that is not clear to you.

Just on this bit (I'm not going to go through your entire post), I didn't say that's what you meant. I was the one who made that point. The indisputable fact of high speed connections is that when you're doing a task that uses all of it you finish that task quicker. The bit of my post that you decided not to quote? That went into why I disagreed with what you're saying. As I said, there's a reason why services like Netflix are the ones causing congestion issues.

 

When they switched on Netflix in Australia there were serious issues. Was it because people were using all of their bandwidth? No, not really. The issue was more that a lot of end users were using a decent amount of their bandwidth all at once. Not only that but for an extended period of time. In other words it was the "GB/mo" that hammered the network more than the "Mbps"

[braneopbru]

Just on this bit (I'm not going to go through your entire post), I didn't say that's what you meant. I was the one who made that point. The indisputable fact of high speed connections is that when you're doing a task that uses all of it you finish that task quicker. The bit of my post that you decided not to quote? That went into why I disagreed with what you're saying. As I said, there's a reason why services like Netflix are the ones causing congestion issues.

 

When they switched on Netflix in Australia there were serious issues. Was it because people were using all of their bandwidth? No, not really. The issue was more that a lot of end users were using a decent amount of their bandwidth all at once. Not only that but for an extended period of time. In other words it was the "GB/mo" that hammered the network more than the "Mbps"

You're missing the point. You've committed to this idea that total monthly transfers are what's causing the congestion and you're refusing to look at the situation logically.

 

The big cable that lands in perth; it's got 960 Gb/s of capacity. Let's say that as a small ISP, I lease 10 Gb/s on that line. I want to offer my customers 100 Mb/s service, so that means that I can only have 100 customers. OK, no problem. After a couple of months I look at my logs and realize that since my customers are all doing different things at different times and rarely even hit their max speed, my average bandwidth usage from all my customers is about 1 Gb/s and so I get a brilliant Idea. If my customers are only using one tenth of the bandwidth that I'm paying for, I could have 10 times more customers without any added expense of leasing more bandwidth.

 

So now I have 1000 customers that I've committed to offering 100 Mb/s to. That means that all my customers have and aggregate bandwidth of 100 Gb/s, but I only have 10 Gb/s to offer them. It's all fine and dandy as long as the use remains evenly spread out. I could even have every single customer running a 10 Mb/s stream 24 hours a day, 7 days a week and my customers would have no connection issues at all. The problem is when the handful of users that only come online twice a month, but download big files at the max speed that I offered them when they do come online. They aren't worried about their cap because they only use half of it, but 1% of my customers downloading at the speed that I promised are now using 10% of my leased bandwidth.

 

In my fictional scenario, I could easily provide all my customers with a stable connection that won't ever be overloaded if I offered them all 10 Mb/s service. But since customers are willing to pay three times more for the 100Mb service, I'll gladly let them pay me more for something that isn't going to cost me any extras.

 

All ISPs do this to a certain degree and it's the biggest difference between residential and business plans. They'll oversell the bandwidth on residential plans, but business plans have whatever bandwidth they pay for dedicated for them.

 

There's a reason that while my brother's package is capped at 300 GB, any download between 2 am and 7 am is unmetered. The ISP wouldn't offer anything free if it negatively impacted their network. His package that's capped at 300 GB would allow him to conceivably download 5 TB in a month. 

 

TL;DR: Monthly caps are used by ISPs as a deterent to customers using the speeds that they pay for so that the ISP can hide the fact that they oversell their bandwidth and offer bigger faster plans than they can realistically provide. 

[leadeater]

Just on this bit (I'm not going to go through your entire post), I didn't say that's what you meant. I was the one who made that point. The indisputable fact of high speed connections is that when you're doing a task that uses all of it you finish that task quicker. The bit of my post that you decided not to quote? That went into why I disagreed with what you're saying. As I said, there's a reason why services like Netflix are the ones causing congestion issues.

 

When they switched on Netflix in Australia there were serious issues. Was it because people were using all of their bandwidth? No, not really. The issue was more that a lot of end users were using a decent amount of their bandwidth all at once. Not only that but for an extended period of time. In other words it was the "GB/mo" that hammered the network more than the "Mbps"

 

Yea sorry, was just trying to cut down my already long post length by cutting out some of the quote and keeping what I was mostly addressing directly. Yea it's nice to have a fast connection but the real difference to a user between 20, 50 and 100 Mbps is not too great. If it takes 30 minutes vs 1 hour to get a game downloaded then I'd say that is an acceptable difference. These higher speeds however do not increase internet browsing speed or make any different to video streaming other than the amount you can stream at once on a single connection.

 

As you are already aware the law of averages applies with the amount of users that use the internet. By moving from ADSL2+ 8-16 Mbps to VDSL2 30-50 Mbps and Fibre 30-200 as quickly as what has happened the infrastructure has not been able to keep up. Not enough time was taken to wait for internet users shift in habits before offering connections greater than 50 Mbps. Average sustained network load was increasing as expected but what was not taken in to account was the ability to use new content types further pushing this up all while giving vast amounts of users the ability to give very large peak traffic bursts on the network. The rate at which average network load was increasing was much slower than the rate of possible load so as new content types were being discovered or the use of previously unusable content there were quick sharp jumps in usage. Recipe for disaster.

 

Fibre has more benefits than higher connection speeds, lower latency and cheaper for the ISP to maintain once it is in place. It would have been easy to market fibre plans with lower speeds to many internet users simply from the lower latency. A 20Mbps fibre connection is noticeably better than 20Mbps VDSL2. As infrastructure is upgraded you could either drop the prices or introduce new faster plans or even both. By having a more controlled release of increased bandwidth rather than just giving the fastest as technically possible at the time would have been better for the consumer and the ISP. 

[skywake]

@braneopbru

I never said it was as simple as saying that the data consumption was "spread out" entirely. Obviously there are times of peak demand, obviously there are times when connections are idle. My point was that when average out lots of users you'll never get times when everyone is hitting the connection 100%. That giving people slower connections to "solve" the problem isn't going to help. Because when you scale to lots of users its the MB that kills the network more than the Mbps.

 

@leadeater

Going back to the power network example for a second because that analogy is sound. You said that with data there is infinite supply and so that the analogy didn't work, I think you missed the point there. The issue with air-conditioners in the summer wasn't much to do with expensive energy at all. Infact per kwh power has never been cheaper than it is now. No, the issue was with capacity. And not because people were drawing more W than they ever had before, people could always draw that many W. No the issue was that people were drawing a fair but non-crippling amount of W over a sustained period of time. i.e. lots of overlap, lots of kwh. It's the same deal with Netflix.

[leadeater]

@skywake

 

A power plant can only give so many watts that it can generate. It can do that for as long as it has the resources to do so. Generators are rated in Watts or VA (Volt Ampere), exceed that rated output it will brown out or cut off. For a power company both the Watts and the length of time you use it for cost them money directly. We also don't know how much watts our house needs and some devices have very short and extremely high power draw so buying a fixed rate power connection is actually dangerous, electrical equipment can break when under powered unless protected. For an ISP only the bandwidth (Watts) costs them money, the length of time you use it for does not.

 

For power the best charging unit is kWh for internet connections Mbps is the best.

 

By charging for a data unit that is by nature implying that it can run out or has an unit cost, while data cannot run out bandwidth can. There is a cost for proving an internet connection: cabling, equipment, support etc all have a cost that the ISP must account for in delivering these services to customers. This is typically made up in the standard line rental and base monthly connection fee. On top of those costs are the bandwidth fees they must pay to higher tier internet network providers which they then pass on you the consumer. So if the cost of giving you the customer the base connection is already covered the only extra cost that can be incurred is by giving out faster connections. Data moving within an ISP's network costs them nothing just to be clear.

 

Each road side cabinet has an uplink or connection from that box to a local exchange and the local exchange to a larger distribution etc. A road side box uplink only has so much bandwidth it can sustain, this is a technology and equipment limit. Multiple road side boxes feed in to local exchanges which also have bandwidth limits. All these things are ISP owned so bandwidth costs nothing to them directly.

 

These bandwidths are always oversubscribed and rightly so as not every connection is used all the time or at the full rated speed. The industry calls this contention ratio. The typical contention ratios for each type of connection are as follows:

• ADSL: 50:1 to 100:1
• VDSL: 30:1 to 50:1
• Fibre: 10:1 to 20:1

Notice as the faster the connection type that can be delivered to the consumer the contention ratio goes down. This is due to the available bandwidth increasing far less than the client connections. This is not the ISP trying to give the consumer a better service. What this means is that a single connection can have a bigger impact on the available bandwidth causing it to run out and there for congestion. By offering lower connection speeds you can have larger contention ratios reducing single connection impact and also lowering the cost per connection. By offering lower data cap plans this does nothing, by offering higher data cap plans this also does nothing. For quality of service less speed is better. The rate at which client connection speeds increase should not significantly outpace the uplink increase. 

 

 

Network congestion is the situation in which an increase in data transmissions results in a proportionately smaller increase, or even a reduction, in throughput. Throughput is the amount of data that passes through the network per unit of time, such as the number of packets per second.

 

Data does not cause congestion, data is a measurement on an amount. Networking equipment are measured in rates, Gbps and Packets Per Second. The total amount of data that has passed through a networking device in a month does not matter, no one monitors network utilization on a month time scale.

[skywake]

@skywake

For power the best charging unit is kWh for internet connections Mbps is the best.

kWh and GB are amounts, Mbps and W are rates. Why should ISPs not charge for the amount when every other utility charges in that way?

 

I'll put it this way, a power utility could give a house access to way more W than they would ever need. I would argue that that's precisely what they do. How often does a single house blow their fuses let alone a transformer? Not very often I'd bet. So I'd argue the power network is setup to be the equivalent of giving everyone 1Gbps internet connections and then charging per GB. And you'd argue it's not the same because "data doesn't cost". The problem with that argument is that the infrastructure and the maintenance is not free. Same deal with powerlines and other utilities. Even if you run on solar/wind/hydro you still pay for the poles, wires, substations etc. In Australia it's at the point now where the infrastructure itself is where most of the cost of power goes towards. That's the side of the network that buckles under the load. Same as an ISP struggles to cope with streaming services like Netflix.

 

So it stands to reason that the people who use more units, GB or kWh, should be the ones to pay more. We should give people huge data connections that they "don't need" because sometimes people will use that extra bandwidth even if most of the time they won't. We should then charge per GB via quotas because it's the heavier users who are responsible for most of the congestion. It's not a complicated argument. I honestly don't know why you are so convinced I am wrong.

[leadeater]

@skywake

As I have already mentioned this is a field I know very well. I have formal training and certifications in networking. I also converse with internet network engineers and the network that is operated at my work place is configured the same way the internet is and uses the same routing protocols. Our network spans the entire North Island to multiple locations and we have very high connection bandwidths.

 

Your assessment of the issues or more so the causes are mostly correct. Netflix has played a huge part in internet bandwidth issues in both Australia and New Zealand. What you are missing is that network issues happen in very small time scales, seconds and minutes but have large flow on effects that last much longer than the original issue. Also all networking devices and connections are measured in data rates, Gbps for example. These connections are shared with many users and the faster their connection the less number of users can share that connection and the harder it is to share it.

 

Previously before the roll out of very high internet connection speeds to the masses it required thousands of users to have noticeable impact on the network as a whole, now it can be as little as a hundred or so. That has nothing to do with Netflix and everything to do with the excessive amounts of users with high speed connections in comparison to the available network bandwidth.

 

How much data a user uses in a month is of zero concern to an ISP, they truly have no interest in this figure. It has very little impact on the required infrastructure. What does matter is the average bandwidth used and peak possible per user and also as a combined average of all users in each network segment over smaller time scales of seconds, minutes and hours. Small peaks in traffic that exceed the available bandwidth in a network segment is not hard to deal with, QoS/throttling applies here. These are only so effective and if the peak is high enough you get very big problems. These bandwidth utilization statistics are used for capacity and upgrade planning.

 

Data caps have historically been used as a way to get the average bandwidth utilization down not on a per user basis but on larger scales. This technique worked well enough with small data caps but as they increase get less effective. Since data caps have been proven to not be effective at the task they are used for continuing to use them is pointless. Data caps are also only common in what are essentially 3rd world internet countries such as ours. The truth sucks. The top internet countries don't charge for data usage.

 

I am just glad that our FTTH network roll out is going much better than Australia and isn't getting screwed around with and plans changed. Also at least for us data caps are slowly becoming a thing of the past.

[Donut417]

@skywake. If we charged per usage that would mean the ISP a utility. That would me they would have to conform to the rules of a utility. Does Comcast or any other ISP want to do that? Also ISP can not exempt services from the bandwidth caps. Like Tmobile does and Comcast does with their new video service. 

[leadeater]

kWh and GB are amounts, Mbps and W are rates. Why should ISPs not charge for the amount when every other utility charges in that way?

 

Because if the amount has no cost to them they have not right to charge for it, like every other half decent country in the world. I state again no proper country charges for data usage. Continuing to think this is an acceptable practice is harmful to you as the consumer.

 

Also I already said that infrastructure costs money.

 

There is a cost for [providing] an internet connection: cabling, equipment, support etc all have a cost that the ISP must account for in delivering these services to customers. This is typically made up in the standard line rental and base monthly connection fee.

 

The higher the connection speed of each user the harder it is to deal with. This is a fundamental truth of networking. If the ratio of client connection speed to available network upload connection speed gets to far out of balance there will be issues.

 

You are focusing too much on the broad overview and completely forgetting about the equipment that actually makes up these networks and what they are capable of. Sustained traffic like Netflix is easy to deal with and is a predicable load. As it stands now Netflix causes no issues here in NZ and hasn't for a long time, it was only a problem for a very short time.