Comcast extends data caps

[Bombastinator]

Just now, wamred said:

Yeah, the bad thing about it is data caps are completely fake. They are just arbitrary numbers that ISPs use. It is infuriating.

There is the reason and the rationalization and they are not always the same.  The rationalization for data caps is they are supposed to prevent abusive users from gaming the system and effectively forcing the group to pay for their usage.  They can also be used simply to make money for the carrier though.   Carriers are service repackages much like garbage collectors.  Garbage collectors pay by weight but charge by volume.  This means they make more money when people throw away light things that fill cans. This is also why they tend to have a maximum weight limit on cans.  Carriers are similar.  They pay by the byte, but sell by speed. It makes a certain amount of sense to have a maximum cap.  The problem is that cap can be set low enough that non abusers pop the cap too.   

[wamred]

8 minutes ago, Bombastinator said:

There is the reason and the rationalization and they are not always the same.  The rationalization for data caps is they are supposed to prevent abusive users from gaming the system and effectively forcing the group to pay for their usage.  They can also be used simply to make money for the carrier though.   Carriers are service repackages much like garbage collectors.  Garbage collectors pay by weight but charge by volume.  This means they make more money when people throw away light things that fill cans. This is also why they tend to have a maximum weight limit on cans.  Carriers are similar.  They pay by the byte, but sell by speed. It makes a certain amount of sense to have a maximum cap.  The problem is that cap can be set low enough that non abusers pop the cap too.   

Um, all I’m saying is I work for an ISP and we pay for our internet by bandwidth not bytes. 

[Donut417]

15 minutes ago, Bombastinator said:

There is the reason and the rationalization and they are not always the same.  The rationalization for data caps is they are supposed to prevent abusive users from gaming the system and effectively forcing the group to pay for their usage.  They can also be used simply to make money for the carrier though.   Carriers are service repackages much like garbage collectors.  Garbage collectors pay by weight but charge by volume.  This means they make more money when people throw away light things that fill cans. This is also why they tend to have a maximum weight limit on cans.  Carriers are similar.  They pay by the byte, but sell by speed. It makes a certain amount of sense to have a maximum cap.  The problem is that cap can be set low enough that non abusers pop the cap too.   

Other providers dont have caps. Also, even if you pay for "Unlimited data' at about 10 TB Comcast is going to call and bitch you out. Sorry if Im paying $30 extra each month and I want to use 100 TB, they have no right to call me up and bitch me out. 

 

I understand they have to make sure a few dont ruin it for the others. But for Comcast this is just a money grab. They saw cellular providers doing this in the early to mid 2000s and they jumped on the bandwagon. 

 

Another fact is their usage meter has NEVER been inspected by a government organization. They are not under any regulations in regards to their usage caps. DTE is under government supervision, gas pumps are under government supervision, the scales at my grocery store are under government supervision. Why not Comcast's meter? How do I know they aint fucking me over? 

[leadeater]

14 minutes ago, Donut417 said:

Sorry if Im paying $30 extra each month and I want to use 100 TB, they have no right to call me up and bitch me out. 

 

I understand they have to make sure a few dont ruin it for the others. But for Comcast this is just a money grab.

100TB over 30 days is only ~323Mbps sustained bandwidth, really nothing an ISP of that size would worry about from a network bandwidth perspective. In reality ISP's only have to worry about peak capacity, everything below that is near irrelevant other than peering agreements at internet exchanges with other networks where you pay by the average percentile bandwidth over the month, "inside network" bandwidth usage is "free".

 

The touted reason for data caps is for network bandwidth control but it's so indirect that it is completely ineffective as a control measure because it in no way prevents or even statistically reduces the chance of links within the network having demand exceed the bandwidth capability.

 

Data caps exist to make more money, that's all they do.

[Donut417]

9 minutes ago, leadeater said:

00TB over 30 days is only ~323Mbps sustained bandwidth,

They only allow 10 TB before you get a warning. They will force you on a business plan. 

[leadeater]

1 minute ago, Donut417 said:

They will force you on a business plan. 

Yea and I bet that is real cheap...., not.

[Bombastinator]

29 minutes ago, wamred said:

Um, all I’m saying is I work for an ISP and we pay for our internet by bandwidth not bytes. 

Then things have changed.  It’s possible the ISP is paying a company that already does this exchange.  There are still some very small ISPs out there that might not actually be connected to a backbone.   

[Donut417]

Just now, leadeater said:

Yea and I bet that is real cheap...., not.

If I ever pay for unlimited data Im going to run a Bittorrent client on my server. Ill share Ubuntu or something Legal. Dedicate 2 Mbps out of my 10 Mbps for the project. Just to spite them assholes. Because they HATE bittorent. 

 

Business class runs on the same lines as residential. I think for the $86 we pay for 200/10, you might get 75/10. 

[leadeater]

3 minutes ago, Bombastinator said:

Then things have changed.  It’s possible the ISP is paying a company that already does this exchange.  There are still some very small ISPs out there that might not actually be connected to a backbone.   

Transit agreements and wholesale network rates have always been charged on bandwidth not volume of data. Only consumers have paid for volume of data. The reason for that is networks are managed by bandwidth not consumption of data (volume), the ability to generate data doesn't run out by why of consumption only the rate at which it can be generated.

 

In network management perspective customers consume bandwidth not data so putting a unit of charge on to data makes no sense and that is why it's not done between networks, big or small.

 

Larger networks where there is generally equal amount of bandwidth between them will enter a Peer agreement where there is no charging between each network as they are making a mutual fair exchange agreement.

 

Transit: Pay by rate used between network

Peer: Free exchange between network

[Bombastinator]

12 minutes ago, leadeater said:

Transit agreements and wholesale network rates have always been charged on bandwidth not volume of data. Only consumers have paid for volume of data. The reason for that is networks are managed by bandwidth not consumption of data (volume), the ability to generate data doesn't run out by why of consumption only the rate at which it can be generated.

 

In network management perspective customers consume bandwidth not data so putting a unit of charge on to data makes no sense and that is why it's not done between networks, big or small.

 

Larger networks where there is generally equal amount of bandwidth between them will enter a Peer agreement where there is no charging between each network as they are making a mutual fair exchange agreement.

 

Transit: Pay by rate used between network

Peer: Free exchange between network

Im pretty sure always isnt accurate, though I may be out of date.  I’m only familiar with the original system of internet interconnects which started as being university based.  Internet was new then.  Before that was AOL, independent BBSes and such things.  Usenet was dark magic.  I stopped paying attention Back when block numbers were simply being given away for the asking.  Bandwidth can be converted into a flat data number  as long as the bandwitdth is saturated.  

[leadeater]

1 hour ago, Bombastinator said:

Bandwidth can be converted into a flat data number  as long as the bandwitdth is saturated.  

Yes but network providers do as much as possible to ensure links are not saturated, once they are then packets or frames depending on link layer transport technology get dropped which effects quality of service. That is why it's charge rated out on percentile of bandwidth over time not total volume of data used, you cannot take total volume of data and translate back to a peak bandwidth requirement for any given link to ensure quality of service.

 

If you go back far enough there wasn't even an ability to monitor things like bandwidth or volume of data usage, everything was flat rate charged on the link bandwidth i.e. 100% committed charged. For a short period of time the charging model was indeed data usage over a month but it was very quickly identified as not a useful way to charge.

 

Quote

The Origin of the 95th Percentile Measurement

How did the 95th percentile come into existence? 

The Internet Transit pricing model went through three distinct pricing phases:

• Phase 1 - Internet Transit was sold by access capacity. In the early Internet days, Internet Transit was billed on a circuit-capacity basis. You would purchase a T1 circuit (1.544 Mbps) to the Internet and pay as if you used the entire circuit capacity 24 hours a day. But if you didn’t use very much of this capacity, you were still paying as if you did. At the time, this reality made Internet Transit tough to sell to some low-volume prospects.
• Phase 2 - Internet Transit sold by average usage. One of the early ISP pioneers started charging based on average use, but this measure ended up being skewed by the burst nature of Internet traffic.  Customers who used only a little transit were paying as if they used much more because the average was skewed upwards by these occasional peaks.
• Phase 3 - Internet Transit sold by 95/5 model. To address this situation, one ISP adopted the 95th percentile measure to prevent overly punishing a customer for the occasional spike in traffic volume, while still allowing the ISP to bill based on the load placed on its network. This approach seemed palatable to the market and sold well. The rest of the industry adopted this pricing model, and within a year or so 95/5 became the pricing paradigm for the Internet Transit service.

https://drpeering.net/core/ch2-Transit.html

 

Phase 3 has been the model in place for what is now the majority of the existence of the internet, Phase 1 being the next longest standing method. Phase 2 was insignificant, not even worth talking about for  Transit agreement discussions due to how shortly it was used..

 

Data usage is literally a cumulative average of bandwidth over time, it's not useful information at all.

 

I would however like to point out that it can be easy to abuse these charging methods, something Google has been guilty of. Google has extremely high bandwidth links between networks and they used to delay all data replication over the month to the very end then in the last day or couple of days completely max out the bandwidth and only be charged a much smaller amount as it's the 95th percentile over the month and there are significantly more bandwidth sample data points at very low bandwidth than very high. That was dealt with when they were found out.

[Bombastinator]

I’m not understanding this inability to measure data.  That predated even the internet.  The T1 leased line system had nothing to do with internet stuff. that was ma bell. A t1 was literally a wire. The phone company didn’t care what the line was used for or what was put through it.  There wasn’t just one bill.  The company leased the line from the phone company at a flat rate, but it also paid the owner of the backbone for the data it ran to other networks.  The two were not connected.  The issue is a given network like Comcast or CenturyLink or whomever run their own little network that is not precisely the internet.  The internet is a collection of smaller networks that all use the same protocols, A given network can have much larger capacity than the actual connection between it and other networks. When it passes information through to another network it has to pay for the privilege. Whether this is by byte or by pipe is irrelevant.  Bandwidth IS data rates. a given line of a given type can only pass so much data a second.  Lines have developed more and more capacity and costs have gone down, but it’s not and can never be zero. You want to pass more data you need more or bigger lines. And they need to be paid for.  There is an assumption of infinity in This concept that  a company doesn’t actually  care how much data it is moving.  To a degree they don’t, but they do have to pay for the connections.  Higher bandwidth connections cost more money.  It used to be there was just analog phone copper, t1, and t3.  Now there are many more but each still has a capacity. There is no automagical.  Somewhere there has to be a wire (be it copper or glass) that passes data.  A physical thing.  More data passes more physical thing is needed. 

[leadeater]

1 hour ago, Bombastinator said:

I’m not understanding this inability to measure data

In the early days there was no AAA software to do the accounting of data usage, there was no way to track usage so how do you bill it? You don't you just charge for the link bandwidth outright i.e assuming 100% of it is utilized 100% of the time. Which obviously isn't the case but there was no other way to charge.

 

The example was only with a T1 line, really replace that with dialup if you want it doesn't change anything.

 

The reason they could not track it was that even if a switch or router interface had a data counter it was incremental forever and would reset to 0 if the link goes down. Now I ask you how do you reliably bill on data usage like that? Take a start value then end value and bill the difference? Fat load of good that does if the link goes down in the middle of the month, or worse I'm a jerk customer and link downs on the last day so you end up with negative value to bill me on.

 

Seriously actually read the link I gave, it explains all of this.

 

1 hour ago, Bombastinator said:

More data passes more physical thing is needed. 

Wrong, you need to take measurements of bandwidth usage at points in time over a month and take a percentile of that. 100GB over 10 days is not the same as 100GB over 30 days, a 95th percentile has at least a better chance to come out with a higher bandwidth figure to bill on than 100 divided 30 then divided down to Mbps.

 

Think about the math difference here, it's very clear what the difference is if you toss out your pre-conceived notion that data usage means anything. Break it down, only look at the math theory difference between an average over time versus a 95th percentile of data points. There's a significantly large difference. Hence why Transit agreements between networks are 95th percentile not data usage. Until this mathematical difference clicks you won't understand why.

[Bombastinator]

AOL had data limits. That was pre-internet.  Your math thing is based on amortization over time.  You stated that backups could not be allowed so therefore a  maximum exists on a moment by moment basis.  You can’t hold data to pass it through the channel more than a few microseconds or there is lag. 

Edited February 20, 2021 by Bombastinator

[leadeater]

8 minutes ago, Bombastinator said:

AOL had data limits. That was pre-internet

AOL service was for consumers, are you reading anything? Transit peer agreements is the discussion which has come up on the basis that charging consumers for data usage in the name of network bandwidth control is a lie because this is done nowhere else in the network, it is exclusive to consumers.

 

If it were about bandwidth control we, the consumer, would have the same method of being billed as network transit partners get billed, 95th percentile of bandwidth usage taken by measuring bandwidth at set intervals.

[leadeater]

@BombastinatorHere's an extremely rough example for you, sample granularity is only daily rather than say every 5 minutes but I think you'll see the problem.

 

 

Your way you're effectively charging for 34Mbps bandwidth usage, how it's actually done would result in 73Mbps being charge for.

 

Assumptions used, each day those bandwidth figures are sustained for the entire 24 hour period with no change. Before you say that is a flaw that actually favors data usage more not less.

[Bombastinator]

2 minutes ago, leadeater said:

AOL service was for consumers, are you reading anything? Transit peer agreements is the discussion which has come up on the basis that charging consumers for data usage in the name of network bandwidth control is a lie because this is done nowhere else in the network, it is exclusive to consumers.

 

If it were about bandwidth control we, the consumer, would have the same method of being billed as network transit partners get billed, 95th percentile of bandwidth usage over time taken by measuring bandwidth at set intervals.

The statement was whether it existed or not.  It did.  Remember there isn’t one user.  There are frequently hundreds of thousands on a given section of a single link in a subnetwork.  There was a period where water companies could tell when the Super Bowl was on because everyone would flush their toilets near the same time and create massive strains on the infrastructure. The water companies had to build for that “high water” crest.  The system could be overbuilt for 99% of the year, but it still had to handle the few minutes of a Super Bowl commercial.  Say you’ve got that one section with a hundred thousand people on it.  10 of them do  90% of the network traffic.  If those 10 weren’t there there would be 1/10th the traffic.  Do ISPs overbook their networks?  Of course.  If every one of those hundred thousand people used as much as the 10, which they would technically be allowed to do, the system would likely exceed its capacity a keel over.  But they don’t.  So it works.  The company spends less money on connections so it can charge less money per customer. 

[Bombastinator]

6 minutes ago, leadeater said:

@BombastinatorHere's an extremely rough example for you, sample granularity is only daily rather than say every 5 minutes but I think you'll see the problem.

 

 

Your way you're effectively charging for 34Mbps bandwidth usage, how it's actually done would result in 73Mbps being charge for.

 

Assumptions used, each day those bandwidth figures are sustained for the entire 24 hour period with no change. Before you say that is a flaw that actually favors data usage more not less.

Image is failing I’m seeing a white box that flickers to light grey.

[leadeater]

Just now, Bombastinator said:

The statement was whether it existed or not.  It did.  Remember there isn’t one user.

Yes, for 1 ISP for about a year, as covered in the link I gave you. Charging for data usage simply wasn't common so it CAN be discarded, it's not useful.

[leadeater]

7 minutes ago, Bombastinator said:

Image is failing I’m seeing a white box that flickers to light grey.

Try now, if not TL;DR your way would result in charging for 34Mbps of link bandwidth utilization and how it's actually done and been done that way for decades (data usage charging existed for a year done by 1 ISP) resulting in charging for 73Mbps of link bandwidth utilization.

 

Peak utilization of the link was 85 Mbps, 73 is closer to 85 than 34 is.

 

Edit:

Forgot to add context for why it matters. If multiple customers are sharing a link and the link maximum bandwidth is 100Mbps then you can actually only commit another 27Mbps to another customer not 66Mbps.

[Bombastinator]

2 minutes ago, leadeater said:

Yes, for 1 ISP for about a year, as covered in the link I gave you. Charging for data usage simply wasn't common so it CAN be discarded, it's not useful.

By which you mean charging by the byte per user.  Is it an effective fiction?  Sure.  It’s a methodology though.  A given channel can’t carry more data than it can carry in a given slice of time.  If you make that slice a month the amount of data it can carry is very very large because data moves very very fast.  If you make that slice one microsecond it is much smaller.  How much lag do people willingly put up with? I’m remembering something like 40 microseconds. This puts an upper limit on how much a given data pathway can pass data. If as you say data is NEVER allowed (which I’m not totally sure of because lag is a thing) to build up things get even tighter. Got to have more or bigger channels.

[leadeater]

9 minutes ago, Bombastinator said:

By which you mean charging by the byte per user.  Is it an effective fiction?  Sure.  It’s a methodology though.  A given channel can’t carry more data than it can carry in a given slice of time.  If you make that slice a month the amount of data it can carry is very very large because data moves very very fast.  If you make that slice one microsecond it is much smaller.  How much lag do people willingly put up with? I’m remembering something like 40 microseconds. This puts an upper limit on how much a given data pathway can pass data. If as you say data is NEVER allowed (which I’m not totally sure of because lag is a thing) to build up things get even tighter. Got to have more or bigger channels.

Dude you're not getting it, there is no delay, no stopping traffic. Literally what is happening is that every 5 minutes the link bandwidth is recorded, so 8640 sample points over 30 days. These samples could be any bandwidth number below or up to the maximum bandwidth of the link. At the end of the month the 95th percentile of all these points is taken and that is the bandwidth charged for.

 

This is measuring actual bandwidth, each and every time. If you want to bill for bandwidth because bandwidth is the thing you need to control to prevent the link getting saturated and therefore a loss of quality then you need to be measuring bandwidth, not volume. Sum total of data over time is useless, it's completely inaccurate. 

 

The way it is done has 8640 sample points, the way you think it happens has 1 sample point because it's an aggregate figure and by doing that all measurement resolution is lost.

[Bombastinator]

13 minutes ago, leadeater said:

Try now, if not TL;DR your way would result in charging for 34Mbps of link bandwidth utilization and how it's actually done and been done that way for decades (data usage charging existed for a year done by 1 ISP) resulting in charging for 73Mbps of link bandwidth utilization.

 

Peak utilization of the link was 85 Mbps, 73 is closer to 85 than 34 is.

 

Edit:

Forgot to add context for why it matters. If multiple customers are sharing a link and the link maximum bandwidth is 100Mbps then you can actually only commit another 27Mbps to another customer not 66Mbps.

Got it that time.  The only one I suspect an isp would care about is Wednesday.  Probably a tiny fractional section of that day.  Got to find those fractional sections though.  Any user could produce one or two.  What gets to be an issue is if a given user is producing hundreds. Or thousands, or hundreds of thousands.  Is a monthly number the best possible way to deal with such issues?  Almost certainly not, but bills are monthly so it’s what gets used.  Is that what they are actually doing?  Probably not.  As I said before a rationalization and a reason don’t have to be related.

[leadeater]

1 minute ago, Bombastinator said:

Almost certainly not, but bills are monthly so it’s what gets used.  Is that what they are actually doing?  Probably not.  As I said before a rationalization and a reason don’t have to be related.

It's still charged monthly, you do not lose that ability. You however get to charge for 73Mbps of bandwidth rather than 34Mbps, here the ISP wins out. Not great for the consumer right? So if your usage, like most consumers, is very bursty you want to be charged for data usage total over the month. Thing is if ISPs are saying data caps are for managing network bandwidth like they are saying that is actually false, the maths doesn't stack up and it's not actually how networks work either. If it were about bandwidth control they would be taking those 8640 measurement points and charging you on the basis of bandwidth.

[Bombastinator]

19 minutes ago, leadeater said:

Dude you're not getting it, there is no delay, not stopping traffic. Literally what is happening is that every 5 minutes the link bandwidth is recorded, so 8640 sample points over 30 days. These samples could be any bandwidth number below or up to the maximum bandwidth of the link. At the end of the month the 95th percentile of all these points is taken and that is the bandwidth charged for.

 

This is measuring actual bandwidth, each and every time. If you want to bill for bandwidth because bandwidth is the thing you need to control to prevent the link getting saturated and therefore a loss of quality then you need to be measuring bandwidth, not volume. Sum total of data over time is useless, it's completely inaccurate. 

 

The way it is done has 8640 sample points, the way you think it happens has 1 sample point because it's an aggregate figure and by doing that all measurement resolution is lost.

I agree that it’s inaccurate.  Every 5 minutes doesn’t matter. It could be every second. Either might as well be annual. It’s still way too slow.  It’s how much money they pay to have enough bandwidth to keep the thing from ever  filling up. The more traffic there is the more they need.  It’s in theory not actually about correctly charging by volume.  It’s about finding the top1% of users that produce the lions share of the traffic, and moving them on to a different group.  In theory anyway.  Whether there even are a top 1% of users that produce the lions share of the traffic is something one has to take on faith, and having faith in a for profit company is rarely a good practice long term. 
 

6 minutes ago, leadeater said:

It's still charged monthly, you do not lose that ability. You however get to charge for 73Mbps of bandwidth rather than 34Mbps, here the ISP wins out. Not great for the consumer right? So if your usage, like most consumers, is very bursty you want to be charged for data usage total over the month. Thing is if ISPs are saying data caps are for managing network bandwidth like they are saying that is actually false, the maths doesn't stack up and it's not actually how networks work either. If it were about bandwidth control they would be taking those 8640 measurement points and charging you on the basis of bandwidth.

Doesn’t matter for purposes of finding abusers.  So there is inflation of a number.  It would be happening equally to everyone.  Reality is still reality.  The descriptor is arbitrary for purposes of finding difference.   It’s not the raw number a person uses it’s how it compares to others.  What is the average number? It could be 34 or 78. Either one could be X. They’re in theory not looking for x, they’re looking for 2x or 200x.

Edited February 20, 2021 by Bombastinator