Best Way To Measure Network Usage Across All Devices?

[RadiatingLight]

Every month, my household passes the 1024GB limit imposed by Xfinity/Comcast, and I have no idea why. personally, my PC uses 250GB / Month, and all the other devices I can measure add up to about 350 More.

Factoring in a HUGE margin of error of 250GB, then I still don't see how we cross this limit.

 

What's the best way to measure what device is using all the data?

I have a Nighthawk X8 router (overkill i know) if that helps.

[GDRRiley]

open your routers software and turn on the tracking system it should say what is eating it all up. 

[RadiatingLight]

1 minute ago, GDRRiley said:

open your routers software and turn on the tracking system it should say what is eating it all up. 

It doesn't though. it says the cumulative total across all devices, and it says what each device is using right now, but it doesn't say the total a device has used this month.

[GDRRiley]

1 minute ago, RadiatingLight said:

It doesn't though. it says the cumulative total across all devices, and it says what each device is using right now, but it doesn't say the total a device has used this month.

really update the firmware for it and maybe it will show up. 

[SCHISCHKA]

if your router can't track your usage then you can put a computer between your modem and router to record packets. you will see every web site, video service, torrents and the local IP the traffic is from

[P4ZD4]

I'm really curious to know whats eating that much data 

[RadiatingLight]

39 minutes ago, P4ZD4 said:

I'm really curious to know whats eating that much data 

4K Youtube

Netflix / Video Streaming

Backups of 4K Video from phones

 

But all that adds up to about 600GB

[RadiatingLight]

43 minutes ago, SCHISCHKA said:

if your router can't track your usage then you can put a computer between your modem and router to record packets. you will see every web site, video service, torrents and the local IP the traffic is from

What do you mean, between your modem and router?

I would need a computer with two Ethernet ports at least (so i would need to buy a NIC)

what special program would run on said computer to make it track everything? Can I simply use my existing PC with a NIC in it?

[SCHISCHKA]

3 minutes ago, RadiatingLight said:

What do you mean, between your modem and router?

I would need a computer with two Ethernet ports at least (so i would need to buy a NIC)

what special program would run on said computer to make it track everything? Can I simply use my existing PC with a NIC in it?

you could run a proxy on your computer to direct all traffic to your machine and avoid multiple NICs but you would not be able to force other users to use it.

http://www.squid-cache.org

https://blogs.msdn.microsoft.com/canberrapfe/2012/03/30/capture-a-network-trace-without-installing-anything-capture-a-network-trace-of-a-reboot/

[RadiatingLight]

1 minute ago, SCHISCHKA said:

you could run a proxy on your computer to direct all traffic to your machine and avoid multiple NICs but you would not be able to force other users to use it.

http://www.squid-cache.org

https://blogs.msdn.microsoft.com/canberrapfe/2012/03/30/capture-a-network-trace-without-installing-anything-capture-a-network-trace-of-a-reboot/

Well i want the network to function like normal, which means that all my computer should do is monitor the network traffic, and not alter it in any way.

I'm fine with buying a NIC like this one (https://www.amazon.com/Ethernet-Realtek-Standard-Brackets-SD-PEX24041/dp/B00O1AEFLK/ref=sr_1_1?ie=UTF8&qid=1485934693&sr=8-1&keywords=2+Port+LAN+PCIe+card) to use exclusively for monitoring if it means I can monitor everything, but I don't think it will work, because I want to know which device on my network is using data, so I would need to know the internal IP or mac address.

[SCHISCHKA]

3 minutes ago, RadiatingLight said:

Well i want the network to function like normal, which means that all my computer should do is monitor the network traffic, and not alter it in any way.

I'm fine with buying a NIC like this one (https://www.amazon.com/Ethernet-Realtek-Standard-Brackets-SD-PEX24041/dp/B00O1AEFLK/ref=sr_1_1?ie=UTF8&qid=1485934693&sr=8-1&keywords=2+Port+LAN+PCIe+card) to use exclusively for monitoring if it means I can monitor everything, but I don't think it will work, because I want to know which device on my network is using data, so I would need to know the internal IP or mac address.

you can capture MAC addresses. also have a look at wireshark.org it has a nice GUI for analysing packets

[P4ZD4]

You can always flash a custom firmware like dd-wrt. But that's a pretty expensive router so i don't know if I'd experiment on it

[SCHISCHKA]

https://wiki.wireshark.org/Ethernet

 

[Falconevo]

To capture all the information from every device on the network, you would need logging to be enabled on the edge device to capture the information.

This would need to be the router itself or a device in front of the router that devices on the network utilize as a proxy/router as their default gateway to pass data back to the X8. 

 

DD-WRT would probably be an option as it has remote syslogging capability to send the capture to a syslog server for building pretty graphs out of using something like graylog.  Very few residential modem/routers have the capability of logging without custom firmware or just using them as a bridge and putting something more sensible in place.

 

Personally I do my device graphing and usage via a pfSense box to capture what each device is using and if anyone is taking the piss

[Brightglaive]

I was amazed to find out that 1080p youtube video rates start at 3Mbps-6Mbps and goes up from there. As you go up in resolution and framerate you can jump up to as much as 51Mbps (2160p60). If you watch a ten minute 1080p Youtube video that's 600 (Seconds) x 6Mbps = 3600Megabits/8 bits per byte = 450Megabytes for *ONE* video. jump up to 60fps on the 1080p and  your rate jumps to 4.5Mbps-9Mbps! that's around 680 Megabytes for one ten minute video. If you use a smartphone that can go up to 1440p or 1440p60 then now you range between 6 and 18Mbps. That's over a Gigabyte per 10-minute video.  How many ten minute videos can you watch in one day?  I know I watch at least 6 to 10 myself (and that's not counting netflix, or hulu, or web surfing, or streaming audio, amazon prime audio and video titles, audible book downloads, playing games etc. ) 

 

If you have a large household with multiple smart TVs and Hulu/Netflix/Amazon Prime accounts. And 2 - 4 computers (that's counting Laptops AND desktops), and multiple smartphones connecting to your wireless network.... I could see how you might be able to chew through 1 TiB of data

 

(incidentally you can save a TON of data just by locking your video on  your mobile phones youtube app to 360p and by turning off the autoplay for videos on facebook)

 

And for the tech heads....Yes,yes I know...conversion rates and transmission compression...yaddah,yaddah, yaddah...the final amount of data you get is not the same....etc. etc. True...The downloaded size may not be the same, but the data is not measured on your storage now, is it? It's measured at the telco transmitter/ISP-side router or at worst at the ISP "modem" before having all that extra network transmission garbage stripped off by your network card or at the modem when it converts it to ethernet and IP.