Cloudflare DNS outage

[AngryBeaver]

7 hours ago, TehDwonz said:

GRC's DNSBench is a good way to find the lowest latency DNS to use, based on your location in cyberspace. Try to pick 3-4 different "providers" for your list.

 

Here's mine, set on the firewall so everything uses them:

 

DNSBench is here: https://www.grc.com/dns/benchmark.htm and this tool will also tell you if the provider redirects non-existent domains to adverts etc.

 

Dns latency for most is rather pointless. It only affects the time it takes for the name to Ip translation to happen. So even if we are talking about a 2000 ms ping we are talking 2 seconds. So it isn't even a big inconvenience in that case. 

 

I mean if you KNEW to look for it you might be able to notice a 30ms vs 2000ms delay from dns, but I think the vast majority wouldn't even know it was slow. Plus 2000 ms isn't even a real scenario just an example I am using.

[TehDwonz]

10 minutes ago, AngryBeaver said:

Dns latency for most is rather pointless. It only affects the time it takes for the name to Ip translation to happen. So even if we are talking about a 2000 ms ping we are talking 2 seconds. So it isn't even a big inconvenience in that case. 

 

I mean if you KNEW to look for it you might be able to notice a 30ms vs 2000ms delay from dns, but I think the vast majority wouldn't even know it was slow. Plus 2000 ms isn't even a real scenario just an example I am using.

 

You can't tell the difference between 30ms and 2 seconds? Really?

 

Did you see the part about redirects being flagged too?  

[AngryBeaver]

3 hours ago, TehDwonz said:

 

You can't tell the difference between 30ms and 2 seconds? Really?

 

Did you see the part about redirects being flagged too?  

My point is that it isn't super obvious because of how fast internet speeds are and once the connection is established dns delay isn't part of the equation.

 

https://www.dnsperf.com/

 

So the fastest average is 12ms

the slowest average is 132ms

 

so yes one is 11 times higher that the other... but we are talking about a 100th of a second compared to a 10th of a second. That, IMO, much quicker than most people would ever notice.

 

Now if the DNS speed is super slow (in the range of seconds) it is possible to have a webpage load and have ads and pictures that might be coming from other domains take that delay to show up... which can make things pop in or cause the page to load images and scroll on you, but again we aren't talking about a huge amount of time here.

[TehDwonz]

2 minutes ago, AngryBeaver said:

My point is that it isn't super obvious because of how fast internet speeds are and once the connection is established dns delay isn't part of the equation.

It was never really the issue, more a point of interest. It was about using multiple providers. I posted a way to find some reliable ones and/or geo-local ones, and also to flag any that redirect to ads if you make a typo and try to access a non-existent domain. In addition, the tool tests DNSSEC authentication. So for anyone wanting their own list of DNS providers to use, it's a good tool.

[AngryBeaver]

1 hour ago, TehDwonz said:

It was never really the issue, more a point of interest. It was about using multiple providers. I posted a way to find some reliable ones and/or geo-local ones, and also to flag any that redirect to ads if you make a typo and try to access a non-existent domain. In addition, the tool tests DNSSEC authentication. So for anyone wanting their own list of DNS providers to use, it's a good tool.

Pi hole with DOT or DOH for dns sec and filtering.  If you are trying to just protect your family for free then 

 

Filtering out malicious sites

1.1.1.2

1.0.0.2   

 

For filtering out adult content and malicious sites.

1.1.1.3

1.0.0.3

[Tellos]

surprise network outages happen. News at 11.

[StDragon]

On 7/18/2020 at 11:41 AM, jasonvp said:

Looks like CloudFlare published an RCA of sorts which basically backs up what I suspected: someone goofed.

 

 

Maintenance on a Friday afternoon.  Never, ever a good idea.  Heh.  What could possibly go wrong?!

 

Yeah, no kidding! 

 

Our local data center NOC thought it was a swell idea to perform UPS maintenance and generator testing without informing tenants (that was later corrected). What happened was when they performed a test, it caused a bank of UPS units to fail. This shifted the load to the other leg and thus a cascade failure when the entire data center was without power.

 

Dirty shutdowns on a SAN full of running VMs is never good. A few of them were so corrupted that I had to restore from backup. We spend a good portion the night getting the effected networks back up and running. I never did go home that night to get sleep. I literally worked 2x 8 hour shifts back to back.

 

And yeah, I'm still pissed about that!

 

This kind of failure should never happen. To make such sweeping changes of this magnitude should involve a team of engineers to cross the "T"s and dot the "I"s. Check, double-check, triple-check, sign off on final deliverable of change by management.

[Tony13555]

I had this, so I didn't notice. I mean my internet alarm (my kid) didnt notice. :D  

 

Static DNS 1 1.1.1.1
Static DNS 2 1.0.0.1
Static DNS 3 9.9.9.9