Spotty

Timeline of the outage: (times in UTC)
Starting at 02:03 on Saturday, requests started intermittently returning error 502, and many of the requests that were served successfully were significantly slower than normal At 05:59, most of the services on the server crashed. All subsequent requests were served by Cloudflare with error code 502 At about 11:00, I came online and attempted to diagnose the problem. Due to the previous failures I was unable to access the server, so all I was able to do was route traffic to an offline page At 16:35, with extra help, the server was forcibly restarted At 16:43 the server seemed to have come up successfully, so we enabled traffic and monitored the status By 16:53 it was clear that performance very poor and a significant fraction of the requests were resulting in error 502 (which means in this case that the server was already processing too many requests, so where were no workers available), so we disabled traffic again to investigate the situation further At 17:36, there was nothing clearly wrong so we tried enabling traffic again At 17:39 the performance was significantly regressing again so the site was turned offline again At 18:00 we ordered a new server At 18:35 the server was ready to be set up with all of the forum-related things, and for the data to be migrated to it At 21:28, the new server was fully set up and the forum was turned back online  
What were the symptoms?
IOWait accounted for the majority of CPU time, but IO utilisation was relatively normal In the syslog we were seeing a number of IO timeouts for the primary drive Prior to rebooting, the limited errors that we could see indicated that there had been disk corruption  
What was the root cause?
Although we aren't 100% sure, we think it's fairly likely that one of the two RAID 1 disks that form the primary disk had failed, and the poor performance was a consequence of trying to rebuild the array.
 
Why replace the server rather than the disk?
There were already plans to replace the server, this failure just accelerated them. We were also of the opinion that having the disk replaced and getting the array rebuilt would likely not end up being faster, especially as we did not have sufficient information to pinpoint the failure.

Well guys, we had a hick up for the day one stats and blog, posting in the wrong blog... Total error on my part.
 
We go through a rigorous validation procedure, most of it is automated, but the final validation is done by hand, which means there is a chance for human error. In this case one value was missed in the sheet of 1754 records, we record both point and WU data too. If you see anything not quite right feel free to drop any of us a DM.
 
any users that have not produced any points will not be seen here, if you think this is a mistake please also DM one of the team.
 
 
Please remember we fold as a team first and for personal glory second, give reactions to posts that have helped you and be kind.
 
Happy folding,
 
Spec.

The AOC G2460PF supports HDMI 1.4. I will now demonstrate it operating at 1920 × 1080 @ 120 Hz over HDMI. These tests are performed with an NVIDIA GeForce GTX 780 Ti, which also only supports HDMI 1.4.
 
Display Settings Demonstration
These settings show the G2460PF (EDID identifies itself as the "2460G4", Windows however does not read the name) connected via HDMI at 1920 × 1080 @ 120 Hz with full RGB color. A custom resolution was necessary to expose the 120 Hz option (CVT-RB timing was used, with a resulting pixel clock of 285 Mpx/s). Without custom resolutions, only options up to 60 Hz were available. Higher formats such as 144 Hz were also attempted, but failed. The monitor's HDMI port appears to support a maximum TMDS clock of approximately 300 MHz.
Timing Parameters and EDID
The EDID on this monitor reports a maximum of 170 Mpx/s, around the same as the maximum limit of SL-DVI or HDMI 1.2 (165 Mpx/s). However, in practice, the monitor's hardware works up to around 300 Mpx/s. Several custom resolutions were attempted. 1920 × 1080 @ 120 Hz worked with both CVT-RB timing (285 Mpx/s) and CTA-861 timing (297 Mpx/s), but anything above this point resulted in a black screen with a floating "Input Not Support" text. I attempted 1920 × 1080 @ 144 Hz at 317 Mpx/s without success, and even 138 Hz with a pixel rate of 304 Mpx/s (shown below) was rejected.

This monitor makes a good demonstration for two important points:
The maximum limit of an HDMI device can be any arbitrary limit that the manufacturer decides, or that the hardware is capable of. It is not simply "a device can support either HDMI 1.4 speed (340 Mpx/s) or be limited to HDMI 1.2 speed (165 Mpx/s)", or anything like that. The limitations can be anything, and may differ on every individual model. The limits listed in the EDID are simply values typed in by the manufacturer. The EDID does not have some method of magically detecting the actual hardware capabilities of the display. The EDID limits therefore do not necessarily represent the capabilities of the actual hardware. Verification
Of course, it is possible that the monitor is simply skipping frames, or failing to truly operate at 144 Hz in some other way. Some form of verification would be desirable.
Verification By Oscilloscope
This is measured using a Keysight EDUX1002A oscilloscope and a Texas Instruments TSL14S light-to-voltage converter. A pattern of alternating black and white frames was generated by the blurbusters flicker test (https://testufo.com/flicker). Since oscilloscopes are designed for measuring oscillating waveforms, a set of one white frame and one black frame is counted as a single "wave" (indicated by the two vertical orange lines marking the boundary of "one wave"). For this reason, the frequency displayed on the scope is half the actual refresh frequency, and the displayed period is twice the actual refresh period. In this case, 60.00 Hz indicates 60 sets of black-white transitions (2 frames) per second, for a total of 120.00 frames per second. This demonstrates flawless 120 Hz operation.
Verification By High-Speed Camera
This is a high-speed video of the blurbusters frame skipping test (https://testufo.com/frameskipping) shot with a Casio Exilim ZR100 at 1,000 FPS. Each frame of video represents 1 ms of real time. The video is played back at 30 FPS, meaning that every 1 second of video shows 30 ms of time. At 120 Hz, the display refreshes at intervals of 8.333 ms. This means that we should see slightly fewer than 4 refreshes per second of video, which the video does show. This can also be verified more precisely by examining the video frame by frame and counting 8–9 frames between each refresh. We can also observe from this video that the display is operating properly, without any frame skipping.
High-Speed Camera Complete Demonstration
Just for good measure, this video shows the display operating at 1920 × 1080 @ 120 Hz over HDMI with the frame skipping test in a single take at 1,000 FPS.