Where Intel is in REAL Trouble...

[Kilrah]

Becasue server/enterprise grade reliability specs.

[ny0c]

Why not try SnapRAID with MergerFS?

[4CSmith]

Especially as of PCIe Gen 3/4, there are extensive per-lane stats available within the endpoints. It's possible that due to extended cabling and cross-talk within the cable bundles that the bit error-rate (BER) is exceeding a threshold that triggers the bus reset.

 

Also are the Gigabyte CNV3024 riser cards PCIe retimers or only simple redrivers? It would appear that four of the drives are potentially directly connected to CPU PCIe lanes and those are long paths to the front of the chassis. Signal integrity becomes more challenging with PCIe Gen 3 and 4 rates.

[Fr8Train]

Rewatching these vids.. I seem to remember that the actual issue was found later to be a bad raid card, or backplane.  Does anyone remember this?  A link to the vid would be appreciated

 

[danwat1234]

On 3/18/2023 at 8:41 PM, Fr8Train said:

Rewatching these vids.. I seem to remember that the actual issue was found later to be a bad raid card, or backplane.  Does anyone remember this?  A link to the vid would be appreciated

 

Use polling mode, somehow
4:58 in; 

Good engineering. In my experience with our AMD-based NVMe storage server, I learned that AMD has a whole server/data center

platform out there that, as far as I can tell, no one ever thought to go, hey, what happens if I light up all these PCIe lanes with data,

and then actually load up the CPU at the same time? Turns out that the data gets retrieved so fast from that big flash array that,

by the time the CPU comes back around to get it, it's gone.

Ruh-roh.

Wendell made an offhand comment to me that really stood out while we were troubleshooting the issue. I said to him, man, I never saw any of these kinds of stupid issues when we rolled our Intel all in DME storage server, and he goes, "Yeah, Intel's got fewer PCIe lanes, "and they're a generation behind, "but they put some really good engineering into them."

See also