Say NO to the Paywall of PCI-E Bifurcation

[grayxu]

The Paywall of PCI-E Bifurcation

PCI-E provides CPUs the ability to connect with fast devices, like >10 Gbps networks, NVMe SSD, etc. Especially for the usage of workstations or servers, the number PCI-E slots means a lot. As a result, Xeon CPUs and server-level MBs provide more PCI-E slots than on the consumer level. On the other hand, the number of PCI-E slots on cheap MB is usually only 1 for GPU. However, those server-level MBs are more expensive.

PCI-E bifurcation is an approach to split x16 PCI-E channels into different combinations, like 8+8 or 8+4+4. Since some PCI-E devices are only in x4 or x8 speed. But unlike AMD, Intel consumer-level MBs rarely offer such an option in the BIOS, let alone the lower-end products.

Note: PLX chips can also help you like a switch, but they are also expensive.

Bypass the paywall

Let’s do some hacking to bypass such paywall. My platform is Intel Core i5-3450 + Z77, but I guess this method is also useful for any Intel CPU with public datasheet information, like 12th gen CPU[3].

First, go to the Intel ark document webpage to check the “PCI Express Configurations”, which will show all bifurcation modes that your CPU supports.

For my CPU, 16=8+8=8+4+4, which provides the opportunity to place one x8 HBA card and 2 x4 NVMe SSD in one PCI-E slot! 

 

 

Second, get the datasheet of your CPU, and search for bifurcation-related content. You will find the config (CFG) to control PCI-E bifurcation.

note: the datasheet contains infos like the definitions of LGA1155, so that we can control it manaully.

 

 

As the fig above, the value of CFG[6:5] controls bifurcation modes on my platform, and:

• 00: 8+4+4
• 10: 8+8
• 11: 16

note: CFG[17:0] means 18 pins of LGA1155.

And then we search the positions of CFG pins to hard-set their values as we wish.

 

It shows that CFG[5] is N35, and CFG[6] is L37. So that we can find the actual physical position of CFG[5] and CFG[6].

 

 

As mentioned above, I want to set 16=8+4+4, so the values of CFG[5] and CFG[6] should be both 0. So we can ground CFG[5] and CFG[6] by connecting N35<->M35 and L37<->M37. (VSS is GND, and VCC is PWR)

I used some silver paste to make such hard strap.

 

 

 

Last, get a cheap PCI-E bifurcation card with clock MUX, like multiple NVMe SSD in one card.

 

my config places one LSI2308 with other NVMe SSDs in one PCI-E card. 

 

 

Now you can enjoy multiple devices with fast connections!

 

 

 

Small modification, big impact. So why MB producers disable this feature?

 

FYI: I tried to use AMIBCP tools to modify potentially hidden options in BIOS firmware, but sadly found nothing

 

[grayxu]

note: I am not the first person to do this, such discussions have happened before on some Chinese forums. But curiously, I didn't find any related article in English, so I wrote this to document the process as well.

 

ref:

https://zhuanlan.zhihu.com/p/70125574

https://www.bilibili.com/read/cv15596863/

https://www.bilibili.com/read/cv16530665/

[manikyath]

calling this "the paywall of bifurcation" is...  at least a bit excessive.

 

every motherboard i own is at least 8+8 capable. the reason why most consumer boards these days only do x16 is quite simple:

- the majority of people doesnt use more than one slot (for their GPU) and one or two M.2 slots, which these days dont tap into those x16 GPU lanes.

- the majority of other PCIe slot devices are 1x, which means that past the 3 slots we keep for the GPU these days, everything else can get a 1x slot from the chipset without tapping into the GPU's lanes, and still have bandwidth for an NVME SSD.

 

the market of people who want to have a desktop platform with 8x, 4x, 4x, 4x, 4x (assuming 20 cpu lanes and 4 chipset lanes) is so small there's literally douzens of us. And economy of scale tells us that if you want a board with a market of douzens of people, it will be more expensive.

 

also, where the FLIP did you find a Z77 board without crossfire support?

[TetraSky]

10 minutes ago, manikyath said:

calling this "the paywall of bifurcation" is...  at least a bit excessive.

 

every motherboard i own is at least 8+8 capable. the reason why most consumer boards these days only do x16 is quite simple:

- the majority of people doesnt use more than one slot (for their GPU) and one or two M.2 slots, which these days dont tap into those x16 GPU lanes.

- the majority of other PCIe slot devices are 1x, which means that past the 3 slots we keep for the GPU these days, everything else can get a 1x slot from the chipset without tapping into the GPU's lanes, and still have bandwidth for an NVME SSD.

 

the market of people who want to have a desktop platform with 8x, 4x, 4x, 4x, 4x (assuming 20 cpu lanes and 4 chipset lanes) is so small there's literally douzens of us. And economy of scale tells us that if you want a board with a market of douzens of people, it will be more expensive.

 

also, where the FLIP did you find a Z77 board without crossfire support?

I might be reading this wrong, but I think this is something different from just having PCI-e slots that are capable of certain speeds.
And more like taking a single slot and making it be 8+4+4 by itself while leaving the others alone, to have 3 different devices plugged in that one slot and be recognized individually ?
Whichever the case, I don't personally have any need for this.

[manikyath]

1 minute ago, TetraSky said:

I might be reading this wrong, but I think this is something different from just having PCI-e slots that are capable of certain speeds.
And more like taking a single slot and making it be 8+4+4 by itself while leaving the others alone, to have 3 different devices plugged in that one slot and be recognized individually ?

from what i can make of it, OP somehow found a Z77 board that only has a single full length slot, and is upset that the bios doesnt magically let him break up that slot into the options that the cpu could theoretically output.

[LAwLz]

OP can you please elaborate what you are trying to say?

It seems like everyone is confused. What is the practical end result of this guide? What paywall are you talking about? How does AMD and Intel differ in this regard?

[grayxu]

1 hour ago, manikyath said:

from what i can make of it, OP somehow found a Z77 board that only has a single full length slot, and is upset that the bios doesnt magically let him break up that slot into the options that the cpu could theoretically output.

Z77 is only my case.

For 12 gen CPU, 8+8 bifurcation in single PCI-E slot is only provided on high-end Z690+DDR5. With hard strap, you can get same feature in any Z690 board.

1 hour ago, TetraSky said:

I might be reading this wrong, but I think this is something different from just having PCI-e slots that are capable of certain speeds.
And more like taking a single slot and making it be 8+4+4 by itself while leaving the others alone, to have 3 different devices plugged in that one slot and be recognized individually ?
Whichever the case, I don't personally have any need for this.

Yeah, such scenarios are not for gamers like I mentioned in the first paragraph.

This tutorial is for those building workstation or servers on cheap MBs.

[grayxu]

34 minutes ago, LAwLz said:

OP can you please elaborate what you are trying to say?

It seems like everyone is confused. What is the practical end result of this guide? What paywall are you talking about? How does AMD and Intel differ in this regard?

"practical end result": You can get PCI-E bifurcation feature in low-end MBs.

"paywall": PCI-E bifurcation is only available on Intel high-end MBs, however, it is enabled on almost all AMD MBs.

[LAwLz]

27 minutes ago, grayxu said:

"practical end result": You can get PCI-E bifurcation feature in low-end MBs.

"paywall": PCI-E bifurcation feature in Intel high-end MBs, which is enabled for almost every AMD MBs.

Again, can you elaborate?

 

What exactly do you mean by "PCI-E bifurcation"? I assume you mean splitting for example one PCIe x16 port into two PCIe x8 ports.

I looked up some lower end Z77 boards and found that for example the Gigabyte GA-Z77-D3H has two full size x16 slots, but one runs at x16 speeds and the other runs at x4 speeds. It is not possible to make them both run at x8. Is that what you are saying you have gotten around by shortening some pins on the CPU?

 

I also looked up some lower end AMD motherboards such as the Gigabyte GA-AX370-Gaming 3 and the spec sheet there says the exact same thing.

One x16 slot running at x16.

One x16 slot running at x4.

 

No mention of running both at x8.

So I don't really think this is an Intel-exclusive thing that doesn't exist on AMD motherboards. 

 

 

Are you sure that it is a software lock and not a hardware limit?

I am fairly sure this is a cost-saving measurement done by motherboard manufacturers. That they only physically make traces in the motherboard for an x4 connection to the second PCIe slot.

 

Maybe I am complete off base here and you are right that motherboard manufacturers are locking this with software and you have found a way around it, but as it stands right now I kind of doubt it to be honest. 

Without looking into it too much, I wouldn't be surprised if you shortening the pins on your CPU just results in your CPU being tricked into thinking it has two x8 connections, even though the second port still only has an x4 connection. If that is the case, you might get a lot of corrupt data if you start pushing the second "x8" connection so that it tries to use all 8 lanes.

[TetraSky]

44 minutes ago, grayxu said:

"practical end result": You can get PCI-E bifurcation feature in low-end MBs.

"paywall": PCI-E bifurcation is only available on Intel high-end MBs, however, it is enabled on almost all AMD MBs.

Ah I see, I get it now. So it's just Intel being Intel as usual. Just like how they lock down overclocking to specific CPU models and I believe specific chipset as well (unless that changed, haven't kept up with Intel).

[grayxu]

18 minutes ago, LAwLz said:

Again, can you elaborate?

 

What exactly do you mean by "PCI-E bifurcation"? I assume you mean splitting for example one PCIe x16 port into two PCIe x8 ports.

I looked up some lower end Z77 boards and found that for example the Gigabyte GA-Z77-D3H has two full size x16 slots, but one runs at x16 speeds and the other runs at x4 speeds. It is not possible to make them both run at x8. Is that what you are saying you have gotten around by shortening some pins on the CPU?

 

I also looked up some lower end AMD motherboards such as the Gigabyte GA-AX370-Gaming 3 and the spec sheet there says the exact same thing.

One x16 slot running at x16.

One x16 slot running at x4.

 

No mention of running both at x8.

So I don't really think this is an Intel-exclusive thing that doesn't exist on AMD motherboards. 

 

 

Are you sure that it is a software lock and not a hardware limit?

I am fairly sure this is a cost-saving measurement done by motherboard manufacturers. That they only physically make traces in the motherboard for an x4 connection to the second PCIe slot.

 

Maybe I am complete off base here and you are right that motherboard manufacturers are locking this with software and you have found a way around it, but as it stands right now I kind of doubt it to be honest. 

Without looking into it too much, I wouldn't be surprised if you shortening the pins on your CPU just results in your CPU being tricked into thinking it has two x8 connections, even though the second port still only has an x4 connection. If that is the case, you might get a lot of corrupt data if you start pushing the second "x8" connection so that it tries to use all 8 lanes.

Well, I guess you just don't get the definition "PCI-E Bifurcation". 

 

exapmle 1: only one x16 PCI-E slot

before: only one PCI-E devices in this slot.

now: you can attach three devices (x8+x4+x4) in one x16 PCI-E slot

 

example 2: 2 x16 PCI-E slot (not PCH)

before: only x8+x8

now: with a x8 device in the first slot, you can install two x4 devices in the second x16 slot

 

You should check https://www.10gtek.com/new-1414.

[tkitch]

Here's how you avoid needing to do all of this hardware hacking:

If you're enough of a power-user to actually need the 8x8 setup, make sure you buy hardware that supports it out of the box, rather than by voiding warranties and potentially killing hardware with sketchy modifications.  

 

Because amongst other things:

Just because your board has an x16 slot doesn't mean it's wired for x16.

 

My most recent B550 purchase was stacked with 5x x16 slots.  Only 1 of which is wired for 16, the rest are x1 or x4.  Without the physical wiring, you can't make it run at 8x8.

[grayxu]

16 minutes ago, tkitch said:

Here's how you avoid needing to do all of this hardware hacking:

If you're enough of a power-user to actually need the 8x8 setup, make sure you buy hardware that supports it out of the box, rather than by voiding warranties and potentially killing hardware with sketchy modifications.  

I have to say I agree with this. That's also the reason why I "hacking" an old CPU instead of those for production.

Still, I want to say that this feature is easy to enable for MB makers on any platform, but they don't. It's called a "paywall".

 

16 minutes ago, tkitch said:

Just because your board has an x16 slot doesn't mean it's wired for x16.

 

My most recent B550 purchase was stacked with 5x x16 slots.  Only 1 of which is wired for 16, the rest are x1 or x4.  Without the physical wiring, you can't make it run at 8x8.

but it seems like you also don't get the definition "PCI-E Bifurcation"... This has NOTHING to do with HW cabling.

Edited October 24, 2022 by grayxu

[tkitch]

10 minutes ago, grayxu said:

but it seems like you also don't get the definition "PCI-E Bifurcation"... This has NOTHING to do with HW cabling.

Incorrect:

You can't push x8 lanes through a slot that only has wires for x1 or x4.  It physically doesn't work.  

 

See the reverse on the RX-6500

It's an x16 slot on the card, but it only has wires connected for x4 to the pins.  The rest aren't even connected, so no matter what you "hack" it can't do x8.  

[LAwLz]

23 minutes ago, grayxu said:

Well, I guess you just don't get the definition "PCI-E Bifurcation". 

 

exapmle 1: only one x16 PCI-E slot

before: only one PCI-E devices in this slot.

now: you can attach three devices (x8+x4+x4) in one x16 PCI-E slot

 

example 2: 2 x16 PCI-E slot (not PCH)

before: only x8+x8

now: with a x8 device in the first slot, you can install two x4 devices in the second x16 slot

 

You should check https://www.10gtek.com/new-1414.

I see.

 

So you are running multiple devices through a single PCIe x16 slot. You were not talking about distributing lanes across multiple lanes, but rather run multiple devices through a single slot.

 

Didn't know that was a thing. Thanks for the info.

 

 

11 minutes ago, grayxu said:

Still, I want to say that this feature is easy to enable for MB makers on any platform, but they don't. It's called a "paywall".

 

I don't really agree with your definition of a paywall.

That word is usually used for things line online content, where you get blocked from using/viewing something until you pay up. In this scenario, you apparently don't even get the option to pay them to unlock the feature. It's not really a "paywall" if you can't pay them to pass the wall.

[grayxu]

16 hours ago, LAwLz said:

I don't really agree with your definition of a paywall.

That word is usually used for things line online content, where you get blocked from using/viewing something until you pay up. In this scenario, you apparently don't even get the option to pay them to unlock the feature. It's not really a "paywall" if you can't pay them to pass the wall.

as strict definition? maybe. But:

 

I LOVE Paywalls. Thanks Intel! - ECC Support on Alder Lake

 

[Dunn]

On 10/24/2022 at 1:12 PM, grayxu said:

But unlike AMD, Intel consumer-level MBs rarely offer such an option in the BIOS, let alone the lower-end products.

Huh? Last I checked both AMD and Intel mobos had this feature only in the expensive models. I couldn't find any cheap AMD mobo that supports it. It's a feature that depends on a hardware implementation, it's not just artificially limited.

 

 

Oh and this forum is mostly into gaming and discussing anything not related to gaming will get you blank stares.

That's actually the case for most computer forums, I'm still looking for one that isn't like that.

[Holmes108]

My AMD Gigabyte B550 D3SH has it, and I think it's literally the cheapest MB you can buy with that chipset. But maybe that's the key, 550 vs 450, etc..

[toasty99]

Now, I don't have the hardware to test this but I can think of a use case for this: 

 

i9 12900k engineering samples. Those have the x16 slot disabled. I wonder if some type of hack like this could work on them with some of the boards to get at least x8 in some slot working. I don't know, but hey if anyone happens to have one it would be cool to see you try! 

[Dunn]

4 hours ago, Holmes108 said:

My AMD Gigabyte B550 D3SH has it, and I think it's literally the cheapest MB you can buy with that chipset. But maybe that's the key, 550 vs 450, etc..

Oh you mean bifurcation of the one x16 PCIe slot. Yeah, that's possible.

But bifurcation between two (or more) slots is for expensive mobos only. On B550 that would mean PCIe 4.0 lanes from the CPU to another slot.

[Blue4130]

On 10/24/2022 at 11:18 PM, LAwLz said:

I see.

 

So you are running multiple devices through a single PCIe x16 slot. You were not talking about distributing lanes across multiple lanes, but rather run multiple devices through a single slot.

 

 

It's pretty much only a server thing. Many 1u and 2u servers use risers with two slots that plug into a single x16 slot. I've never seen it in a gaming machine. 

[Dunn]

The B550M DS3H mentioned before does support it, according to the manual. And probably many others in the same category do.

I think this is more relevant and useful now, with NVMe drives being the standard. And considering that some new (AMD) video cards only work at x4 or x8 (PCIe 4.0), giving them more lanes is wasteful.

Still, bifurcating across multiple slots on the mobo is more practical. That riser-splitter looks nice, but it's hard to fit in a typical case.

 

[grayxu]

30 minutes ago, Dunn said:

Still, bifurcating across multiple slots on the mobo is more practical. That riser-splitter looks nice, but it's hard to fit in a typical case.

 

Indeed, PCI-E riser cards are designed for servers on racks.

But, you can use vertical GPU kits to "rise".

 

This guy installed his GPU and 2 NVMe SSDs on a single PCI-E x16 slot with 12700k.

[xinww]

can coreboot/tianocore set it in software thx

[distorter]

Have been searching for this info for sometime, thanks for the post. I knew my CPU supported PCI-E Bifurcation (Skylake i7/6700) but my motherboard(Q170m) didn't.

 

@grayxu The grounding pins are quite far from the CFG 5 and 6 points for the 6700. So I assume you would need to do this on the socket instead with a tiny bit of wire?