Mining Expansion Risers

[pjsmith727]

Can someone please explain to me the concept of these 1x to 16x expansion/riser pci-e cards. You plug in a 1x card and over usb connection with supplementary power, usually sata or molex, like magic you have a 16x slot to use. This all makes sense to me, the part I am caught up on is when you are mining with a multi gpu setup, lets say 6 cards, do nvidia cards still not require a minimum of 8 lanes with windows and AMD cards a minimum of 4? does this not mean that best case scenario with 6 AMD cards on lets say a second hand q6600 you still require a minimum of 24 pci-e lanes when the cpu only provides 16 plus maybe 4 on the chipset? Or is it the case that most miners with setups like this are running linux or windows server edition which would have some sort of work around. I haven't been able to find any articles or videos on this only small explanations of what these 1x to 16x expansion cards do, which to me was pretty self explanatory. 

[Lurick]

Those lane requirements are for SLI and Crossfire respectively. Cards not in SLI or Crossfire can easily run at 1x speeds without issue.

[mariushm]

Those risers don't create additional pci-e lanes.

just the physical shape of slot changes to x16, electrically slot is still x1

The riser only has a small dc converter to produce 3.3v from the 12v you provide through the power connector, there's no power transferred through that usb cable, only the data (the x1 lane)

.

 

[pjsmith727]

3 minutes ago, mariushm said:

Those risers don't create additional pci-e lanes.

just the physical shape of slot changes to x16, electrically slot is still x1

ok so that i all understood, so how do you translate that to more than 4 gpus on a 16 lane cpu? Is this a function of linux or windows server?

[mariushm]

You don't.

You have 16 pci-e lanes which come from cpu and can often be configured as either 1 x16 or 2 x8 (when you plug 2 cards in the "special" x16 slots they

are automatically reconfigured as x8 electrically)  * AM4 socket boards with  b350 chipset can't split x16 into 2x8, x370 chipset can, that's why most b350 based boards have only 1x16 slot. 

The rest of the slots are created using lanes coming from chipset.

For example in case of socket AM4 the B350 chipset can create 6 pci-e lanes and the x370 can make 8 lanes.

The motherboard maker can add pci-e x16 slot physically, x4 electrically, can add several x1 slots or can route x1,x2 or x4 intpo extra m.2 slot or

can use x1 lanes in the motherboard for extra usb 3.1 controllers, 10 gbps lan, extra sata etc

 

* socket am4 has 24 lanes , 16 (2x8) to graphics, 1x4 to m.2 (could use adapter to create pci-e x4 slot / riser), x4 goes to chipset)

* threadripper has 64 lanes, 4 go to chipset and the rest can be arranged by mb maker into m.2 slots and pci-e slots (if i remember correctly a x16 segment can be divided up to 5 slots, mix of x1, x4 and x8 slots)

* epyc processors have 128 lanes, with 4 reserved for chipset link. 

* some high end intel cpus have more than 16 lanes coming from cpu , i think common values are 28 and 44 lanes or something like that.

 

edit: corrected threadripper and epyc pci-e lane count.

[mariushm]

40 minutes ago, pjsmith727 said:

ok so that i all understood, so how do you translate that to more than 4 gpus on a 16 lane cpu? Is this a function of linux or windows server?

Those x16 slots that use pci-e lanes directly from cpu get preferential treatment , they're better for gaming, less latency, faster speeds in general.

The video cards connected to the other slots pass the data to the chipset and the chipset mixes the data with the data from the sata controllers, usb controllers and whatever else the chipset has to offer and sends this to the cpu, cpu processes the data and talks back to the chipset which then forwards the data to the slots it creates.

The chipset basically acts like a network switch ... a chipset is like a network switch that has let's say a 1 gbps port that you use to connect the switch to a router or the internet, and maybe 16 or 24 100 mbps ports to which various things are attached ( 1 port for each pair of sata connectors, 1 port for each usb 3.0, 1 port for each set of 4 usb 2.0, 1 port for each pci-e lane it can create).

In the case of the AM4 motherboards from AMD,  the chipset can talk to the processor at pci-e x4 speeds, which basically means a maximum of 4 GB/s in either direction. The chipset then creates 6 or 8 pci-e v2.0 lanes, each such lane being capable of 500 MB/s in either direction. Also, the usb 3.0 controller inside the chipset uses internally something similar to a pci-e lane , as is the sata controller. 

So for example, let's say you plug a video capture card into a pci-e x1 slot and this capture card captures some content in real time with 100 MB/s and at the same time, you're reading some from a SSD with 300 MB/s and you're copying some stuff from a USB stick with 50 MB/s

 

The chipset takes 100 MB from pci-e x1 slot it created  + 300 MB/s from SSD + 50 MB from usb stick and creates a single stream of data that goes to the CPU at a speed of 450 MB/s, using a tenth of the 4 GB/s link so everything is smooth.

When you're mining, you're not transferring or receiving from the video card a lot of data, think of it like sending to the video card a constant flow of data that's maybe 1 MB/s ... the pci-e x1 lane with its 500 MB/s capability is more than enough to mine.

 

However, when you're playing games, the game engine has to upload and exchange data with the video card very fast and often in big bursts, so that's why it's not great to have something in between like a chipset which adds a few nanoseconds or microseconds of delays (the time the chipset needs to mix or split the data and distribute it to various slots or internal devices like sata controllers, usb controllers and so on)... for games also having a lot of lanes helps increase the framerate by transferring small amounts of information faster.  With 1 pci-e v2.0 lane, let's say you have 100 MB of information you want to upload into the video card... you'd upload it in 100 MB / 500 MB/s = 0.2s ... but if you have a pci-e x16 v3.0 with 1 GB/s per lane, you use 16 lanes in parallel, so those 100 MB are uploaded in the video card in 100 MB / 1000 MB/s / 16 lanes = 0.00625 seconds or around 32 times faster.

 

 

 

[pjsmith727]

On 12/26/2017 at 8:53 AM, mariushm said:

Those x16 slots that use pci-e lanes directly from cpu get preferential treatment , they're better for gaming, less latency, faster speeds in general.

The video cards connected to the other slots pass the data to the chipset and the chipset mixes the data with the data from the sata controllers, usb controllers and whatever else the chipset has to offer and sends this to the cpu, cpu processes the data and talks back to the chipset which then forwards the data to the slots it creates.

The chipset basically acts like a network switch ... a chipset is like a network switch that has let's say a 1 gbps port that you use to connect the switch to a router or the internet, and maybe 16 or 24 100 mbps ports to which various things are attached ( 1 port for each pair of sata connectors, 1 port for each usb 3.0, 1 port for each set of 4 usb 2.0, 1 port for each pci-e lane it can create).

In the case of the AM4 motherboards from AMD,  the chipset can talk to the processor at pci-e x4 speeds, which basically means a maximum of 4 GB/s in either direction. The chipset then creates 6 or 8 pci-e v2.0 lanes, each such lane being capable of 500 MB/s in either direction. Also, the usb 3.0 controller inside the chipset uses internally something similar to a pci-e lane , as is the sata controller. 

So for example, let's say you plug a video capture card into a pci-e x1 slot and this capture card captures some content in real time with 100 MB/s and at the same time, you're reading some from a SSD with 300 MB/s and you're copying some stuff from a USB stick with 50 MB/s

 

The chipset takes 100 MB from pci-e x1 slot it created  + 300 MB/s from SSD + 50 MB from usb stick and creates a single stream of data that goes to the CPU at a speed of 450 MB/s, using a tenth of the 4 GB/s link so everything is smooth.

When you're mining, you're not transferring or receiving from the video card a lot of data, think of it like sending to the video card a constant flow of data that's maybe 1 MB/s ... the pci-e x1 lane with its 500 MB/s capability is more than enough to mine.

 

However, when you're playing games, the game engine has to upload and exchange data with the video card very fast and often in big bursts, so that's why it's not great to have something in between like a chipset which adds a few nanoseconds or microseconds of delays (the time the chipset needs to mix or split the data and distribute it to various slots or internal devices like sata controllers, usb controllers and so on)... for games also having a lot of lanes helps increase the framerate by transferring small amounts of information faster.  With 1 pci-e v2.0 lane, let's say you have 100 MB of information you want to upload into the video card... you'd upload it in 100 MB / 500 MB/s = 0.2s ... but if you have a pci-e x16 v3.0 with 1 GB/s per lane, you use 16 lanes in parallel, so those 100 MB are uploaded in the video card in 100 MB / 1000 MB/s / 16 lanes = 0.00625 seconds or around 32 times faster.

 

 

 

thanks dude, super helpful. I will pass this information along