2 out of 6 GPUs aren't recognized by nvidia-smi

[Morgiana]

My specs :

mobo : asrock h170/pro4 hyper 

CPU : intel G4400

gpus: 6 x Gigabytes GTX 1070

psu: corsair 1500 watt

OS : ubuntu desktop 16.04

 

notes: this is a mining rig.

5 GPUs connected to mobo through x1 pcie risers, 6th GPU connect to Mobo through M2 slot. Risers and gpus aren't the problem because for any combination of 4 out of 6 cards. System operates as expected.

 

When all 6 cards are installed to mobo,

lspci | grep -i vga  reports all 6 cards with busID from 1 through 6, but only 4 are detected by nvidia-smi and operate. 

 

dmesg | grep -i  nvidia reports this for the 2 cards not detected by smi (busID either 4 and 5, 5 and 6, or 4 and 6):

 

NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:

 

So I thought my chipset doesn't have enough pcie lanes and the M2's x4 took all the lane, but when I look up in intel main page. Intel H170 chipset (my mobo's chipset) has 16 pcie lanes. My 5 gpus (x1), 2 of which use cpu's lanes (the x8 and x16 slot), so 3 + 4 (the M2 slot) = 7 lanes taken from chipset's 16 pcie lanes. Therefore, my mobo should be able to handle 6 gpus, right? How do I get it going??

[thorsong]

 Even if you technically have the lanes, the bios can still block you from using more than xnumber of gpus. I have a pro4s for one of my miners. It will only use 4 gpu's max. Asrock's website confirms this as Pro4 boards support quad-sli and quad-crossfire. Even though you are not using sli or crossfire, in my experience this also translates to support for 4 GPU's total. I do not believe there is a workaround for this. If anyone else has found one, I would love to know of it myself.

[Morgiana]

4 hours ago, thorsong said:

 Even if you technically have the lanes, the bios can still block you from using more than xnumber of gpus. I have a pro4s for one of my miners. It will only use 4 gpu's max. Asrock's website confirms this as Pro4 boards support quad-sli and quad-crossfire. Even though you are not using sli or crossfire, in my experience this also translates to support for 4 GPU's total. I do not believe there is a workaround for this. If anyone else has found one, I would love to know of it myself.

Oh shit damn them  my h170/pro4 hyper doesn't even support SLI though.