Mellanox IS5023 40Gbit link stays down

Aelita Sophie · May 27, 2018

Hi there!

Recently I bought an entire lot of servers and gear for a very very good price.

This lot contained Mellanox MHQH19B-XTR cards, necessary QSFP cables and 3x Mellanox IS5023 18-port InfiniBand Switches.

Now to my understanding, in general these switches are unmanaged. But they do have manage capabilities. If I'm right it has a WebGUI or atleast a programmable I²C interface through RJ45.

My problem:

Server A and Server B both have a QSFP-card installed. Both detected. Connecting them directly with a QSFP cable, the link goes up and signal goes through. So cards and cables are working fine.

Connecting Server A through a InfiniBand Switch to Server B, nothing happens. No light turns on on the cards, nor on the switch. The switch seems to either ignore the connection or I might be doing something wrong? From my understanding it's plug and play.

I figured it might be a faulty switch. So I tried the other 2 as well. Same results. Tried connecting the switches together. Switch 1 and 2 worked for a moment, but when I disconnected them to test the other switch out, they havent worked since. (Or atleast it seemed to have a connection, as light showed up on the switch-port.)

There is a possibility that these switches might be programmed, but there is no "Reset" button that I know of. But somehow I doubt that it is programmed. Am I doing something wrong here?

If I were to program it, how would I program through the RJ45 I²C port? The proprietary cable they USED to sell were $600, which is nothing more then a RJ45 to Serial, From Serial to a I²C to USB from my understanding. But I don't have any availability it seems within the EU.

Windows7ge · May 27, 2018

It's possible due to the very high-end nature of the device and its intended application that by default the interfaces are shutdown. Possibly to make installation faster so as to only no shut the interfaces that are needed. This is common with CISCO routers but switches are usually un-managed all ports enabled until you manage them. Perhaps this switch is partial Layer 3. You won't know until you get into the console. If the switch has a WebUI you'll have to find either its IP address or the software required to view it if applicable. If all you need is its IP then you might only be able to get to it by first going through the console seeing as how there's no non-console RJ-45 port and we don't know if it's setup with DHCP. Even if it is you'd be going into your router trying to find what IP it was given. I know with network hardware like this manuals are just non-existent you're expected to know everything about the device before hand. Googling the switch every port is in the front which tells me this is meant for server racks where the back of the rack isn't very accessible (not really useful information but that's what I found). You'll have to source a console cable. You'll just have to wait for shipping. Even if you can't get the additional adapter all you need is a motherboard with a built in RS232 port. Some laptops even come with it. I'd start with Ebay. You might have to pay a little extra for shipping if it's not a good deal for the seller but it won't cost you anywhere near $600

Here's one. - $4.25

They even have Serial USB to RJ-45 - $4.54 (I actually never knew a all-in-one adapter was sold. I might buy myself one or two of these.)

Aelita Sophie · May 27, 2018

2 hours ago, Windows7ge said:

It's possible due to the very high-end nature of the device and its intended application that by default the interfaces are shutdown. Possibly to make installation faster so as to only no shut the interfaces that are needed. This is common with CISCO routers but switches are usually un-managed all ports enabled until you manage them. Perhaps this switch is partial Layer 3. You won't know until you get into the console. If the switch has a WebUI you'll have to find either its IP address or the software required to view it if applicable. If all you need is its IP then you might only be able to get to it by first going through the console seeing as how there's no non-console RJ-45 port and we don't know if it's setup with DHCP. Even if it is you'd be going into your router trying to find what IP it was given. I know with network hardware like this manuals are just non-existent you're expected to know everything about the device before hand. Googling the switch every port is in the front which tells me this is meant for server racks where the back of the rack isn't very accessible (not really useful information but that's what I found). You'll have to source a console cable. You'll just have to wait for shipping. Even if you can't get the additional adapter all you need is a motherboard with a built in RS232 port. Some laptops even come with it. I'd start with Ebay. You might have to pay a little extra for shipping if it's not a good deal for the seller but it won't cost you anywhere near $600

Here's one. - $4.25

They even have Serial USB to RJ-45 - $4.54 (I actually never knew a all-in-one adapter was sold. I might buy myself one or two of these.)

Oh you are a life saver with those links! I was really bummed out with that $600 mellanox-cable price! Thanks, I'll try that!