Intel VCA (Visual Compute Accelerator)

Ryphon · September 24, 2023

Hi all,

I recently acquired a couple servers from a company doing some recycling of their old hardware. They'd left in some quite interesting Intel Visual Compute Accelerator cards, and I'm wondering if anyone on these forums has experience with them and can share any information on top of what Intel offers. From my reading, these are PCIe cards that are primarily meant for video transcoding. Each card has 3x E3-1200L V4 series processors, 3x two sticks of 8GB DDR3 SDRAM, and 3x Intel Iris Pro P6300. From the docs I've found, they require a pretty old host OS (CentOS 7.4, Ubuntu 16.04, or Debian 7), and they run a similar level of OS. They have levels of support for doing networking via PCIe, and can be ssh'd into. They can use local storage or NFS as a base OS mount and the VCA2 (2nd gen) cards actually had a built in M2 adapter to be able to have 'local' storage.

I've spent the better part of 2 days trying to get them to 'work', or even prove that they are working, but I've been trying to do it "my" way, which is seeming to turn out to be a bit of a mistake. I should just be following the guides, but man these servers are deafeningly loud, and when the fans spin up to cool down these cards (notice no internal fans), it's tough to even think. My current attempts focused mostly around using some level of Hypervisor, be that ESXi or Proxmox to try to set up a modern OS, and then set up the supported os, and try to pass the PCIe slot through. The architecture of these cards is interesting, as it's using a mix of PCIe switches, so I can't just pass it through. I was hoping to avoid putting a 4+ year old OS on a server that would be up 24/7 that I kinda wanted to use for more than just this.

My next step is just to suck it up and get Ubuntu 16.04 on the base server, and see if I can get the cards working just using the straight up documentation/files/drives that I was actually able to find. If I could ssh into one of the systems on one of the cpus on one of the cards, I'd consider it a success, because then at least I could try the memes and get k3s going just to prove that I can.

The only non Intel info that I've found was Der8auer pulling one apart and talking about it, and a post on STH talking about how they were going to try to run a k8s cluster on one of them. To be fair, that's definitely what I wanted to try doing, but the STH post just ended with "Stay tuned to STH for more on this novel platform." and never a peep later.

I know I can't be the only one messing with these that isn't bound by an NDA, right? I also know I can't be the only one messing with one of these, that doesn't want to be running Ubuntu 16.04 or similar year of OS, right?

Ryphon · September 25, 2023

Replying to update whoever is interested.

CentOS 7.9 as host indeed works, even though docs specify 7.4. Seems the images straight from Intel are still 7.4 (or 16.04 for Ubuntu), so doing a nice yum update/upgrade helps at least a little. I've been able to successfully run 1 card with all 3 'systems' online and ssh-able. In some initial experiments to run K3s, it seems there's still some firewall issues at play (likely firewalld, which I probably didn't actually disable yet, duh). Working on adding in a few more cards, have to do some BIOS/EEPROM updates, which is slow going with the jet engine of a 1u system. End goal is probably 3 cards running a 9 node k8s cluster. The silly part is, these 1u boxes that I picked up have dual E5-2697v3 cpus, which are already 14 core cpus. So 2 of these PCIe cards out-thread count the nodes CPUs themselves hah.

Next steps are to change to a persistent node image on BlockIO rather than volatile to connect and become a proper k3s node for a time, and then to get the persistence to work over NFS (even though I'm sure that speed is going to _suffer_ for that.) Once that's at least mostly successful, I might see if I can get an Ubuntu 16.04 image running, and try doing a do-release-upgrade and see what happens if I upgrade to 18.x. I may at least have a way to run something not as ancient (and something that can handle Sunshine )

Longer term, might see if I can get a Win10 installation running, and get some RDP going potentially. Windows is a whole can of worms that I'm not yet ready to dig into, but may end up being the solution if I'm limited to CentOS7.4 and Ubuntu 16.04.