UnRAID + Ryzen

[Bureaucromancer]

Just checking in to see if anybody is seriously contemplating an attempt at an UnRAID VM/Gaming NAS type thing based around Ryzen fairly soon after launch?

 

At the moment I'm more or less planning on swapping the mobo in my current NAS and experimenting with GPU passthrough using my current GPUs...  It seems good at this point, but is there anyone out there who's worked with virutaliztion and pass-through on AMD stuff?  Anything not obvious that I should know about coming off UnRAID on Intel hardware?

[limegorilla]

2 minutes ago, Bureaucromancer said:

Just checking in to see if anybody is seriously contemplating an attempt at an UnRAID VM/Gaming NAS type thing based around Ryzen fairly soon after launch?

 

At the moment I'm more or less planning on swapping the mobo in my current NAS and experimenting with GPU passthrough using my current GPUs...  It seems good at this point, but is there anyone out there who's worked with virutaliztion and pass-through on AMD stuff?  Anything not obvious that I should know about coming off UnRAID on Intel hardware?

i would like to try but unfortunatly, i dont think i'll get the chance.

[Electronics Wizardy]

Well linux has support for ryzen so it will work fine.

 

IDK about iommu on ryzen. They might limit it to some chipsets or sku's

 

I don't know if they support amd_iommu. I wouldn't be suprised if they didn't as very few use it.

 

Id suggest anouther linux distro than unaid personally.

[Electronics Wizardy]

Also if you making a vm/nas box. Just use windows and storage spaces in windows. No need for linux and a vm. It will just make it more annoying to use.

[Bureaucromancer]

Yeah, no word on IOMMU on Ryzen, but UnRAID does support it on AMD so far as it exists at this point.

 

I do see the argument for something a bit more standardized, but it definitely makes virtualization better than most other things.  Ultimately though it's UnRAID and BTRFS itself that sold me on the platform (and the fact that yes, I'm running it) - other pooling options really don't work as well for a heterogeneous collection of drives.  Until I went UnRAID a while ago I was running Amahi, and god was Greyhole bad.  Storage spaces don't have proper parity and from everything I've heard have pretty awful performance.

 

Suffice it to say I think I've seen enough to make this a bit of a build log if I go ahead.

[Electronics Wizardy]

2 minutes ago, Bureaucromancer said:

Yeah, no word on IOMMU on Ryzen, but UnRAID does support it on AMD so far as it exists at this point.

 

I do see the argument for something a bit more standardized, but it definitely makes virtualization better than most other things.  Ultimately though it's UnRAID and BTRFS itself that sold me on the platform (and the fact that yes, I'm running it) - other pooling options really don't work as well for a heterogeneous collection of drives.  Until I went UnRAID a while ago I was running Amahi, and god was Greyhole bad.  Storage spaces don't have proper parity and from everything I've heard have pretty awful performance.

 

Suffice it to say I think I've seen enough to make this a bit of a build log if I go ahead.

Id probably just use storage spaces in windows. No reason to run vm's and it supports all the same things as btrfs(snapshots, checksumming, compression)

[ChalkChalkson]

On 2/18/2017 at 4:43 AM, Bureaucromancer said:

Just checking in to see if anybody is seriously contemplating an attempt at an UnRAID VM/Gaming NAS type thing based around Ryzen fairly soon after launch?

 

At the moment I'm more or less planning on swapping the mobo in my current NAS and experimenting with GPU passthrough using my current GPUs...  It seems good at this point, but is there anyone out there who's worked with virutaliztion and pass-through on AMD stuff?  Anything not obvious that I should know about coming off UnRAID on Intel hardware?

I am planning to do exactly that but I can't find any reliable information whether I can make it work or not. 

I really want to use unraid though, since I am planning to clone a server that is running at my parents house, as well as sycing them over night to finally have some off-site backup

[ChalkChalkson]

I really hope they included this into their Ryzen review....

Otherwise I can promise you that I will try, and if it doesn't work, I will just run plain Ubuntu (play mostly EU4 atm) and figure out how to host an SMB share and Linux

[boblikepie]

I'm considering it. I've got an old GPU that's not doing anything really at the moment so thinking about trying a 2 gamer's 1 CPU as well as running developer VM's with Linux and Windows.

[Bureaucromancer]

Just as an update, I've ordered an 1800x and the Crosshair.  Will check in when somethings put together.

[VolkovBrad67]

From the limited reports I've seen, the IOMMU groups on the Ryzen motherboards aren't looking good so far so it will probably be challenging to make this work properly. Having the GPU paired with something else isn't nice for GPU-pass-through...

[RedHatTurtle]

I'm thinking of doing this, but without a iGPU how would i access the linux VM while using GPU pass-through to windows? 

[Bureaucromancer]

So at this moment my build is working pretty well.  I went ahead with an 1800X, the crosshair VI, a 970 for the VM and an RX460 in the primary (low enough power consumption and price I could bear using it for the host system and powerful enough to be interesting if I can ever coax the system into running a two-gamers one pc type multi headed thing).  Basically the IOMMU groups are as bad as reported initally, but ACS override works almost perfectly, so splitting them hasn't actually been a problem.  Performance and stability aren't perfect, but I'm reasonably happy at the moment.  In short, yes, it is doable.

[Followtheleader]

@Bureaucromancer That's awesome to hear, but when you say "performance and stability aren't perfect", how bad are we talking here? The occasional stutter, bluescreens, driver issues? And on the performance side, how bad a hit have you taken? Any info would be much appreciated - I live in NZ and while the 1800X is far cheaper than the $1600 6900K, it's still almost 800 bucks, of which I can't really justify a good WAR (Wife Approval Rating) if it's not going to work for her gaming as well

[sazrocks]

On 3/18/2017 at 8:37 PM, Bureaucromancer said:

So at this moment my build is working pretty well.  I went ahead with an 1800X, the crosshair VI, a 970 for the VM and an RX460 in the primary (low enough power consumption and price I could bear using it for the host system and powerful enough to be interesting if I can ever coax the system into running a two-gamers one pc type multi headed thing).  Basically the IOMMU groups are as bad as reported initally, but ACS override works almost perfectly, so splitting them hasn't actually been a problem.  Performance and stability aren't perfect, but I'm reasonably happy at the moment.  In short, yes, it is doable.

I'm considering a build similar to this. Would you be able to elaborate on exactly what slots the GPUs are in and what settings  you're running? Also if you could post the iommu groups that would be helpful.

[Bureaucromancer]

Sorry to disappear, been waiting until I could say something more definitive and it's taken me this long to work through my own hardware quirks with IOMMU and general configuration, but as of now I've got two vms running with IOMMU passthrough to both my graphics cards (a GTX 970 and an RX 460, which I've got some interesting notes on getting working that boil down to 460s seeming to be a little finnicky in their own right).  If anyone cares to take a look there are a number of us working through issues on the Lime-Tech forums, but what it boils down to right now is that VMs seem to be completely stable, and bar the grouping issues, which AMD seems interested in addressing and which ACS passthrough is working as expected for (which is basically almost perfectly for desktop stuff bar GPUs similar enough that they cause cross-talk problems).

 

There are, however, some really odd UnRaid specific (as in not even appearing in Linux desktop installs from what we've heard) stability issues that suggest to me you might want to hold off a bit but that things are promising.  In very short, idle UnRaid systems lock up, quite consistently, within 36 hours of boot.  A running Windows 10 VM, even idle seems to prevent this.  Possibly related is that running a Linux (or at least Ubuntu) VM seems to be able to bring down the whole system, and do so regularly.  This is all really odd, but happening to multiple people - right now looking to be UnRaid itself causing the issue somehow, since, among other things, the current release (6.3.2 until about an hour ago and 6.3.3 at the moment) have had Ryzen kernel updates backported to them, though there is interest in what might happen once the Kernel version they actually come from is available.

 

So, short version, it works, with some issues.  If you're comfortable being an early adopter it's looking good, but if you need guarantees and/or a production ready or just as easy to use as an on-the-metal gaming rig, hold off a big, there are substantial issues.  These issues are probably even worse for anyone trying to run a straight server environment, but that's probably going to be more of an issue once we start getting R5 and R3 than it is right now.  I'd also note that for the gaming crowd (frankly it doesn't bother me much in the NAS role) UnRaid's temperature monitoring and fan control plugins aren't ready for Ryzen hardware at the moment (shame since I've got a cooling setup that would let me push airflow to any combination of CPU, GPU and drive pools under individual control).

[Bureaucromancer]

And in further good news, it looks like the Ryzen related crashing is most likely something to do with idle c-states.  It is, by all appearances, fixed by turning off "Global C-State Control" on the mobo, although at the cost of about 15w idle power consumption.  Lime Tech is looking into it, but I'm now actually comfortable saying that it works properly.

[Tiger]

I'd love to this too using 1600x in the future.. Wondering if I should use UNRAID as NAS and WIndows VM for everything else or... a separate NAS machine.. My QNAP's "home" NAS performance is pretty terrible. The cost of old pc +freenas seems to be a better option..

[Bureaucromancer]

So the short version is do you want something to tinker with or that just works?  Because if it's a question of something that just works, yes, you want separate machines (though I could make the case for UnRaid if you're custom building the server, and definitely agree that the hardware in a lot of off-the-shelf nas' is anemic at best).  That said, as a project pc something like this DOES work.  As far as using a 1600, it definitely sound reasonable, although more as NAS + desktop than something potentially capable of multiple simultaneous desktops.  Practically speaking I'd point out Docker and mismatched drive sizes as pretty major upsides to UnRaid, but suggest that virtualizing your desktop is more a thing to be done for the sake of it, though there definitely is potential to make better use of available resources if you're running heavier server applications.  Multi gamer setups are also definitely fun, if a bit gimmicky for most people.

 

I'd say that this is probably best for people already attracted to, or ideally already familiar with, UnRaid, but quite reasonable if it's something you WANT to do.  I also believe that it will be a lot more meaningful to talk of this kind of thing's long term potential after an update or two to UnRaid and once the Ryzen BIO's have settled down a bit - as things sit we don't have all relevant drivers and updates in the Linux kernel, there are some stability issues that seem early adopter related but are still rather mysterious, and Ryzen itself is getting patches that AMD themselves describe as stability related.

[Tiger]

15 hours ago, Bureaucromancer said:

So the short version is do you want something to tinker with or that just works?  Because if it's a question of something that just works, yes, you want separate machines (though I could make the case for UnRaid if you're custom building the server, and definitely agree that the hardware in a lot of off-the-shelf nas' is anemic at best).  That said, as a project pc something like this DOES work.  As far as using a 1600, it definitely sound reasonable, although more as NAS + desktop than something potentially capable of multiple simultaneous desktops.  Practically speaking I'd point out Docker and mismatched drive sizes as pretty major upsides to UnRaid, but suggest that virtualizing your desktop is more a thing to be done for the sake of it, though there definitely is potential to make better use of available resources if you're running heavier server applications.  Multi gamer setups are also definitely fun, if a bit gimmicky for most people.

 

I'd say that this is probably best for people already attracted to, or ideally already familiar with, UnRaid, but quite reasonable if it's something you WANT to do.  I also believe that it will be a lot more meaningful to talk of this kind of thing's long term potential after an update or two to UnRaid and once the Ryzen BIO's have settled down a bit - as things sit we don't have all relevant drivers and updates in the Linux kernel, there are some stability issues that seem early adopter related but are still rather mysterious, and Ryzen itself is getting patches that AMD themselves describe as stability related.

Man, awesome reply.

 

 

I'm just hoping there's another alternative to UnRAID that are as stable and as user friendly that also does virtualizing and PCI pass through.

 

I'm thinking of 1600x + 2x rx480

Then some hdds.

 

Planning to pass 480s into windows for mining coins on free electricity to reap cost back in about 4 months. Maybe I'll need another gt 740 or something for UnRAID. Correct?

 

I'm trying to save space and costs too. Since ATX are cheaper than 2 itxs + cases.

[2bitmarksman]

Some great info here on Ryzen and UnRAID. I hope they can get most of the Linux kernal and Ryzen issues sorted before the release of Naples; I'd love seeing what a Zen server CPU is capable of, such as X gamers 1 Box type vids.

Also as someone who has never used UnRAID and wants to essentially use it as a cheap hypervisor, would you say it is a decent host OS to learn more about virtualization server structure, NAS/vSAN storage and just to tinker with?

[leletec]

I am planning on running a linux distro (for work and stuff) parallel to Windows10 (just for the games that need it), maybe using UnRAID.

Do you think that this is possible with a Ryzen 1700X and two GPUs or will it be unstable / will a loose performance?

[VolkovBrad67]

6 hours ago, leletec said:

I am planning on running a linux distro (for work and stuff) parallel to Windows10 (just for the games that need it), maybe using UnRAID.

Do you think that this is possible with a Ryzen 1700X and two GPUs or will it be unstable / will a loose performance?

Based on my personal experience with UnRaid and combining this with the testimonies of users running Ryzen with UnRaid, in terms of performance you get pretty much the same thing as a dedicated computer. You can pass through any hardware you need (GPUs, Disks, Network cards, USB cards) and get pretty much native performance. The big drawback is that even though UnRaid is probably the easiest platform to achieve this kind of setup, it's still never guaranteed that this works exactly as expected right out of the box. So you could be "lucky" and just pass through both GPUs, create a Win10 VM, install your drivers and play your game like you would on a regular desktop PC, or you may have to tinker with some settings first before you get it to work exactly as you want it to. If you are ready to spend some time learning how this works, then UnRaid is a phenomenal platform, but if you expect that everything will work just by clicking on a few buttons on the GUI, then you might be disappointed (although it might indeed work for you, right out of the box for your use case with your particular hardware).

 

Personally I would not be able to go back to a different setup without UnRaid. I use it for everything, work, games, dockers for various scenarios (video streaming, etc.), I use VMs, I love the easy storage solution, etc...

[leletec]

3 minutes ago, Ti133700N said:

If you are ready to spend some time learning how this works, then UnRaid is a phenomenal platform, but if you expect that everything will work just by clicking on a few buttons on the GUI, then you might be disappointed

Thanks for that inspiring post and for the quote.
I am indeed willig to put some effort into it, but happy to see that I do not have to be perfect with linux and virtualization to get it to work (as far as I understood it correctly)

[PSYCHOPATHiO]

I'm a lte commer but I''m currently running unRAID 6.4 RC14 with Asus x370 pro and a ryzen 1700x, everything is stable and running smooth.

VMs working fine, gpu pass through is great. cant complain