MSI Creator TRX40 RAID Setup

[aoused]

Hello,

 

I have a query about whether I have set my RAID up in the best way...

 

CPU: Threadripper 3960X

MB: MSI Creator TRX40 (latest BIOS, drivers directly from AMD)

Memory: 8x 8GB Corsair Dominator Platinum RGB (XMP on, no other overclocking)

GFX: 2080Ti

Storage: 7x 2TB Seagate FireCuda 520 PCIe 4

 

I have have single system drive installed in the single chipset M2 slot (M2_3).  I then have the remaining six drives installed in the two onboard slots (M2_1 and M2_2) and the M.2 XPANDER-AERO GEN4 card.  I have switched on AMD RAIDXpert2 with two arrays - one single disk array for the OS disk and one RAID 0 for the remaining six (temporary scratch storage so don't care about redundancy).

 

I set this up in December last year and ran some synthetic disk tests, I got some obscene figures (I think like 20 GB/s).  However I am not convinced I am getting the best performance for my work load (digital forensics).  I kind of feel like I am getting 'brown out' during certain operations.

 

A few questions:

 

1 - Is this a sensible way to configure this lot?

2 - I've gone with default caching options which for SSD storage is Read Ahead and Write Back disabled.  Cache Tag Size is set to 256K.  I assume this is sensible?

3 - I have configured the single OS drive as 'Volume'.  I think I might have been better to set it as 'RAIDABLE'.  As best I can tell, when I enable RAID in the BIOS settings, the OS can no longer directly access the NVMe drive so I have to configure the single disk as an array for the OS to see it?

[Electronics Wizardy]

what os?

 

SInce your not running raid on the boot drive, just use software raid. Motherboard raid isn't great, so juse use the software raid in your os and don't touch raid on the board.

[aoused]

Just now, Electronics Wizardy said:

what os?

 

SInce your not running raid on the boot drive, just use software raid. Motherboard raid isn't great, so juse use the software raid in your os and don't touch raid on the board.

Windows 10 Pro 2004 (could also put Enterprise on it I guess).

 

I toyed with the idea of using software RAID (specifically Storage Spaces) but from looking at few reviews, it seemed RaidXpert2 was fairly competent.  Do you think it would make a big difference?  I assume there isn't an easy way to convert the single volume to a simple standalone without reinstalling everything?

[Electronics Wizardy]

Just now, aoused said:

Windows 10 Pro 2004 (could also put Enterprise on it I guess).

 

I toyed with the idea of using software RAID (specifically Storage Spaces) but from looking at few reviews, it seemed RaidXpert2 was fairly competent.  Do you think it would make a big difference?  I assume there isn't an easy way to convert the single volume to a simple standalone without reinstalling everything?

Id sy softwre raid with storage spaces is much better than the amd raid. Much better options, and much easier to transfer between systems, and much more tested.

[Guest]

RaidXpert2 is trash and often breaks with BIOS updates.

I had it tell me a perfectly functional RAID was degraded, then after getting it to stop that, the tray icon was permanently in an error state.

Overall it was an unpleasant two years that I used it.

Heck, it was even a massive security hole for a long time.

Just take @Electronics Wizardy's advice and use storage spaces.

[aoused]

2 minutes ago, Den-Fi said:

RaidXpert2 is trash and often breaks with BIOS updates.

I had it tell me a perfectly functional RAID was degraded, then after getting it to stop that, the tray icon was permanently in an error state.

Overall it was an unpleasant two years that I used it.

Heck, it was even a massive security hole for a long time.

Just take @Electronics Wizardy's advice and use storage spaces.

I appreciate yours and @Electronics Wizardy's advice.  I have certainly found that each time I update the BIOS, I loose the RAID setup and have to re-build it.  I also find it frustrating that everytime I put a SATA drive into my dock to work on (we store a lot of our case work on HDDs), it sets it up as a new array which I then have to manually remove later - I kind of wish it was possible just to put AMD's RAID on for specific devices.

 

I welcome other's views.  I assume Storage Spaces is sensible for my six 520s in RAID 0 (perhaps I'll use ReFS too) without causing further bottle necks?

 

Looks like I'll do a rebuild in the near future to try it.

[Electronics Wizardy]

Just now, aoused said:

I appreciate yours and @Electronics Wizardy's advice.  I have certainly found that each time I update the BIOS, I loose the RAID setup and have to re-build it.  I also find it frustrating that everytime I put a SATA drive into my dock to work on (we store a lot of our case work on HDDs), it sets it up as a new array which I then have to manually remove later - I kind of wish it was possible just to put AMD's RAID on for specific devices.

 

I welcome other's views.  I assume Storage Spaces is sensible for my six 520s in RAID 0 (perhaps I'll use ReFS too) without causing further bottle necks?

 

Looks like I'll do a rebuild in the near future to try it.

ye there is some tweking you need to do, and you won't get the theroticl speeds those drives can give.

 

What bandwidth do you current see when using your programs?

[Ankh Tech]

24 minutes ago, aoused said:

Hello,

 

I have a query about whether I have set my RAID up in the best way...

 

CPU: Threadripper 3960X

MB: MSI Creator TRX40 (latest BIOS, drivers directly from AMD)

Memory: 8x 8GB Corsair Dominator Platinum RGB (XMP on, no other overclocking)

GFX: 2080Ti

Storage: 7x 2TB Seagate FireCuda 520 PCIe 4

 

I have have single system drive installed in the single chipset M2 slot (M2_3).  I then have the remaining six drives installed in the two onboard slots (M2_1 and M2_2) and the M.2 XPANDER-AERO GEN4 card.  I have switched on AMD RAIDXpert2 with two arrays - one single disk array for the OS disk and one RAID 0 for the remaining six (temporary scratch storage so don't care about redundancy).

 

I set this up in December last year and ran some synthetic disk tests, I got some obscene figures (I think like 20 GB/s).  However I am not convinced I am getting the best performance for my work load (digital forensics).  I kind of feel like I am getting 'brown out' during certain operations.

 

A few questions:

 

1 - Is this a sensible way to configure this lot?

2 - I've gone with default caching options which for SSD storage is Read Ahead and Write Back disabled.  Cache Tag Size is set to 256K.  I assume this is sensible?

3 - I have configured the single OS drive as 'Volume'.  I think I might have been better to set it as 'RAIDABLE'.  As best I can tell, when I enable RAID in the BIOS settings, the OS can no longer directly access the NVMe drive so I have to configure the single disk as an array for the OS to see it?

Off topic question here, how good is the board?

[aoused]

1 minute ago, Electronics Wizardy said:

ye there is some tweking you need to do, and you won't get the theroticl speeds those drives can give.

 

What bandwidth do you current see when using your programs?

A lot of my work load is taking forensic copies of hard drives (stored in a multi-part compressed Expert Witness Format files) and processing the data.  Such processing involved hashing files, looking for certain artefacts, copying snippets of data to the application's SQLite DB file.  Other processes are more 'multimedia' orientated - taking folders full of millions of images and videos and creating thumbnails, extracing EXIF data, running facial recognition.  It is therefore really difficult to get an idea of what the bandwidth is.  In striaght copy operations (e.g. moving 2GB files from system drive to RAID) I get OK speeds - perhaps 2-300 mb/s (I think, I might be a bit off as I'll need to check in the office on Monday).

 

One question I did have is whether a single RAID is the way to go.  Whether I would see better performance if I stored my raw data (e.g. what's being read from) on one array and my 'processed' (e.g. what's being written to) data on another.  I had assumed it didn't really matter for NVMe drives of this speed, but I could be wrong.

[Electronics Wizardy]

2 minutes ago, aoused said:

A lot of my work load is taking forensic copies of hard drives (stored in a multi-part compressed Expert Witness Format files) and processing the data.  Such processing involved hashing files, looking for certain artefacts, copying snippets of data to the application's SQLite DB file.  Other processes are more 'multimedia' orientated - taking folders full of millions of images and videos and creating thumbnails, extracing EXIF data, running facial recognition.  It is therefore really difficult to get an idea of what the bandwidth is.  In striaght copy operations (e.g. moving 2GB files from system drive to RAID) I get OK speeds - perhaps 2-300 mb/s (I think, I might be a bit off as I'll need to check in the office on Monday).

 

One question I did have is whether a single RAID is the way to go.  Whether I would see better performance if I stored my raw data (e.g. what's being read from) on one array and my 'processed' (e.g. what's being written to) data on another.  I had assumed it didn't really matter for NVMe drives of this speed, but I could be wrong.

What Io latency are you getting? 

 

That speed seems low for copying between the boot ssd and the data array, Im guessing atleast a few GB/s, at least at the start.

 

But Im guessing that raid array really won't help performance, Im guessing those programs can't process data faster than a single drive can supply it. 

 

Can you show crystal disk mart screenshots?

 

 

[aoused]

4 minutes ago, Ankh tech tips said:

Off topic question here, how good is the board?

It's the first PC of this calibre (and probably the last for a while) that I've been able to build.  I found the whole process relatively painless.  I had some issues of having to pull parts out (memory) to get it to boot but now everything is 'bedded in' I have absolutely no stability issues; it's been on pretty much 24/7 since December last year.

 

I personally dislike the BIOS and the bundled control software.  Finding some of the options (SVM in my case) caused me some head scratching.  To simply change the RGB colours I had to install a lot of bloated software.

 

I haven't really done any overclocking (other than my GPU) as I don't know how! And in any case, it appears my software is the bottle neck rather than the Threadripper.

[Ankh Tech]

1 minute ago, aoused said:

It's the first PC of this calibre (and probably the last for a while) that I've been able to build.  I found the whole process relatively painless.  I had some issues of having to pull parts out (memory) to get it to boot but now everything is 'bedded in' I have absolutely no stability issues; it's been on pretty much 24/7 since December last year.

 

I personally dislike the BIOS and the bundled control software.  Finding some of the options (SVM in my case) caused me some head scratching.  To simply change the RGB colours I had to install a lot of bloated software.

 

I haven't really done any overclocking (other than my GPU) as I don't know how! And in any case, it appears my software is the bottle neck rather than the Threadripper.

Thanks

[aoused]

2 minutes ago, Electronics Wizardy said:

What Io latency are you getting? 

 

That speed seems low for copying between the boot ssd and the data array, Im guessing atleast a few GB/s, at least at the start.

 

But Im guessing that raid array really won't help performance, Im guessing those programs can't process data faster than a single drive can supply it. 

 

Can you show crystal disk mart screenshots?

 

 

I'll have a look when I'm back in the office on Monday.  I can certainly share some screenshots.   How is it best to measure the IO latency?  I have definitely seen performance higher than the figures I've suggested but it seems to depend on what I'm copying (which can either be 2 GB chunks of data or a folder of 100,000 2kb files!)

 

The main bit of software I use can currently only use 32 threads during certain operations.  I've asked for them to add more threads but they have cited it being a 'rule of diminishing returns' for those particular multithreaded operations (essentially one thread processes a single file from the forensic image of a computer's hard drive, hashing, looking for keywords etc and storing results into a SQLite DB).  They have however said it may be worth changing the location of their cache folder.  They recommend source files, their cache files and the processed data to all be on separate drives.  I'm somewhat reluctant to split up the six drives as having 12TB available is useful for certain cases but I could probably split into two 6TBs.

[Electronics Wizardy]

2 hours ago, aoused said:

I'll have a look when I'm back in the office on Monday.  I can certainly share some screenshots.   How is it best to measure the IO latency?  I have definitely seen performance higher than the figures I've suggested but it seems to depend on what I'm copying (which can either be 2 GB chunks of data or a folder of 100,000 2kb files!)

 

The main bit of software I use can currently only use 32 threads during certain operations.  I've asked for them to add more threads but they have cited it being a 'rule of diminishing returns' for those particular multithreaded operations (essentially one thread processes a single file from the forensic image of a computer's hard drive, hashing, looking for keywords etc and storing results into a SQLite DB).  They have however said it may be worth changing the location of their cache folder.  They recommend source files, their cache files and the processed data to all be on separate drives.  I'm somewhat reluctant to split up the six drives as having 12TB available is useful for certain cases but I could probably split into two 6TBs.

How about a 2tb(or other drive) for boot, a 2tb for media, and a 10tb pool for project files?

 

You can monitor io latency with task manager, or get much more detail in perfmonitor.