poor seq read performance using cachecade on dell h700

_BJE_ · January 29, 2020

here is the run down of the situation i recently decided to give cachecade a try to see if i can boost the performance of the server but after a few trys and benchmarks it seems instead of benefiting there are more problem and i have no idea what is causing it. without using ssd cache i get 8-900 MB/s seq read and usually 6-700 MB/s seq write but once i used cachecade my seq read plummeted to around 350 MB/s sometime even 100-200 MB/s write speed is around the same since dell said it only caches read. during benchmark if i use file smaller than 1gb in seq red i get the usual performance since it uses the on board 1gb catch it has on the raid controller but yea any input on what is causing will be a massive help

ps. im no expert just someone that likes tinkering so sorry if i sound dumb

EDIT: reandom read saw improvement across the board while using cachecade just seq read took a massive impact so its kinda working i guess and anything below 500mb file size uses the on board cache.

Cachecade on

Cachcade off

system

duo intel x5667 cpu

16g ddr3 ecc mem

dell perc h700 raid controller

6x4tb ironwolf drives in raid 5

ssd samsung 850evo 500gb

Edited January 29, 2020 by _BJE_

leadeater · January 29, 2020

I've had problems before with LSI RAID controllers and Samsung consumer SSDs, using and not using CacheCade. I think using ESXi was a contributing factor. Anyway SSDs on RAID cards means TRIM isn't supported and a lot of SSDs just don't like that and quite quickly lose performance. I was using 840 and 850 Pros which would eventually get stuck at around 30-40 MB/s seq read/write, I'd take them off the RAID card and put them on to my Windows desktop and run a diskpart clean and that would get the performance back but it would always happen.

I'd suggest getting a used server SSD off ebay if you really want to use that feature of your RAID card.

Jarsky · January 29, 2020

It sounds like you're using a single SSD? Consider that each disk is limited by its SATA interface speed, so if you have a single SSD then the absolute max you're going to get is approx 550MB/s. Really Cachecade should probably have at least a couple of them for a benefit on most HDD arrays.

Also add to that that the Samsung EVO's are TLC and use an onboard small SLC buffer like most other cheaper consumer SSD's to get their speed.

On the 500GB EVO its about 6GB, so if you're writing files larger than this, after that SLC buffer is consumed, then the raw speed of an SSD can drop considerably to about 50MB/s or less. They also suffer due to the lack of TRIM support on the RAID Card as leadeater pointed out.

Ideally you want at least an MLC SSD such as a Samsung Pro and most of the affordable Enterprise SSDs, or at the very high end an SLC SSD such as some of the expensive enterprise SSD's.