Jump to content

TrueNas Core slowing down to 2-3 mb/s despite still having lots of ram

As the title says, I don't know why my write is slowed down, I interact with the drive via Windows 11's file explorer SMB network drive feature, the specs of my truenas is:

KingFast 512gb SSD

Intel i5-6500 CPU

16GB Ram

Ethernet 1,000 mbps

 

image.thumb.png.577d20c2fc04256b1eb73b5a12f47950.png

I'm very new to truenas, so I don't know 100% how this is supposed to work, I think there's something called ZFS that leveraged ram for accelerated file transfers, I assume when I upload, it uploads directly to the ram and from the ram to the SSD, it doesn't seem to be doing much of that. Is this the expected behavior? Or do I need to configure something? What are considerations should I take into account? Both machines are wired to the modem via ethernet

Link to comment
Share on other sites

Link to post
Share on other sites

7 minutes ago, Flux Azreal said:

my truenas is:

KingFast 512gb SSD

Like, your disk array is one SSD? You're probably exceeding the SSD's cache and it's writing straight to TLC/QLC flash, which is slow (Kingfast is not exactly top-of-the-line).

Main System (Byarlant): Ryzen 7 5800X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 16GB G.Skill DDR4 3200MT/s CAS-14 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 960 PRO 512GB / 4× Crucial MX500 2TB (RAID-0) | Corsair RM750X | a 10G NIC (pending) | Inateck USB 3.0 Card | Hyte Y60 Case | Dell U3415W Monitor | Keychron K4 Brown (white backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB RAM (soldered) | Vega 6 Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi (all-around awesome machine)

 

Proxmox Server (Veda): Ryzen 7 3800XT | AsRock Rack X470D4U | Corsair H80i v2 | 64GB Micron DDR4 ECC 3200MT/s | 4× WD 10TB / 4× Seagate 14TB Exos / 8× WD 12TB (custom external SAS enclosure) / 2× Samsung PM963a 960GB SSD | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-16i HBA | Fractal Design Node 804 Case (side panels swapped to show off drives) | VMs: TrueNAS Scale; Ubuntu Server (PiHole/PiVPN/NGINX?); Windows 10 Pro; Ubuntu Server (Apache/MySQL)


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s CAS-22 | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | Mellanox ConnectX-2 10G NIC | LG UH12NS30 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Camera: Sony ɑ7II w/ Meike Grip | Sony SEL24240 | Samyang 35mm ƒ/2.8 | Sony SEL50F18F | Sony SEL2870 (kit lens) | PNY Elite Perfomance 512GB SDXC card

 

Network:

Spoiler
                           ┌─────────────── Office/Rack ─────────────────────────────────────────────────────────────────────┐
Google Fiber Webpass ────── UniFi Security Gateway ─── UniFi Switch 8-60W ─┬─ UniFi Flex XG ═╦═ Veda (X550->Proxmox)
(500Mbps↑/500Mbps↓)                             UniFi CloudKey Gen2 (PoE) ─┴─ Veda (IPMI)    ╠═ Veda-NAS (X550->PM->TrueNAS)
╔════════════════════════════════════════════════════════════════════════════════════════════╩═ Narrative (Asus USB 2½G NIC)
║ ┌── Closet ───┐    ┌─────────────── Bedroom ───────────────────────────────────────────────┐
╚═ UniFi Flex XG ═╦╤═ UniFi Flex XG ═╦═ Byarlant
   (PoE)          ║│                 ╠═ Narrative (Cable Matters USB-PD 2½G Ethernet Dongle)
    Kitchen Jack ═╝│                 ╚═ Jesta Cannon*
    (testing)      │        ┌──────── Media Center ──────────────────────────────────────────┐
                   └──────── UniFi Switch 8 ─────────┬─ UniFi Access Point nanoHD (PoE)
Notes:                                               ├─ Sony Playstation 4 
─── is Gigabit / ═══ is Multi-Gigabit                ├─ Pioneer VSX-S520
* = cable passed to Bedroom from Media Center        ├─ Sony XR65A80K (Google TV)
** = cable passed from Media Center to Bedroom       └─ Work Laptop** (Startech USB-PD Dock)

Retired/Other:

Spoiler

Laptop (Rozen-Zulu): Sony VAIO VPCF13WFX | Core i7-740QM | 8GB Patriot DDR3 | GT 425M | Samsung 850EVO 250GB SSD | Blu-ray Drive | Intel 7260 Wifi (lived a good life, retired with honor)

Testbed/Old Desktop (Kshatriya): Xeon X5470 @ 4.0GHz | ZALMAN CNPS9500 | Gigabyte EP45-UD3L | 8GB Nanya DDR2 400MHz | XFX HD6870 DD | OCZ Vertex 3 Max-IOPS 120GB | Corsair CX430M | HooToo USB 3.0 PCIe Card | Osprey 230 Video Capture | NZXT H230 Case

TrueNAS Server (La Vie en Rose): Xeon E3-1241v3 | Supermicro X10SLL-F | Corsair H60 | 32GB Micron DDR3L ECC 1600MHz | 1x Kingston 16GB SSD / Crucial MX500 500GB

Link to comment
Share on other sites

Link to post
Share on other sites

Any particular reason you're running Core? Basically all future development is going to be on Scale. 

 

That won't explain your slow speeds though, since they're both running ZFS (which should use up to half your physical RAM as a cache).

 

Is it just from that one client, or from any machine?

I sold my soul for ProSupport.

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, AbydosOne said:

Like, your disk array is one SSD? You're probably exceeding the SSD's cache and it's writing straight to TLC/QLC flash, which is slow (Kingfast is not exactly top-of-the-line).

Nope! I just have a single ssd, I don't use truenas for backups just for conveniant file upload and sharing (nothing important) lol

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

Link to comment
Share on other sites

Link to post
Share on other sites

Just now, Flux Azreal said:

Nope! I just have a single ssd, I don't use truenas for backups just for conveniant file upload and sharing (nothing important) lol

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

Do you want a RAM drive?

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, Needfuldoer said:

Any particular reason you're running Core? Basically all future development is going to be on Scale. 

 

That won't explain your slow speeds though, since they're both running ZFS (which should use up to half your physical RAM as a cache).

 

Is it just from that one client, or from any machine?

I don't know much about truenas/truenascore, I picked core because I found some forums saying it's more performant, but I'll switch to truenas scale in a bit since that's where the development is, and better for support.

And yeah it's all clients, ive tried it with my laptop and it did the same thing!

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, ToboRobot said:

Do you want a RAM drive?

No, a ram drive (afaik) is just storing files in ram, I want it so when I upload a file it'll be in ram first, and progressively put into my ssd so I don't run into this issue! I have enough ram for the operation, but it's just choosing to not use my ram and just putting it right on my ssd

Link to comment
Share on other sites

Link to post
Share on other sites

3 minutes ago, Flux Azreal said:

No, a ram drive (afaik) is just storing files in ram, I want it so when I upload a file it'll be in ram first, and progressively put into my ssd so I don't run into this issue! I have enough ram for the operation, but it's just choosing to not use my ram and just putting it right on my ssd

So cache? 

Link to comment
Share on other sites

Link to post
Share on other sites

20 minutes ago, Needfuldoer said:

(which should use up to half your physical RAM as a cache)

Pretty much all available RAM. Has always been the case in CORE and has recently been fixed in SCALE.

 

20 minutes ago, Flux Azreal said:

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

ZFS needs to write to disk eventually, and the incoming cache isn't very large (as it shouldn't be). RAM is volatile, so keeping data in RAM is antithetical to storage. ZFS primarily uses RAM as a read-cache, but with an SSD being your primary storage and only 1Gb Ethernet, it won't be of much advantage.

Main System (Byarlant): Ryzen 7 5800X | Asus B550-Creator ProArt | EK 240mm Basic AIO | 16GB G.Skill DDR4 3200MT/s CAS-14 | XFX Speedster SWFT 210 RX 6600 | Samsung 990 PRO 2TB / Samsung 960 PRO 512GB / 4× Crucial MX500 2TB (RAID-0) | Corsair RM750X | a 10G NIC (pending) | Inateck USB 3.0 Card | Hyte Y60 Case | Dell U3415W Monitor | Keychron K4 Brown (white backlight)

 

Laptop (Narrative): Lenovo Flex 5 81X20005US | Ryzen 5 4500U | 16GB RAM (soldered) | Vega 6 Graphics | SKHynix P31 1TB NVMe SSD | Intel AX200 Wifi (all-around awesome machine)

 

Proxmox Server (Veda): Ryzen 7 3800XT | AsRock Rack X470D4U | Corsair H80i v2 | 64GB Micron DDR4 ECC 3200MT/s | 4× WD 10TB / 4× Seagate 14TB Exos / 8× WD 12TB (custom external SAS enclosure) / 2× Samsung PM963a 960GB SSD | Seasonic Prime Fanless 500W | Intel X550-T2 10G NIC | LSI 9300-16i HBA | Fractal Design Node 804 Case (side panels swapped to show off drives) | VMs: TrueNAS Scale; Ubuntu Server (PiHole/PiVPN/NGINX?); Windows 10 Pro; Ubuntu Server (Apache/MySQL)


Media Center/Video Capture (Jesta Cannon): Ryzen 5 1600X | ASRock B450M Pro4 R2.0 | Noctua NH-L12S | 16GB Crucial DDR4 3200MT/s CAS-22 | EVGA GTX750Ti SC | UMIS NVMe SSD 256GB / TEAMGROUP MS30 1TB | Corsair CX450M | Viewcast Osprey 260e Video Capture | Mellanox ConnectX-2 10G NIC | LG UH12NS30 BD-ROM | Silverstone Sugo SG-11 Case | Sony XR65A80K

 

Camera: Sony ɑ7II w/ Meike Grip | Sony SEL24240 | Samyang 35mm ƒ/2.8 | Sony SEL50F18F | Sony SEL2870 (kit lens) | PNY Elite Perfomance 512GB SDXC card

 

Network:

Spoiler
                           ┌─────────────── Office/Rack ─────────────────────────────────────────────────────────────────────┐
Google Fiber Webpass ────── UniFi Security Gateway ─── UniFi Switch 8-60W ─┬─ UniFi Flex XG ═╦═ Veda (X550->Proxmox)
(500Mbps↑/500Mbps↓)                             UniFi CloudKey Gen2 (PoE) ─┴─ Veda (IPMI)    ╠═ Veda-NAS (X550->PM->TrueNAS)
╔════════════════════════════════════════════════════════════════════════════════════════════╩═ Narrative (Asus USB 2½G NIC)
║ ┌── Closet ───┐    ┌─────────────── Bedroom ───────────────────────────────────────────────┐
╚═ UniFi Flex XG ═╦╤═ UniFi Flex XG ═╦═ Byarlant
   (PoE)          ║│                 ╠═ Narrative (Cable Matters USB-PD 2½G Ethernet Dongle)
    Kitchen Jack ═╝│                 ╚═ Jesta Cannon*
    (testing)      │        ┌──────── Media Center ──────────────────────────────────────────┐
                   └──────── UniFi Switch 8 ─────────┬─ UniFi Access Point nanoHD (PoE)
Notes:                                               ├─ Sony Playstation 4 
─── is Gigabit / ═══ is Multi-Gigabit                ├─ Pioneer VSX-S520
* = cable passed to Bedroom from Media Center        ├─ Sony XR65A80K (Google TV)
** = cable passed from Media Center to Bedroom       └─ Work Laptop** (Startech USB-PD Dock)

Retired/Other:

Spoiler

Laptop (Rozen-Zulu): Sony VAIO VPCF13WFX | Core i7-740QM | 8GB Patriot DDR3 | GT 425M | Samsung 850EVO 250GB SSD | Blu-ray Drive | Intel 7260 Wifi (lived a good life, retired with honor)

Testbed/Old Desktop (Kshatriya): Xeon X5470 @ 4.0GHz | ZALMAN CNPS9500 | Gigabyte EP45-UD3L | 8GB Nanya DDR2 400MHz | XFX HD6870 DD | OCZ Vertex 3 Max-IOPS 120GB | Corsair CX430M | HooToo USB 3.0 PCIe Card | Osprey 230 Video Capture | NZXT H230 Case

TrueNAS Server (La Vie en Rose): Xeon E3-1241v3 | Supermicro X10SLL-F | Corsair H60 | 32GB Micron DDR3L ECC 1600MHz | 1x Kingston 16GB SSD / Crucial MX500 500GB

Link to comment
Share on other sites

Link to post
Share on other sites

Just now, AbydosOne said:

Pretty much all available RAM. Has always been the case in CORE and has recently been fixed in SCALE.

 

ZFS needs to write to disk eventually, and the incoming cache isn't very large (as it shouldn't be). RAM is volatile, so keeping data in RAM is antithetical to storage. ZFS primarily uses RAM as a read-cache, but with an SSD being your primary storage and only 1Gb Ethernet, it won't be of much use.

 

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

With write speeds that low, no matter which configuration you change, you will run out ram cache before it system dumps it all to disk.

mY sYsTeM iS Not pErfoRmInG aS gOOd As I sAW oN yOuTuBe. WhA t IS a GoOd FaN CuRVe??!!? wHat aRe tEh GoOd OvERclok SeTTinGS FoR My CaRd??  HoW CaN I foRcE my GpU to uSe 1o0%? BuT WiLL i HaVE Bo0tllEnEcKs? RyZEN dOeS NoT peRfORm BetTer wItH HiGhER sPEED RaM!!dId i WiN teH SiLiCON LotTerrYyOu ShoUlD dEsHrOuD uR GPUmy SYstEm iS UNDerPerforMiNg iN WarzONEcan mY Pc Run WiNdOwS 11 ?woUld BaKInG MY GRaPHics card fIX it? MultimETeR TeSTiNG!! aMd'S GpU DrIvErS aRe as goOD aS NviDia's YOU SHoUlD oVERCloCk yOUR ramS To 5000C18

Link to comment
Share on other sites

Link to post
Share on other sites

Out of curiosity, what is the *exact* model of the SSD? Lots of KF's SSDs don't even have a DRAM cache so it uses a small amount of the storage (a gig or two) as cache.

5950X/4090FE primary rig  |  1920X/1070Ti Unraid for dockers  |  200TB TrueNAS w/ 1:1 backup

Link to comment
Share on other sites

Link to post
Share on other sites

8 hours ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

The cache is the size of ram, but the write cache is fairly small. You don't want large amounts of data lost if the power goes out. 

 

Look at the disk usage reports, this seems like a slow ssd here, pretty common for low end drives.

Link to comment
Share on other sites

Link to post
Share on other sites

If you want to you can try setting the zfs_arc_max to something higher like 8GB using the command

echo <SIZE IN BYTES> >> /sys/module/zfs/parameters/zfs_arc_max

 

e.g  

 

cli
echo 8589934592 >> /sys/module/zfs/parameters/zfs_arc_max

reboot

 

But yeah ultimately youre hitting cache limits on the actual nvme drive; as its trying to flush from ARC to Pool

Spoiler

Desktop: Ryzen9 5950X | ASUS ROG Crosshair VIII Hero (Wifi) | EVGA RTX 3080Ti FTW3 | 32GB (2x16GB) Corsair Dominator Platinum RGB Pro 3600Mhz | EKWB EK-AIO 360D-RGB | EKWB EK-Vardar RGB Fans | 1TB Samsung 980 Pro, 4TB Samsung 980 Pro | Corsair 5000D Airflow | Corsair HX850 Platinum PSU | Asus ROG 42" OLED PG42UQ + LG 32" 32GK850G Monitor | Roccat Vulcan TKL Pro Keyboard | Logitech G Pro X Superlight  | MicroLab Solo 7C Speakers | Audio-Technica ATH-M50xBT2 LE Headphones | TC-Helicon GoXLR | Audio-Technica AT2035 | LTT Desk Mat | XBOX-X Controller | Windows 11 Pro

 

Spoiler

Server: Fractal Design Define R6 | Ryzen 3950x | ASRock X570 Taichi | EVGA GTX1070 FTW | 64GB (4x16GB) Corsair Vengeance LPX 3000Mhz | Corsair RM850v2 PSU | Fractal S36 Triple AIO + 4 Additional Venturi 120mm Fans | 14 x 20TB Seagate Exos X22 20TB | 500GB Aorus Gen4 NVMe | 2 x 2TB Samsung 970 Evo Plus NVMe | LSI 9211-8i HBA

 

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Jarsky said:

But yeah ultimately youre hitting cache limits on the actual nvme drive; as its trying to flush from ARC to Pool

I believe this is a SATA drive, not NVMe.

"It pays to keep an open mind, but not so open your brain falls out." - Carl Sagan.

"I can explain it to you, but I can't understand it for you" - Edward I. Koch

"I didn't die! I performed a tactical reset!" - Apollolol

Link to comment
Share on other sites

Link to post
Share on other sites

18 hours ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

The RAM Cache (or specifically the Adaptive Replacement Cache) on TrueNAS is intended for accelerating read requests in concurrent sessions, and in fact, it would allocate the remaining RAM as more as possible with time elapsing, on both Core and the latest Scale releases. The simplest way to increase the cache size is to install more physical RAM. However, since having concurrent sessions is virtually not the case in home use, the ARC would not help a lot, and more importantly, it would not work at all in writing requests.

For acceleration in writing requests, another SSD may be required as the SLOG vDev attached to the storage pool (in this case, the KingFast SSD). However, its benefits would not be apparent in Ethernet connections of just 1 Gbps, and a more elegant way to utilize this SSD would be assigning it to a separate storage pool.

 

This documentation can be referred to for understanding how RAM or SSD caches are utilized in TrueNAS or ZFS file system.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×