TrueNas Core slowing down to 2-3 mb/s despite still having lots of ram

[Flux Azreal]

As the title says, I don't know why my write is slowed down, I interact with the drive via Windows 11's file explorer SMB network drive feature, the specs of my truenas is:

KingFast 512gb SSD

Intel i5-6500 CPU

16GB Ram

Ethernet 1,000 mbps

 

I'm very new to truenas, so I don't know 100% how this is supposed to work, I think there's something called ZFS that leveraged ram for accelerated file transfers, I assume when I upload, it uploads directly to the ram and from the ram to the SSD, it doesn't seem to be doing much of that. Is this the expected behavior? Or do I need to configure something? What are considerations should I take into account? Both machines are wired to the modem via ethernet

[AbydosOne]

7 minutes ago, Flux Azreal said:

my truenas is:

KingFast 512gb SSD

Like, your disk array is one SSD? You're probably exceeding the SSD's cache and it's writing straight to TLC/QLC flash, which is slow (Kingfast is not exactly top-of-the-line).

[Needfuldoer]

Any particular reason you're running Core? Basically all future development is going to be on Scale. 

 

That won't explain your slow speeds though, since they're both running ZFS (which should use up to half your physical RAM as a cache).

 

Is it just from that one client, or from any machine?

[Flux Azreal]

2 minutes ago, AbydosOne said:

Like, your disk array is one SSD? You're probably exceeding the SSD's cache and it's writing straight to TLC/QLC flash, which is slow (Kingfast is not exactly top-of-the-line).

Nope! I just have a single ssd, I don't use truenas for backups just for conveniant file upload and sharing (nothing important) lol

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

[ToboRobot]

Just now, Flux Azreal said:

Nope! I just have a single ssd, I don't use truenas for backups just for conveniant file upload and sharing (nothing important) lol

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

Do you want a RAM drive?

[Flux Azreal]

6 minutes ago, Needfuldoer said:

Any particular reason you're running Core? Basically all future development is going to be on Scale. 

 

That won't explain your slow speeds though, since they're both running ZFS (which should use up to half your physical RAM as a cache).

 

Is it just from that one client, or from any machine?

I don't know much about truenas/truenascore, I picked core because I found some forums saying it's more performant, but I'll switch to truenas scale in a bit since that's where the development is, and better for support.

And yeah it's all clients, ive tried it with my laptop and it did the same thing!

[Flux Azreal]

1 minute ago, ToboRobot said:

Do you want a RAM drive?

No, a ram drive (afaik) is just storing files in ram, I want it so when I upload a file it'll be in ram first, and progressively put into my ssd so I don't run into this issue! I have enough ram for the operation, but it's just choosing to not use my ram and just putting it right on my ssd

[ToboRobot]

3 minutes ago, Flux Azreal said:

No, a ram drive (afaik) is just storing files in ram, I want it so when I upload a file it'll be in ram first, and progressively put into my ssd so I don't run into this issue! I have enough ram for the operation, but it's just choosing to not use my ram and just putting it right on my ssd

So cache? 

[Flux Azreal]

9 minutes ago, ToboRobot said:

So cache? 

Ram cache

[AbydosOne]

20 minutes ago, Needfuldoer said:

(which should use up to half your physical RAM as a cache)

Pretty much all available RAM. Has always been the case in CORE and has recently been fixed in SCALE.

 

20 minutes ago, Flux Azreal said:

but I thought the whole idea of ZFS was to use my ram for operations like these, but it's not happening! Is it disabled or something? Or is ZFS for something else?

ZFS needs to write to disk eventually, and the incoming cache isn't very large (as it shouldn't be). RAM is volatile, so keeping data in RAM is antithetical to storage. ZFS primarily uses RAM as a read-cache, but with an SSD being your primary storage and only 1Gb Ethernet, it won't be of much advantage.

[Flux Azreal]

Just now, AbydosOne said:

Pretty much all available RAM. Has always been the case in CORE and has recently been fixed in SCALE.

 

ZFS needs to write to disk eventually, and the incoming cache isn't very large (as it shouldn't be). RAM is volatile, so keeping data in RAM is antithetical to storage. ZFS primarily uses RAM as a read-cache, but with an SSD being your primary storage and only 1Gb Ethernet, it won't be of much use.

 

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

[Levent]

6 minutes ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

With write speeds that low, no matter which configuration you change, you will run out ram cache before it system dumps it all to disk.

[OddOod]

Out of curiosity, what is the *exact* model of the SSD? Lots of KF's SSDs don't even have a DRAM cache so it uses a small amount of the storage (a gig or two) as cache.

[Electronics Wizardy]

8 hours ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

The cache is the size of ram, but the write cache is fairly small. You don't want large amounts of data lost if the power goes out. 

 

Look at the disk usage reports, this seems like a slow ssd here, pretty common for low end drives.

[Jarsky]

If you want to you can try setting the zfs_arc_max to something higher like 8GB using the command

echo <SIZE IN BYTES> >> /sys/module/zfs/parameters/zfs_arc_max

 

e.g  

 

cli
echo 8589934592 >> /sys/module/zfs/parameters/zfs_arc_max

reboot

 

But yeah ultimately youre hitting cache limits on the actual nvme drive; as its trying to flush from ARC to Pool

[Godlygamer23]

3 hours ago, Jarsky said:

But yeah ultimately youre hitting cache limits on the actual nvme drive; as its trying to flush from ARC to Pool

I believe this is a SATA drive, not NVMe.

[Bersella AI]

18 hours ago, Flux Azreal said:

I see, I thought the ram cache would be equivilent to the amount of free ram I have, is there a way to increase this? I don't care if something bad happens in the case of outage or whatever

The RAM Cache (or specifically the Adaptive Replacement Cache) on TrueNAS is intended for accelerating read requests in concurrent sessions, and in fact, it would allocate the remaining RAM as more as possible with time elapsing, on both Core and the latest Scale releases. The simplest way to increase the cache size is to install more physical RAM. However, since having concurrent sessions is virtually not the case in home use, the ARC would not help a lot, and more importantly, it would not work at all in writing requests.

For acceleration in writing requests, another SSD may be required as the SLOG vDev attached to the storage pool (in this case, the KingFast SSD). However, its benefits would not be apparent in Ethernet connections of just 1 Gbps, and a more elegant way to utilize this SSD would be assigning it to a separate storage pool.

 

This documentation can be referred to for understanding how RAM or SSD caches are utilized in TrueNAS or ZFS file system.