Copying a large file has random freezes on Windows??

[SergeyB]

Hi,
I recently installed TrueNAS as a vitrual machine on Proxmox on my home server.
The server is a 12-core Core i7 with 32GB of DDR3 RAM (picked it up from my workplace...)

On that machine, my TrueNAS VM gets 8GB of RAM and 4 CPU cores to work with.
There are 3 ZFS pools, and each pool has 1 HDD of 6-8TB and some smb shares on the pools' datasets.
There is also an Ubuntu VM installed under Proxmox on that same machine. That Ubuntu mounts some of the shares from TrueNAS for services and such,
and those shares are also mounted as network drives on my Windows machine where I do my work.

Here comes the strange part...
If I copy a large file from the network drive to my Windows local drive, it copies smoothly at peak network bandwidth (1Gbit) but a couple of times throughout the process, it randomly freezes for about 10 seconds, the speed goes down and the TrueNAS VM halts too. Then it unfreezes and continues to copy.
This doesn't happen on the Ubuntu VM, only on my Windows machine.

I would really appreciate if anyone had any shape of idea for why this might be happening??
Thanks in advance!
 

[10leej]

Sounds to me like there one of three things.

 

1. ZFS needs some further tuning (most likely)

2. SMB version incompatibility (Only really relevant if your using Windows 8.1 or older)

3. Hardware Limitations (Is the TruNAS VM using a dedicated ethernet port or a network bridge? Are there other services running on the Proxmox host?)

[SergeyB]

1. Interesting. What kind of ZFS tuning would you look into in this case?
2. It's probably not that, as I'm using Windows 10 (though I read about this in older posts).
3. It's using a network bridge with one 1Gig network port, but there aren't any other services on Proxmox and I brought down all ubuntu services for the testing. I mentioned that it copies at peak bandwidth (when it does), but it actually copies at ~85-90% bandwidth, which I imagine is the toll for bridging.

So could it actually be related to the fact that it shares a NIC?
Otherwise what ZFS settings should I look into?

Thanks!

[SergeyB]

Some more information about my setup:
The disks are passed to TrueNAS CORE through the following VM configuration:
Scsi(2-4) in the image below are the storage HDDs connected to the machine through a USB3 4-bay enclosure. Scsi1 is a cache drive connected through SATA internally, but isn't used in the pool where I'm copying the files from. Scsi0 is TrueNAS's system virtual disk stored on the machine's internal HDD.

(Image attached)

The machine has the following hardware:
CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz (12 threads)
RAM: 4x DIMMs: Kingston 8GB KHX1600C10D3 1333 MT/s (Non-ECC, I think, though TrueNAS says they are)
Chipset: Intel Corporation C600/X79 series chipset
Storage: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller
Network: Intel Corporation 82579V Gigabit Network Connection
GPU: NVIDIA Corporation GK104 [GeForce GTX 660 Ti]

USB3 4-Bay HDD enclosure: JMicron Technology Corp. / JMicron USA Technology Corp. JMS567 SATA 6Gb/s bridge

Storage drives:
1. 1TB HDD for Proxmox and VM virtual disks (Connected internally through motherboard controller)
2. 500GB SSD for caching one of TrueNAS's pool (Connected internally through motherboard controller)
3. 3x 6~8TB SATA3 HDDs (Connected through USB3 4-Bay controller )

Note: The HDDs were previously connected through the same USB3 4-bay enclosure while running just Windows 10 instead of Proxmox and all was running at peak speed with no issues at all.

Any thoughts are greatly appreciated!

[Ralphred]

6 hours ago, SergeyB said:

the storage HDDs connected to the machine through a USB3 4-bay enclosure.

Is the only thing that sticks out, but you aid it works over net.lo so...

Bridging should have no cost, have you changed the ethernet cable*, run a graphical ping, checked the disk/network logs?

 

*cable faults aren't binary like people think, cables exist on a spectrum from "always works" to "only useful as a animal leash"

[SergeyB]

Yeah, it's generally working, it's just showing some strange hiccups.

I checked all the logs but there's nothing suspicious there.
As for the cables, I've been using the same ethernet cables in the last 4 years, nothing changed there. The ping is completely fine too.

I'm wondering if these hiccups are due to the fact that I passed the HDDs individually one by one, instead of passing the enclosure PCI controller...
I've asked about this on the TrueNAS forums and people there seem to be terrified of passing drives through individually, but also running TrueNAS as a VM in the first place...
 

[SergeyB]

Update:
It was suggested on the TrueNAS forum that the freezing is due to having SMR drives, but according to this:
https://nascompares.com/answer/list-of-wd-cmr-and-smr-hard-drives-hdd
only one of the drives I'm passing to TrueNAS is SMR and the other 2 are CMR.

So I'm wondering if that freezing is being caused solely by passing the drives individually, instead of a physical PCIe controller.

Since I have the drives plugged through a single USB3 enclosure box cable (Mediasonic PROBOX 4 Bay 3.5” SATA Hard Drive Enclosure),
which shows up in Proxmox as a USB device, I'm gonna try to pass that instead and see if it makes it better...
An alternative would be to get a PCIe controller card and an enclosure box that connects each drive with its own SATA cable and pass the PCIe controller to Proxmox. But I wanna see if the first option works before I start spending money