Jump to content

ZFS Raid Z1

BLACKRIPPER
By BLACKRIPPER
in Servers, NAS, and Home Lab
 Share
Posted

Hey guys, ive set up a raid z1 with 4 x 8 tb drives. Is it possible when I add another 4x8tb drives to stripe it and enhance the speed without loose the data? Ty BLVCK

Software: FreeNAS-11.2-U7

Hardware: Ram 16gb | cpu i5 3470 | 40 gbit Mellanox Card | Case intertech 4416 | Controller Sas9305-16i LSI Logic 12gb/s

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

yep, you can add anouther vdev to a zfs pool at any time. It won't balance the data though, so the speed will be slower than peak, esp if you have lots of space already used.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
2 minutes ago, Electronics Wizardy said:

yep, you can add anouther vdev to a zfs pool at any time. It won't balance the data though, so the speed will be slower than peak, esp if you have lots of space already used.

So, should I rebuild the Pool. Is the chematic correct with Raidz1 or should I use something other.

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
Just now, BLACKRIPPER said:

So, should I rebuild the Pool. Is the chematic correct with Raidz1 or should I use something other.

You don't need to rebuild it, i personally wouldn't. raidz1 seems like the best option here.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
Just now, Electronics Wizardy said:

You don't need to rebuild it, i personally wouldn't. raidz1 seems like the best option here.

Yeah but at the moment I only get like ~150 mb/s over smb

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
10 minutes ago, BLACKRIPPER said:

Yeah but at the moment I only get like ~150 mb/s over smb

what speed do you get on the local server storage benchmark?

 

What iperf speeds do you get?

 

 

Whats your network setup?

 

That drive setup should get well over 150mb/s, id expect about 400mB/s sequentical.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

When I build my freenas box I referred to this spreadsheet here. It basically lists all the raid options with different amount of drives and their speeds. 

 

https://calomel.org/zfs_raid_speed_capacity.html

Gaming HTPC:

R5 5600X - Cryorig C7 - Asus ROG B350-i - EVGA RTX2060KO - 16gb G.Skill Ripjaws V 3333mhz - Corsair SF450 - 500gb 960 EVO - LianLi TU100B


Desktop PC:
R9 3900X - Peerless Assassin 120 SE - Asus Prime X570 Pro - Powercolor 7900XT - 32gb LPX 3200mhz - Corsair SF750 Platinum - 1TB WD SN850X - CoolerMaster NR200 White - Gigabyte M27Q-SA - Corsair K70 Rapidfire - Logitech MX518 Legendary - HyperXCloud Alpha wireless


Boss-NAS [Build Log]:
R5 2400G - Noctua NH-D14 - Asus Prime X370-Pro - 16gb G.Skill Aegis 3000mhz - Seasonic Focus Platinum 550W - Fractal Design R5 - 
250gb 970 Evo (OS) - 2x500gb 860 Evo (Raid0) - 6x4TB WD Red (RaidZ2)

Synology-NAS:
DS920+
2x4TB Ironwolf - 1x18TB Seagate Exos X20

 

Audio Gear:

Hifiman HE-400i - Kennerton Magister - Beyerdynamic DT880 250Ohm - AKG K7XX - Fostex TH-X00 - O2 Amp/DAC Combo - 
Klipsch RP280F - Klipsch RP160M - Klipsch RP440C - Yamaha RX-V479

 

Reviews and Stuff:

GTX 780 DCU2 // 8600GTS // Hifiman HE-400i // Kennerton Magister
Folding all the Proteins! // Boincerino

Useful Links:
Do you need an AMP/DAC? // Recommended Audio Gear // PSU Tier List 

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author

edit 

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
22 minutes ago, Electronics Wizardy said:

what speed do you get on the local server storage benchmark?

 

What iperf speeds do you get?

 

 

Whats your network setup?

 

That drive setup should get well over 150mb/s, id expect about 400mB/s sequentical.

if the copy is finished I will run the benchmarks. But the short Iperf test show low results while Downloading with ~350mb/s  and windows still copying. Im looking for the interfaces. I think ive got some db loss on the line with dirt on the connectors -> used cables from work.

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
37 minutes ago, BLACKRIPPER said:

if the copy is finished I will run the benchmarks. But the short Iperf test show low results while Downloading with ~350mb/s  and windows still copying. Im looking for the interfaces. I think ive got some db loss on the line with dirt on the connectors -> used cables from work.

what is the link speed?

 

Can you show the iperf screenshot? That seems very odd.

 

Dirt on connectors shouldn't cause this problem.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
3 minutes ago, Electronics Wizardy said:

what is the link speed?

 

Can you show the iperf screenshot? That seems very odd.

 

Dirt on connectors shouldn't cause this problem.

10 gbit Multimode for just testing. Later ive got singlemode cleaned with 10 gbit too.

image.png

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
10 minutes ago, Electronics Wizardy said:

what is the link speed?

 

Can you show the iperf screenshot? That seems very odd.

 

Dirt on connectors shouldn't cause this problem.

These are the stats from the gbic          

sfp-tx-power: -3.239dBm
sfp-rx-power: -3.808dBm

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
3 hours ago, BLACKRIPPER said:

10 gbit Multimode for just testing. Later ive got singlemode cleaned with 10 gbit too.

image.png

what nics?

 

What switches?

 

What specs on both ends?

 

Thts way low

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
20 hours ago, Electronics Wizardy said:

what nics?

 

What switches?

 

What specs on both ends?

 

Thts way low

Hardware PC1: ASUS (XG-C100F) Hardware FreeNas: Mellanox MCX324A ConnectX-3 40GbE Dual-Port Switch: MikroTik CRS326-24S+2Q+RM Gbic: Rapidcon Multimode J9150A-C

Freenas -> QSFP 40gbe -> Switch -> Gbic 10gbe -> multimode cable -> Gbic 10gbe-> PC

 

image.png

GPU: Powercolor 7900XTX Hellhound, CPU: 5900X, MB: MSI MAG X570 TOMAHAWK WIFI, RAM: 2kits F4-3600C18D-32GVK, PSU: Seasonic Focus GX-1000, Case: BeQuiet 601 Silenced, CPU Cooler: BeQuiet Dark Rock Pro3 + AM4 Plate, NIC: Asus XG-C100F 10G, Soundcard: Creative Sound Blaster Z, 1TB Kingston NVME, Some old SSDs, HDD: 32TB@7200, Beyerdynamic DT 990 Pro 250 Ohm, WQHD 144hz IPS Display

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
29 minutes ago, BLACKRIPPER said:

Hardware PC1: ASUS (XG-C100F) Hardware FreeNas: Mellanox MCX324A ConnectX-3 40GbE Dual-Port Switch: MikroTik CRS326-24S+2Q+RM Gbic: Rapidcon Multimode J9150A-C

Freenas -> QSFP 40gbe -> Switch -> Gbic 10gbe -> multimode cable -> Gbic 10gbe-> PC

 

image.png

You don't happen to have any more nics to test with?

 

What have you set up on this switch? Using some features like l3 will kill speed

 

Maybe try a linux live disk on both ends.

Link to comment
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Servers, NAS, and Home Lab

×