FreeNAS network recommendations?

[Reaper 28]

Hey guys, I will try to keep this as short as possible. Most of what I'm asking is that I'm just curious and will possibly purchase some of the parts/items recommended to me.

 

I'm currently running a storage server WHS11 (Raid 5, 5x3TB drives on a LSI controller) and am considering making the switch to something more reliable like FreeNAS. I have never used it before and I'm not the best with network stuff so need some other people's opinions.

 

I plan on running 5-6 drives in RaidZ or RaidZ2 with a SSD cache. On my server setup I hit a bottleneck on my router the server is attached to as I can get 300+MB/s and am being throttled. I was considering 4Gbit when switching to FreeNAS with a switch I'm just not sure which ones to get. I know Intel is one of the best NIC brands to use. I have a x4 PCIE slot open for a network card in my main rig to connect to the switch and NAS. I don't want to go 10Gbit yet as copper etc. is a little to expensive for me right now. I would like to keep costs as low as possible but I want speed.

 

I'm going to assume my array will get over 200MB/s (using a LSI controller on the mobo, flashed to IT mode... 2308) and I want to take full advantage of what speed I can get.

 

But I'm open to suggestions and parts within a price reason lol.

 

Thanks

[Gerr]

Get a 2x or 4x port Intel NIC and put it into that x4 port, then get a switch which supports link aggregation.  That will give you a 2-4 gig connection to your server.  But that wont help if you have a system bottleneck anywhere.

[leadeater]

Link Aggregation won't speed up a single network data copy session between two endpoints, not what it is designed to do. It is for giving increased bandwidth for multiple connections/sessions from multiple different sources, one-to-many and many-to-many only not one-to-one.

 

If you managed to get equipment that supports the hashing algorithm IP Address + TCP/UDP port you could get more bandwidth for each unique data transfer but individually they would still be limited to 125MB/s.

 

If you want to reliably get increased bandwidth between your computer and NAS upgrading to 10Gb is the only option that is guaranteed to work. Also 10Gb is much cheaper than you probably think, go looking on ebay for cheap used 10Gb NICs.

[Reaper 28]

14 hours ago, leadeater said:

Link Aggregation won't speed up a single network data copy session between two endpoints, not what it is designed to do. It is for giving increased bandwidth for multiple connections/sessions from multiple different sources, one-to-many and many-to-many only not one-to-one.

 

If you managed to get equipment that supports the hashing algorithm IP Address + TCP/UDP port you could get more bandwidth for each unique data transfer but individually they would still be limited to 125MB/s.

 

If you want to reliably get increased bandwidth between your computer and NAS upgrading to 10Gb is the only option that is guaranteed to work. Also 10Gb is much cheaper than you probably think, go looking on ebay for cheap used 10Gb NICs.

 

I know the Intel X cards are well supported and are roughly $100-150 per card but looking at 10Gb switches they are $600+ unless you can recommend a cheaper one? I may also have a third PC connected to the FreeNAS system with a SSD array from another LSI card...

[leadeater]

4 hours ago, Reaper 28 said:

 

I know the Intel X cards are well supported and are roughly $100-150 per card but looking at 10Gb switches they are $600+ unless you can recommend a cheaper one? I may also have a third PC connected to the FreeNAS system with a SSD array from another LSI card...

Just directly connect the computer and NAS on the 10Gb NICs, it's what I do. Then have standard 1Gb connections for internet access and for any non 10Gb clients.

 

I use Intel X540-T1's but there are much cheaper options.

[TapfererToaster]

Try getting a pair of used SPF+ 10 Gb adapter on ebay. As far as i know they sell for <30-40$ most of the time in the US (check on the Freenas forum if the type you want is supported).

Then get some cables (those are more expensive than typical cat.5/6 cables) and get a used switch with ~4 SPF+ ports + 1 Gb RJ45 Ports.

 

Here is a Blog post about it on STH:

http://www.servethehome.com/time-go-10gb-ethernet-awesome-deal-10gb-nic-dac/

[Reaper 28]

18 hours ago, leadeater said:

Just directly connect the computer and NAS on the 10Gb NICs, it's what I do. Then have standard 1Gb connections for internet access and for any non 10Gb clients.

 

I use Intel X540-T1's but there are much cheaper options.

Wait isn't the X540-T1 a card that uses CAT cabling? I found the X520 DA1 which uses SFP. What would be the better option both are priced around the same? The NAS will probably always be within 50 ft. of my main rig it would connect to.

 

@TapfererToaster

That might be a little too expensive lol. The switch alone with those ports are $900 or more even used. The adapters are around $30 as I've looked at them before..
 

 

[leadeater]

17 minutes ago, Reaper 28 said:

Wait isn't the X540-T1 a card that uses CAT cabling? I found the X520 DA1 which uses SFP. What would be the better option both are priced around the same? The NAS will probably always be within 50 ft. of my main rig it would connect to.

 

@TapfererToaster

That might be a little too expensive lol. The switch alone with those ports are $900 or more even used. The adapters are around $30 as I've looked at them before..
 

 

If the computers are close Direct Attached Copper (DAC/DA) SFP+ is better but the cables are more expensive than Cat 6/Cat6a. DAC SFP+ also has lower latency than 10GBase-T and uses less power but really won't matter much at all, not for what you want anyway.

 

Be careful however, there are two types of DAC SFP+ cables: Active and Passive. Passive cables max distance is 10m or 33ft and Active cables max distance is 15m or 50ft.

 

I use the X540-T1 cards since my server is down stairs rather far away, cable run in the wall should be around the 30m-40m mark.

[Reaper 28]

58 minutes ago, leadeater said:

If the computers are close Direct Attached Copper (DAC/DA) SFP+ is better but the cables are more expensive than Cat 6/Cat6a. DAC SFP+ also has lower latency than 10GBase-T and uses less power but really won't matter much at all, not for what you want anyway.

 

Be careful however, there are two types of DAC SFP+ cables: Active and Passive. Passive cables max distance is 10m or 33ft and Active cables max distance is 15m or 50ft.

 

I use the X540-T1 cards since my server is down stairs rather far away, cable run in the wall should be around the 30m-40m mark.

Hmm thanks. Yeah I figured SFP+ would be more expensive I was just looking at the X520... Maybe I will check out the X540 and get some Cat6a or Cat7 for the hell of it. Latency is a concern but not a huge deal I would rather have raw performance and power consumption isn't really a concern regardless. Depending on how I setup backup's and things I could end up copying 400GB+ every few days and random file dumps of 20-80GB's daily.

 

Did you have any problems setting the cards up specially in FreeNAS?. My board will have IPMI and just wondering if that would effect it in anyway even with that connected to the router.

[leadeater]

3 hours ago, Reaper 28 said:

Did you have any problems setting the cards up specially in FreeNAS?. My board will have IPMI and just wondering if that would effect it in anyway even with that connected to the router.

I don't use FreeNAS so no I didn't have a problem . Intel NICs are very well supported so you shouldn't have any issues.

[Reaper 28]

16 hours ago, leadeater said:

I don't use FreeNAS so no I didn't have a problem . Intel NICs are very well supported so you shouldn't have any issues.

Lol alright thanks. One other quick question, I know when setting up FreeNAS you get an IP address to connect to it through the UI if it's connected to my main system will it even give an IP address if the address is the same as my main rig? if that makes sense.. or would I have to connect it to my router anyways and just block file transfers so the NIC would do that. I guess using IPMI would be an "easy" alternative?

[leadeater]

You would connect a 1Gb NIC during install just to make things easier then once installed configure static IP addresses on the 10Gb NICs, you can leave the 1Gb NIC connected.

 

Use a different subnet on the 10Gb NICs than your router is configured to use.

[Reaper 28]

On 6/20/2016 at 9:05 AM, leadeater said:

You would connect a 1Gb NIC during install just to make things easier then once installed configure static IP addresses on the 10Gb NICs, you can leave the 1Gb NIC connected.

 

Use a different subnet on the 10Gb NICs than your router is configured to use.

Alright thanks for all the help, hopefully everything goes as planned