Need to figure out NAS setup for a small business

[nj4ck]

Hi guys.

 

I work at a small company that does 3D-visualizations, animations, mircrosites, that kind of thing. My boss is apparently looking to make me the designated "network guy", and I've been tasked with finding a suitable replacement for our current NAS setup. Problem is, I really have no clue what I'm doing and can only guess as to why my boss would want me of all people to oversee our networking hardware.

Anyways, my problem is that our current NAS, a 10TB box running freenas, isn't quite suiting our needs anymore. We need more storage, if possible more speed, and an easier way to back up our data. Right now, our backup "solution" is to manually copy the entire contents of the main NAS onto a WD Mycloud EX2 every friday and let that run over the weekend.

I've been looking into Rsync as a possible solution for our backup problem, but being a complete noob I'm not quite sure how to set it up yet. First, we need something with more storage and preferably more speed, and I'd like to ask you guys if you have any ideas as to what I could use.

 

To summarize my question, I'm looking for a NAS setup with ~20TB of storage, it should be as fast as possible and we need some kind of backup solution that doesn't require manually copying the data from the NAS to the backup drive. If you were tasked with setting up such a storage system for a small business, what would your solution be?

[PenPoint]

In most cases, my recommendation is Synology. Its DiskStation Manager is just great.

However if you are trying to purchase a genuine Synology NAS for small company, it's quite expensive.

I don't know exactly whether this violates copyright law, but there is a way to use Synology-tweaked OS for free. It's XPEnology.

At least as far as I know, there is a claim that XPEnology isn't illegal. NOT FOR SURE.

Using this, you can build your own NAS and just install XPEnology on it. Then it will be your perfect one.

 

Check this out if you want to get more info about XPEnology : http://xpenology.com/forum/viewforum.php?f=2

[paddy-stone]

The worst part with the scenario you descibed is the "manually" backing up over the weekend... that's just dumb AF. There is a program called Freefilesync that I use to backup data from my FreeNAS server to a WDmycloud, you can set it however you like eg check for moved files so it won't delete and copy stuff you already have, but moved from that folder (in mirror disk mode), there's lots of options, but that's the one I use and is very fast at copying across a gigabit network, mine copies large files at approx 110MB/s. This is using windows as the place where the program is run from, but if I remember correctly they also have a linux version fo the program.

 

http://www.freefilesync.org/

 

 

[nj4ck]

22 minutes ago, paddy-stone said:

The worst part with the scenario you descibed is the "manually" backing up over the weekend... that's just dumb AF. There is a program called Freefilesync that I use to backup data from my FreeNAS server to a WDmycloud, you can set it however you like eg check for moved files so it won't delete and copy stuff you already have, but moved from that folder (in mirror disk mode), there's lots of options, but that's the one I use and is very fast at copying across a gigabit network, mine copies large files at approx 110MB/s. This is using windows as the place where the program is run from, but if I remember correctly they also have a linux version fo the program.

 

http://www.freefilesync.org/

 

 

That's just what I'm looking for, thank you. As you said, our current way of handling things is dumb AF and we simply don't have anyone who knows what the F they're doing around here when it comes to this stuff.

[nj4ck]

36 minutes ago, PenPoint said:

In most cases, my recommendation is Synology. Its DiskStation Manager is just great.

However if you are trying to purchase a genuine Synology NAS for small company, it's quite expensive.

I don't know exactly whether this violates copyright law, but there is a way to use Synology-tweaked OS for free. It's XPEnology.

At least as far as I know, there is a claim that XPEnology isn't illegal. NOT FOR SURE.

Using this, you can build your own NAS and just install XPEnology on it. Then it will be your perfect one.

 

Check this out if you want to get more info about XPEnology : http://xpenology.com/forum/viewforum.php?f=2

Interesting. I was also thinking about putting together a server instead of a NAS, which would also be quite expensive. Boss wants a complete and reliable solution and I'm sure the budget will be set accordingly, so I'll look into Synology.

[paddy-stone]

No problem, it works very well for me... takes a little bit of getting used to, but once you have that it's great. It also has a real time backup type, but for me I don't like to do that just in case I make a mistake and then the mistake is mirrored on the backup device. If you have any questions I'll be happy to try and help.. I'm no expert with it, but I know what works for me

[PixelHat]

I would recommend setting up a bare-bones Ubuntu server with samba (or some other file-sharing protocol) and a software RAID 10 setup.

You can cram in as many drives as your hardware can take, and it will be lightning fast.

Security can be implemented in tons of different ways (it is Linux after all)

 

A few things to note:

• SETUP: Getting this setup will require lots of elbow grease and a bit of Linux know-how. It will be worth it. The customization and support that comes from doing something like this is invaluable.
• DRIVES: I say software RAID 10 because that can be setup independently from the actual Ubuntu installation - you can boot from a single SSD and mount the entire RAID 10 from inside Ubuntu. The RAID also can be changed without having to reinstall Ubuntu or worry about the installation itself mingling with the data. Basically the OS and the DATA are kept separate.
• EXPANTION: Expanding the storage will be a pain (not impossible, but time-consuming) - go big or go home from the beginning with your RAID drive sizes.
• TWEAKS: Any tweaks you want to do to increase speed, reliability, etc. can be done by simply SSHing into the server and popping up an internet tutorial (with permission from le boss of course)
• BACKUPS:  RAID 10 effectively backs itself up, however, there are tons (and I mean TONS) of ways to have your server automatically back itself up to an external source (a SECOND very large server with another RAID 10 setup for example). There is tons of software support (some are even natively implemented with Ubuntu (desktop)) for this kind of thing.
  • If you want to use the native Ubuntu backup program, I suggest fully installing Ubuntu (desktop, not server. The difference is in the extra-nes). That way you get a ton of super fun tools, and you can easily strip it down to what you need by uninstalling things like LibreOffice. Its a matter of personal preference.
• HARDWARE: Hardware matters. A LOT. That means you need power where it counts (for a NAS):
  • CPU: If you want more than 3 people *sarcasm* pushing files at a time, more cores will be important (look into intel's Xeon linup)
  • NETWORK: Gigabit isn't even a question. You need it. A capable router (might be overkill), network card (just google "gigabit card". 10 gigabit for super future proofing and bandwidth if you really want it), and cables are necessary.
  • DRIVES: The drives don't need to be insane. They do need to be good however, and a SATA3 connection to them is needed. I would recommend these for a NAS that operates all day. They're made to be on all the time.

If you don't want to do anything of that sort, then I'd just buy a couple of identical NASs and tell them to back themselves up. Yeah.

[nj4ck]

6 hours ago, PixelHat said:

I would recommend setting up a bare-bones Ubuntu server with samba (or some other file-sharing protocol) and a software RAID 10 setup.

You can cram in as many drives as your hardware can take, and it will be lightning fast.

Security can be implemented in tons of different ways (it is Linux after all)

 

A few things to note:

• SETUP: Getting this setup will require lots of elbow grease and a bit of Linux know-how. It will be worth it. The customization and support that comes from doing something like this is invaluable.
• DRIVES: I say software RAID 10 because that can be setup independently from the actual Ubuntu installation - you can boot from a single SSD and mount the entire RAID 10 from inside Ubuntu. The RAID also can be changed without having to reinstall Ubuntu or worry about the installation itself mingling with the data. Basically the OS and the DATA are kept separate.
• EXPANTION: Expanding the storage will be a pain (not impossible, but time-consuming) - go big or go home from the beginning with your RAID drive sizes.
• TWEAKS: Any tweaks you want to do to increase speed, reliability, etc. can be done by simply SSHing into the server and popping up an internet tutorial (with permission from le boss of course)
• BACKUPS:  RAID 10 effectively backs itself up, however, there are tons (and I mean TONS) of ways to have your server automatically back itself up to an external source (a SECOND very large server with another RAID 10 setup for example). There is tons of software support (some are even natively implemented with Ubuntu (desktop)) for this kind of thing.
  • If you want to use the native Ubuntu backup program, I suggest fully installing Ubuntu (desktop, not server. The difference is in the extra-nes). That way you get a ton of super fun tools, and you can easily strip it down to what you need by uninstalling things like LibreOffice. Its a matter of personal preference.
• HARDWARE: Hardware matters. A LOT. That means you need power where it counts (for a NAS):
  • CPU: If you want more than 3 people *sarcasm* pushing files at a time, more cores will be important (look into intel's Xeon linup)
  • NETWORK: Gigabit isn't even a question. You need it. A capable router (might be overkill), network card (just google "gigabit card". 10 gigabit for super future proofing and bandwidth if you really want it), and cables are necessary.
  • DRIVES: The drives don't need to be insane. They do need to be good however, and a SATA3 connection to them is needed. I would recommend these for a NAS that operates all day. They're made to be on all the time.

If you don't want to do anything of that sort, then I'd just buy a couple of identical NASs and tell them to back themselves up. Yeah.

Thanks for the elaborate suggestion! This looks very intriguing, but I do have a few questions:

 

1. As a complete Linux noob (my only experiences with Linux so far is tinkering with Mint for a few hours), how would you recommend I approach this? I can't exactly afford any fuckups with the company data, so I'd need to familiarize myself with the setup before actually going through with it. Do you know of any good recourses for learning this stuff? Books, websites, that kind of thing?

 

2. Could you elaborate on why exactly expansion would be so difficult? I'm fairly certain that we'll be good if I cram 20TB in there, but why would expanding the storage be more difficult with this option than on a traditional NAS?

 

3. I was thinking 12-core Xeon, 24 threads should be enough for our purposes. I wonder though, would onboard-gigabit be sufficient or should I definitely invest in a dedicated gigabit card? 

Also, I'm wondering what kind of case I should go for here. It seems only rack-mount cases have the required amount of 3.5" bays, but we don't have a server rack. Are there any tower-style cases you know of that can support this kind of setup?

 

Edit, I just realized you were talking about RAID 10, that explains the difficulty upgrading the storage. I'm thinking about going with 4 or 6 8TB WD reds for a total of 16TB in RAID 10, then using the old NAS as an external backup. That should keep the cost down while at the same time delivering the needed performance. 

[PixelHat]

On 9/8/2016 at 4:02 PM, nj4ck said:

Thanks for the elaborate suggestion! This looks very intriguing, but I do have a few questions:

 

1. As a complete Linux noob (my only experiences with Linux so far is tinkering with Mint for a few hours), how would you recommend I approach this? I can't exactly afford any fuckups with the company data, so I'd need to familiarize myself with the setup before actually going through with it. Do you know of any good recourses for learning this stuff? Books, websites, that kind of thing?

 

2. Could you elaborate on why exactly expansion would be so difficult? I'm fairly certain that we'll be good if I cram 20TB in there, but why would expanding the storage be more difficult with this option than on a traditional NAS?

 

3. I was thinking 12-core Xeon, 24 threads should be enough for our purposes. I wonder though, would onboard-gigabit be sufficient or should I definitely invest in a dedicated gigabit card? 

Also, I'm wondering what kind of case I should go for here. It seems only rack-mount cases have the required amount of 3.5" bays, but we don't have a server rack. Are there any tower-style cases you know of that can support this kind of setup?

 

Edit, I just realized you were talking about RAID 10, that explains the difficulty upgrading the storage. I'm thinking about going with 4 or 6 8TB WD reds for a total of 16TB in RAID 10, then using the old NAS as an external backup. That should keep the cost down while at the same time delivering the needed performance. 

1. I'd recommend downloading Ubuntu for a personal computer and setting up a tiny NAS at your house. It'll allow you to become familiar with the installation, setup, and maintenance of a server (which is all lower-skill-required than most people think). One thing to note - if while setting up a NAS for your work, and you do royally screw up, then you can just wipe the installation and restart! Linux is easy to "get a fresh start" with if something goes horribly wrong (during setup mind you. VERY little will go wrong once everything is up!)

 

2. RAID expansion is difficult no matter the platform - commercial, private, server, or personal desktop. RAID setups don't like it when you try to change their drive setup.

 

3. WOA. That CPU will do it for ya. On-board gigabit will do fine, but if you're packing that CPU... I would probably look into 10-gigabit (cat6 cables, fancy network switch, the whole 9 yards). You'll have a realistic bandwidth of a gigabyte for anyone doing simultaneous reads/writes.

 

Edit: That's great. That'll hold you for a while. Upgrading drives simply consists of copying your data to somewhere else, expanding the setup, and putting the data on the expanded setup. At least... That's been my experience with it. Plan ahead

[leadeater]

I would recommend using a QNAP or Synology NAS for business use over any self build/configuration options, it's just safer. You get both product support and warranty which is why there is a higher cost, a lot of people don't see the value in it since they hardly ever use their support.

 

I prefer QNAP over Synology, just a preference both are extremely good, since more of the models support 10Gb network upgrades.

 

For general file storage I highly recommend against any type of RAID 10 configurations, they are very wasteful on usable storage capacity and are NOT any safer than RAID 6. Where RAID 10 is recommended is for high write I/O applications like databases. One of the other core reasons is RAID 6 is expandable where RAID 10 is not, and expansion is very common on file servers and very uncommon on databases servers.

 

You mentioned a hugely powerful 12 core Xeon CPU, that is not necessary if all the server is going to do is NAS shares etc. These larger core count CPU's are for virtual server hosts and rendering servers, even a NAS with 10Gb doesn't need too much.

 

Also RAID is never a backup, any kind in any configuration. If the data is not going to a completely separate system it is not a backup, backups also have versions so anything that does not do that is only a replica of current data.

[Rison]

So much bad information here.

Raid 5 and 6 are super easy to expand. Raid 10 and ZFS is not.

You don't need more than 4 cores on your fileserver, if you build from scratch. Only large fiber arrays with proprietary NOS's will be multi threaded. You need ECC ram (and lots of it) versus CPU processing power in a built NAS.

Synology and Qnap have this market cornered for a reason - they are super easy to setup out of the box. Reliable and (somewhat) scale-able drive arrays, depending on which model you choose. But yeah, that comes with an expense. That just comes down to budget.

I also prefer Qnap over synology, mainly because it's cheaper when you compare them, and the 10Gb network support.

 

If you're a complete linux noob, maybe setting up a system yourself isn't a good option. 

 

As for backing up in the cloud, instead of using a program like freefilesync just use the base program from which it was made from: robocopy. Free on technet, has no overhead, runs in a dos window (set it up in scheduler to run) and has better reporting than freefilesync. I've had freefilesync crap out backing up over long periods of time, runs out of memory (even with the latest version, that comes with a virus)

 

If you don't go the premade route, i'd probably build a system based off a good single processor, 4 core xeon on a supermicro board, with 32G ECC ram, with LSI 9211-8i controller flashed to IT mode (v20) running FreeNAS. But honestly, by the time I had a case and good power supply (possibly redundant) i'm far exceeding the cost of a Qnap unit. 

 

Buying a previously used dell or HP server (which EVERYONE suggests here) could bite you in the ass, as there is no warranty (usually), no support and clearly you're not a hardware tech.. so there's a bit of a learning curve. 

I have this thing where I won't buy used components without warranty / support in a business.. it's your ass on the line when things don't work right. Don't put your job at risk because you're trying to save the boss some money and look good.. it always bites you in the ass. It's fine for your home lab or whatever, but things need to work at business or your boss is looking over your shoulder. It's your recommendation to them, so make it a solid choice.

 

As for onboard NICs, they're fine. Most server boards usually have at least two NICs so you can setup failover if you want. It's more likely your switch will fail versus a NIC, but to each his own. 

[Andster29]

I agree with with everyone who is saying Synology. It really is the best system to use when making a nas. It is super easy and they put out a lot of documentation. I would read up on it before installing. Their support is free and you get what you pay for. They have a number of systems to suit a lot of needs. Somewhere on the web is the same operating system that can be put onto most systems as long as you use sata ports. That may be an option if you want to try it for a cheaper solution. Just be prepared if it does not work. 

[BabyPandaaa]

15 hours ago, leadeater said:

I would recommend using a QNAP or Synology NAS for business use over any self build/configuration options, it's just safer. You get both product support and warranty which is why there is a higher cost, a lot of people don't see the value in it since they hardly ever use their support.

 

I prefer QNAP over Synology, just a preference both are extremely good, since more of the models support 10Gb network upgrades.

 

For general file storage I highly recommend against any type of RAID 10 configurations, they are very wasteful on usable storage capacity and are NOT any safer than RAID 6. Where RAID 10 is recommended is for high write I/O applications like databases. One of the other core reasons is RAID 6 is expandable where RAID 10 is not, and expansion is very common on file servers and very uncommon on databases servers.

 

You mentioned a hugely powerful 12 core Xeon CPU, that is not necessary if all the server is going to do is NAS shares etc. These larger core count CPU's are for virtual server hosts and rendering servers, even a NAS with 10Gb doesn't need too much.

 

Also RAID is never a backup, any kind in any configuration. If the data is not going to a completely separate system it is not a backup, backups also have versions so anything that does not do that is only a replica of current data.

Second using a QNAP - we have 2x TS-869U-RP's with 10Gbit cards in and they work excellently.

 

The first one is in a RAID10 (as everyone wants quick storage and we don't have massive amounts of it), with 8x 6TB WD Red Pros giving ~24TB of usable storage. It's connected to our 10Gbit core, and then mapped to the file server via iSCSI (also on 10Gbit), and then shares are served through the file server.

 

The second one has 8x 6TB WD Reds also giving 24TB of storage, but is used for backups (done through Veeam Endpoint). Works fairly well, and very reliable!