Reliable Data Storage on the Cheap!

[nicklmg]

Amazon: http://geni.us/3R1B
NCIX: http://bit.ly/1PMPt5Z

 

Are expensive NAS units your only option to expand your storage reliably? I think not...

 

 

 

[Dabombinable]

All I can say is, I've still got mah dual Pentium III server/legacy gaming rig.

[LeapFrogMasterRace]

Seems like a huge pain to make it driveless. In most cases it seems like it would just be better to use a normal case?

[mcr983]

I read an interesting article about how google provides massive amounts of storage through the use of inexpensive machines in clusters. It would be interesting for LTT to try and build a functional cluster out of smaller machines. Basically the opposite of 7 Gamers 1 CPU.

[KuJoe]

@LinusTech GlusterFS or Ceph build (7 CPUs, 1 cloud)!

[Deusrex]

@LinusTech You kept talking about server hardware like some people didn't like, I have to say I came in after you started showing of the crazy server builds and so far, they've been my favorites!

I'd love to see you do another UnRaid video, except with a NAS and a Home wireless router in one CPU. this is a build I was thinking about doing with some old parts I have laying around.

[Glenns]

SO....what was the performance like ??

 

Would be nice to see how your home built server compares to say a synology 4 bay with same spec drives. Test it with something interesting like iscsi to a hyper v cluster

 

also @LinusTech I'm also a fan of the server based stuff, much more fun than just tinkering with consumer grade components.

[debraj]

Hi Linus,

 

In your video you have mentioned few eBay items. Can you provide those links as well. I have a core2duo PC which I want to convert into a NAS.

 

Thanks

[PianoPlayer88Key]

Hmm... still using server hardware, ECC RAM... still over my budget for building a backup box.

 

I was looking a little at the idea of using FreeNAS and ZFS, but they strongly recommend ECC RAM.  A Celeron will support ECC, and the RAM isn't that much more expensive than the non-ECC variety, but it requires a ~$180+ server-chipsetted motherboard.  I was hoping to build one that was inexpensive enough so that, not counting the hard drives, the rest of the system would cost about what a single best bang-for-buck 5400RPM NAS drive (like a 4 or 5 TB WD Red) costs, or considerably less.

 

Ideally, since I'm primarily looking for something to do periodic (once every 3-4 months or so) full-image-of-all-storage backups, I'd like to find something where it costs considerably less per TB than a standard green hard drive, and holds everything on one logical uncompressed volume.  I'm thinking something like 20+ TB for like $100 or so, I'm guessing.  (I kind-of remember the days back in the early 1990s when my dad had a tape backup system for his home PC.  If I remember right, it could hold the hard drive a few times over on each tape, and it cost a small fraction per tape compared to what you'd pay for the hard drive.)  I'd be willing to take a performance hit, for example sequential transfer (like imaging the drives) would take 3-4 days to back up all 20 TB or so of my hard drives.  Since this would be for full-image backups only, random access speed is definitely not needed - I'd be fine with something that would make tape look like 7 Intel 750-series PCI-Express SSDs, on whichever enthusiast X99 motherboards have full x16 3.0 lanes in every slot, in Raid 0.   I've looked at tape backup systems recently, but considering the cost of the drives, it's quite expensive and for how much I'm backing up, it'd be cheaper to buy a bunch of 5-6 TB hard drives and copy everything over.

 

Is my best / only bet just getting hard drives and popping it into my PC, copying the stuff to back up, then removing the drives to store on a shelf somewhere in the box the drives came in?

 

Also, I don't know if this would work, but I was thinking of the concept of using the extra PC power in a gaming PC as a NAS, along with the recommendation that FreeNAS (and ZFS) use ECC RAM.  My own PC platform (i7-4790K, Z97 chipset, etc.) doesn't support ECC RAM, so using FreeNAS in a VM might not do very well for me.  But, what about a PCI Express x16 expansion card for like $100 or so that has a chipset on it that supports ECC RAM?  It would be a double-slot width card, kind-of patterned after a high-end GPU, with like 4 DDR4 RAM slots (DIMM if they'd fit, otherwise SODIMM), a few SATA ports, an LGA1150/1 socket (in the same place where the GPU would normally go, and you'd use a GPU cooling solution to cool it), or something like that.  (The CPU and RAM on the card wouldn't be included in the suggested $100 price, of course.)  There'd still be the issue of getting software to talk with the card, do the backups, etc.  I don't think that particular product exists, but is it possible that something could be out there that would enable you to do that adding NAS functionality to your mainstream gaming PC, and adding ECC RAM support to a platform that doesn't natively support it (like my ASRock Z97 Extreme6 motherboard)?

[chrismin1302]

I think that the video went a bit overboard with the PC. I mean, why would you need an i3 for a server? And a gigabit port isn't that essential for small files and lite tasks like streaming video and pictures. So really, even an old laptop with a few Sata to USB Adaptors would be fine. It would also be much more power efficient. Remember, you leave that thing on all the time.

[TomasV]

Hi,

Is there someone who know what is the price of unRaid?

[madtowneast]

@PianoPlayer88Key you should take a look at this board: http://www.asrockrack.com/general/productdetail.asp?Model=C2750D4I#Specifications IMO, It is perfect for use in a NAS system with some headroom for more CPU intensive things. 

[cyriouslydylan]

So I currently have a Synology DS415+, which I love. Problem is, it's only 4 drives and I'm coming up on my full capacity using 3TB drives. I have a few options: Replace all of the drives with 5 or 6 TB ones (about $1000), buy a larger NAS (looking at the 1518+ which is about $1000) that I can add more drives to so at least I can still use my 3TBs for the time being, or build my own NAS.

I'm leaning more and more towards the last option, but I was wondering if you guys had any recommendations for an operating system. I've been looking at FreeNAS, but it requires you to use drives of all the same size, and it seems difficult or impossible to add more to it later.

[arguens]

Just a thought here. How about the power consumption. I was given to understand that such "crappy" NAS actually has lower power consumption.

Also how about DYI Raspberry Pi or Banana Pi or some kind of such cheap makeshift NAS. Possible?

[TheGuyNL]

24 minutes ago, arguens said:

Just a thought here. How about the power consumption. I was given to understand that such "crappy" NAS actually has lower power consumption.

Also how about DYI Raspberry Pi or Banana Pi or some kind of such cheap makeshift NAS. Possible?

Yup, that was my thought also. Those custom systems might be fun and stuff, more upgradable etc. But you can't achieve a 10watt usage like most NAS solutions, and this is a big factor for me. This video seemed to be more like bashing on pre build systems then anything else. No real comparisons, just throwing arguments around.

[aselwyn1]

love that tower of hdd's and the random standalone wd my drive. personally love the server stuff thats been uploaded latly

[kyle1046]

2 hours ago, cyriouslydylan said:

So I currently have a Synology DS415+, which I love. Problem is, it's only 4 drives and I'm coming up on my full capacity using 3TB drives. I have a few options: Replace all of the drives with 5 or 6 TB ones (about $1000), buy a larger NAS (looking at the 1518+ which is about $1000) that I can add more drives to so at least I can still use my 3TBs for the time being, or build my own NAS.

do you mean the 1815+ ?

http://www.amazon.com/gp/product/B00P3RPMEO/ref=s9_hps_bw_g147_i2

[BrownCoat40]

I thought this was quite interesting, as I would completely agree with nearly everything. The cheap prebuilts suck, and the decent pre-builts cost as much as a computer. So why not get a computer?

   I own a WD MyCloud Ex2; one of the prebuilt 2-bay NAS units. It's as slow as a granny using a walker. For example, I needed to make a bootable usb from a 4gb .iso. I copy pasted from the NAS to my desktop, waited 30 seconds, went to the windows site and downloaded it, and the download finished 2 minutes before the NAS copy finished. You can literally download something faster than transferring it over the cheap NAS units. I regret that I spent $150 on that P.O.S.

   And if you're going to spend $600 on a NAS box (not including drives btw), why not get a full computer? Yes they aren't as power efficient, which Linus didn't mention in depth. But here's that depth: If you figure 4 drives at 10W each, and 15W for the prebuilt, thats 55W. Last time I checked, my rig's i3 and mobo only pull about 30 watts from the wall. If I re-purposed the system for a NAS, say adding 5W for the LSI card and 10W each for 4 drives, the system takes 75W. The difference is only 20W, and practically no difference in hardware price. 

   So take your pick; do you want a prebuilt that has a 20W power saving, but limited ability to upgrade and no replaceable parts (minus drives of course), or do you want a full computer where every component is independently upgradable and replaceable? I assume that if you're on this forum, you'll understand that no ability to upgrade or replace components can easily cost more in the long run (as I was recently reminded of when my wife's Macbook Pro bit the dust).

[GodAtum]

So why no freenas recommendation. I thought unraid was worse?

[VictorB]

Sorry Linus. Way to complicated approach to build a cheap fileserver.

 

I have made an tutorial to use old hardware for ZFS. I explain every item you need in much more detail.

 

You dont need a complicated server case. better use a decent miditower much mroe silent much more normal parts.
 

 

[azazelpy]

i think this is more simple

http://www.ebay.com/itm/Supermicro-2U-12-Bay-Server-Intel-Xeon-E5410-2-3GHz-Quad-Core-4GB-12XHD-Tray-/141897324324?hash=item2109bd5724:g:iZ8AAOSwo0JWO6Vk

[Dybre]

might be more simple but have you ever had a 2u server running in your home they are loud, i had one and dumped it for a synology DS1815+ will never go back

[brighteyes890]

Holy crap at the all salt on YouTube!

[site_owner]

Hi.

 

Your video caught me in the middle of some maintenance in my DarkTower So I thought I will brag about it.

- Old Intel Server MB (8 years!) with old cpu. I will replace it this year for something new.

- Two hardware raid cards (adaptec) but I use software raid (more on that below).

- sharkoon rebel12 big tower case.

- two arrays: 4x3tb and 6x1tb. Some room to add new drives. I will migrate old array to 3tb in next two years.

- additional disks.

- I am adding two ssd's for OS. I will probably use hardware raid0 just for speed.

 

I use this as my Lab. I am a IT contractor, developer and I focus on small "big data" processing - up to 20tb. I am getting around 400MB/s on old array and around 500MB/s on new. For now: 7 and a half years without fail - since raid card fail in the beginning. 2 drives failed in that time. Raid5 FTW!!!!!

 

Some tips regarding raid:

- create more than one array. If something goes wrong you will loose only part of your data

- use software raid.  In software raid metadata about an array are stored on drives. In hardware raid they are stored in the controller. I know I have hardware raid but when one of my cards failed I had to order new card to access any data! With software raid you can connect your drive to any computer, boot Linux - even from usb - and access your data. For "home" use software raid will be good enough. If you need raw power then use hardware raid.

- plan big. Use big case because when you see how easy is to just add a drive you will do it There are some NAS like cases (google "u-nas NSC-800") and if you can get one then do. If not use big tower. Rack mounted are pain in the a. if you don't have the place to mount them. I had one then I switched to big tower.

- raid is not an excuse to not have any backup. DO backup. NOW!

- you can prepare compress-encrypt-send-to-aws-glacier.sh really easy. AND DO BACKUP!

- do not install additional software on that box. I use it only as storage. Everything else I do on my workstation. If I need something operating on that data I use containers. If this system crash you might loose some data. Less software = less chance to crash.

- never skip any check in any tool. Creating array needs some time? GOOD.

- monitor! check logs! be sure that array is ok!

- use UPS

- try and learn tools that you need to use. When you are done with hardware assembly create your first array. Put 10gb on it, calculate sha1. Remove one drive. rebuild. Switch cables. Learn. Check sha1 of that files again.

- never use physical, full disk as a member of an array. Use partition. 3tb from WD is X sectors. 3tb from Seagate is Y sectors. You are in trouble now. If you use ZZZZ GB from both drives then there is no problem.

 

That's it for now. Any questions ?

[Dabombinable]

1 minute ago, site_owner said:

Hi.

 

Your video caught me in the middle of some maintenance in my DarkTower So I thought I will brag about it.

- Old Intel Server MB (8 years!) with old cpu. I will replace it this year for something new.

- Two hardware raid cards (adaptec) but I use software raid (more on that below).

- sharkoon rebel12 big tower case.

- two arrays: 4x3tb and 6x1tb. Some room to add new drives. I will migrate old array to 3tb in next two years.

- additional disks.

- I am adding two ssd's for OS. I will probably use hardware raid0 just for speed.

 

I use this as my Lab. I am a IT contractor, developer and I focus on small "big data" processing - up to 20tb. I am getting around 400MB/s on old array and around 500MB/s on new. For now: 7 and a half years without fail - since raid card fail in the beginning. 2 drives failed in that time. Raid5 FTW!!!!!

 

Some tips regarding raid:

- create more than one array. If something goes wrong you will loose only part of your data

- use software raid.  In software raid metadata about an array are stored on drives. In hardware raid they are stored in the controller. I know I have hardware raid but when one of my cards failed I had to order new card to access any data! With software raid you can connect your drive to any computer, boot Linux - even from usb - and access your data. For "home" use software raid will be good enough. If you need raw power then use hardware raid.

- plan big. Use big case because when you see how easy is to just add a drive you will do it There are some NAS like cases (google "u-nas NSC-800") and if you can get one then do. If not use big tower. Rack mounted are pain in the a. if you don't have the place to mount them. I had one then I switched to big tower.

- raid is not an excuse to not have any backup. DO backup. NOW!

- you can prepare compress-encrypt-send-to-aws-glacier.sh really easy. AND DO BACKUP!

- do not install additional software on that box. I use it only as storage. Everything else I do on my workstation. If I need something operating on that data I use containers. If this system crash you might loose some data. Less software = less chance to crash.

- never skip any check in any tool. Creating array needs some time? GOOD.

- monitor! check logs! be sure that array is ok!

- use UPS

- try and learn tools that you need to use. When you are done with hardware assembly create your first array. Put 10gb on it, calculate sha1. Remove one drive. rebuild. Switch cables. Learn. Check sha1 of that files again.

- never use physical, full disk as a member of an array. Use partition. 3tb from WD is X sectors. 3tb from Seagate is Y sectors. You are in trouble now. If you use ZZZZ GB from both drives then there is no problem.

 

That's it for now. Any questions ?

 

I have to say, I'm still not comfortable using HDD on their side after past experiences with HDD suddenly failing.