Ahnzh

Introduction:
 
So what i want to do here is getting a good conversation started about general cloud services. At first i want to point out what a cloud is, what it can be used for and a comparison to NAS. Since I plan on having a conversation here I'm focussing on expanding this over time and round it up with an extensive guide on how to build the 'NAS with cloud services' of everybody's dreams (based on Linux). This will be hand-in-hand with my NAS build and network expansion over the next 1-3 months
 
 
General:
 
It seems that the term 'cloud' consistently the same for everybody. Here's a short list of what private users think, that a cloud is:
A centralized storage for all your data and you can access it from your devices of basically from everywhere A 'Push' service that pushes data to the different devices but without storing the data on a server A combination of services like Calendars, Email, Storage, Collaboration for multiple users Some sort of 'Backup' Now let's get to business people's understanding of a cloud:
Infrastructure as a service Platform as a service Software as a service The private usages are pretty much self explanatory but I think i have to get into detail for the business people's point of view because this explains pretty well what a cloud is based on. 
 
 
'IaaS' - Infrastructure as a service
 
Infrastructure as a service means that you can create your own infrastructure. That means, storage, virtual servers, VLANs and let your applications run on them. Basically it's as if you would buy servers, place them somewhere at work, configure them and let your stuff run on it, just more flexible because you can freely add and remove servers since they all run virtually.
 
 
'PaaS' - Platform as a service
 
This is more about running processes. Think about it this way: you have a bunch of program that you want to run on external hardware. You are not interested in the hardware it runs on, you want it to run and that's it. This is basically needed if you are running a program you created yourself and don't want to create things like a backup server, one or two servers on which VMs run, then configure it and so on. Platform as a service does this automatically.
 
 
SaaS - Software as a service
 
You don't have a program but want  a certain functionality. Let's say you want an exchange server. So you just rent it. You don't have to bother with anything like installing, setting it up or integrating it into other services. It just runs. You could call it 'software on demand' as well.
 
 
 
So the conclusion is: A cloud can be anything as long as it's scaling and flexible. So as a user you can either use it as a backup, as an email server, you can use it which way you want and tune it to your needs. 
 
Amazon/Microsoft Azure/Google Cloud are pretty much the leading global players in the business area. What they do is provide a wide range of services from IaaS to SaaS without needing to build actual infrastructure. It starts somewhere in the middle of IaaS and PaaS, you chose Infrastructure packages like Databases or Storage (without having to configure it from scratch), then create dependencies and let applications run on it. It goes as far as being able to run SaaS apps.
 
But most of them are running consumer cloud services as well, based on SaaS. So you got Amazon Glacier as backup, the Google squadron of Drive, Mail, the old Picasa (now Google Fotos), and whatever there may be. The functionality of these web apps is basically what the common person understands what a cloud is, even though it's a bit misleading. 
 
Recently (100% because of NSA) I regularly hear something like 'I want a cloud. At home! MY precious data, not the NSA's!' so I created this thread since nobody really knew what they actually wanted. With all those NAS devices which are advertised as 'home clouds' or something similar the questions kept coming: can my cloud do this, can my cloud do that. I told them that I would prefer saying 'can my NAS do this or can my NAS do that' and 'A NAS cannot become a cloud, it merely provides services that clouds do'.
 
---CONFUSED FACES---
 
So how becomes a cloud 'a cloud'? It's a combination of various things. Clouds run virtually. Physical hardware is derived into logical hardware. Clouds run on this logical hardware. What's the use? High availability and redundancy. You are able to run multiple instances of virtual machines on different servers / clusters / data centers. This provides failover. Additionally it makes it easy to scale and you are only paying what you actually need. This is not provided by a simple NAS. All you get are services that are provided by clouds as well. But a NAS has got advantages: speed and privacy. You cannot run big apps on them but they are located at home in your speedy LAN and nobody has access to it. What you are missing though is redundancy. If your NAS fails there is a risk of losing data. So, is there a way to build a proper home cloud? yes! easy? no! cheap? no!
 
 
Services:
 
So for now: which services do you expect a NAS to have. What would you like to see here, your point of view, spam me ;D And which Linux Distro do you want to see this running on?

What is this... @Wiff how do get from @pixels8's pretty good comparison to 'WD is better'. WD is faster, not better... We are talking about 3-5 MB/s. When running drive optimized benchmarking. In real world scenarios it's 1-2 MB at max if you are lucky. In 99,999% it will be unnoticable for users. The only advantage I see is a higher warranty and a stronger mechanical build, which comes at the cost of a large price increase. 190€ instead of 130€... Faster, stronger, better performance, yep, but just compare the prices. When looking at comparable prices you get at least 2 TB for the difference.

the thing with amahi is: it's hardcoded to 50%, you are forced to use your server as your DHCP and DNS server. If you do that, everything else will work. 100%. disable your dhcp and dns in your router and the settings are done automatically.

pretty much yeah the cheapest solution would be getting an old pc (2-3 years) and put some HDDs into it. Then get Amahi (since it's foolproof to set up) and enjoy a NAS that's pretty flexible AND does perform (more than 30MB/s)
 
If you are confident in your IT skills, go for FreeNAS, else opt for Amahi.

Probably the core2duo. Check Ebay. Only buy from trusted sellers with Ebay's buyer guarantee thing. You could also build a cheap Pentium system.
Not really. You don't have to use FreeNAS. Amahi is good too and would be best for this setup imo. FreeNAS wouldn't be able to use plugins with that kind of machine while Amahi would. 
Not very. If you need specific guides, I can help. Amahi is built for simplicity and ease of use. FreeNAS is built for versatility and moderate to extreme complexity. It can be simple, but it's not the primary purpose of the OS, but that is Amahi's primary purpose. 
I recommend Amahi over the other OS, specifically, in this order: Amahi > FreeNAS > Linux based OS > Windows Home Server > Windows 20XX Server in terms of What is best for this build?
 
That's considering that you want simple based on what you've wrote.

Baldur - Germanic god of wisdom
Build Log




Introduction

Hey guys, thanks for visiting at my log. First off: English isn't my native language so if somebody finds a mistake he can keep it
 
Let's start with the reasons to build a case from scratch.
My requirements for a case were:
Capacity of 12 SATA/SAS 3.5in drives HotSwap m-ATX Motherboard size 210mm/8.2in width (can be less, but 210 at maximum) Up to 550mm/21.6in maximum height Up to 600mm/23.6in maximum depth Quiet Style because it's going to be located in my living room I was searching for a case for a longer time and finally found one (iStarUSA S-919) but there are no distributors in Germany so i wasn't able to buy it here. But i really wanted it so i wrote some US friends, asked people in this forum, wrote to iStarUSA directly, talked to iSG (iStarUSA appeared to be a distributor rebranding the product), wrote to different shops (Newegg, various US located shops) ... to keep things short there was no way of getting my hands on one of those cases. So i spent some additional days looking for a proper case but didn't find one that met my requirements. Either they were too large in size or m-ITX, or they had 6 or less drive bays. There simply was no reasonable compromise, so building a case by myself is the only possible way to go.
 
 
 
What not to expect
 
Flashy inside LED, graphics power, high end gaming power, water cooling! This is going to be a Data Server. It's going to do it's job quietly and with good performance. So please don't expect stuff that usually belongs into gaming rigs. That doesn't mean though that i don't use any LEDs. Things like HDD drive activity LEDs and a bit of external designing is going to happen. There won't be a side window though.
 
 
 
What to expect
 
I'm not limiting myself to the building of the case so expect benchmarks, some guides for building a Linux NAS and so on. There will be lots of tinkering, smoldering, Aluminum welding (yes, it's difficult but I was in an apprenticeship as a toolmaker/precision engineer in GERMANY before i did my master in computer science, I learned welding Aluminum),  let's say expect craftsmanship. I've got some things i want to do that might surprise some people, including custom made hot swap drive cages (I'm producing the SAS backplanes by myself I found THESE, which do exactly what i wanted to do, just a simple backplane connector with LED pins based on the 11th pin in SAS/SATA drives which give out drive activity. Since they run on 300 micro Ampere the signal needs to be amplified and this single drive backplane does exactly that. I will come up with something else instead, stay tuned). So feel free to follow. I will update the log at least once a week.
 
 
 
Hardware
 
Cpu: Intel® Xeon® Processor E3-1285L v3 (8M Cache, 3.10 GHz)
RAM: 4x Kingston ValueRAM 8GB DDR3 PC3-12800 CL11 (KVR16E11/8) - it's a PDF
Motherboard: Asus P9D-M
SSD: 4x Samsung 840 PRO 2,5 Zoll SATA 256 GB
HDD:12x Seagate Constellation ES.3 7200RPM SATA-6Gb/s 2TB - again, it's a PDF
PSU: BeQuiet Straight Power E9 580W with Cable Management, Workstation/Server certified
Backup: Synology RS814+ with 4x4TB Seagate NAS drives (Server Housing)
Host Bus Adapter: LSI Host Bus Adapter SAS 9201-16i (16x SAS)
Network Card: Intel x540-T1
 
 
 
Software
 
It's going to run Linux. I will do a step to step documentation (expect a lot of stuff here as well)
 
 
 
Network
 
It's going to be integrated into a 10GbE network, so expect benchmarks (iSCSI transfers, general SAMBA performance, IOPS, everything in detail)

Hi, could you please post from where are you getting these ?
 
Edit: never mind i have found them if anyone is interested check this out http://pactech-inc.com/html/pcba-backplanes.htm

Yes, you do indeed.

Since you're running ZFS on FreeNAS, I'll let Vitalius speak on that
one. Personally I've been running ZFS on Linux for about a year now,
and from my experiences with that I would be rather surprised if adding
a third 16 TB set would lead to instabilities and such. Depending on
your amount of RAM, speeds might suffer a bit, but that's about it
(again, FreeNAS might be a bit different on this).
 

That's the usual recommendation. FreeNAS folks seem to be very adamant
about it. While I have indeed come across the 1 GB / TB recommendation
with ZOL as well, I haven't really seen anyone be truly strict about it.
I ran a ZOL setup with 4 GB of RAM and 17 TB of raw storage space for
about nine months (so, way below the recommended amount of RAM). The
only negative effect I could notice was that when I was doing something
on that system which started eating up lots of RAM, ZFS speeds would
noticeably drop (ZOL's memory management will take up as much RAM as it
can up until a defined limit, but free it again when other processes
request RAM). I never encountered any reliability or stability issues
though, it only seemed to affect speed (and for the most part, the pool
was still fast enough to saturate a gigabit link, so I didn't really
care too much because that was a bottleneck I couldn't get around anyway).
I'm not sure why FreeNAS seems to be stricter about this. Maybe there are
actual differences in memory management between ZOL and ZFS on FreeNAS,
maybe ZOL just isn't popular enough yet for people to really have noticed
this issue, maybe it's something completely different, not sure.
Oh, forgot, @Ahnzh might be interested in this as well, sorry for forgetting
you earlier.

I kept on calculating a lot of stuff lately. With traditional server drive bays i would run into problems pretty fast because of the 210mm server width. Today I ordered 2 of these drive bays to figure out if this will work:
 

 
Maybe I have to fix the drive orientation from vertical to horizontal because of this 

oh and I figured that it's hard to understand what it will look like in the end. here's some further detail:
 

  It's just a rough design focussing on dimensions. it's 1:1 for the parts that you see, i have to add a PSU and fans to the design sometime but it should show you the dimensions of everything

as a 'smaller' update i wanted to show the pcb boards that I'm using for the drive cages. I received them some time ago but never published the pictures.
 

So here is the schematic:
 

 
To explain it a bit: To the top left and bottom right there are ventilation areas. The smallest boxes are fans. The slightly larger (horizontal) boxes are HDDs.
 
What you see on the bottom left part of the drawing is the hardware layout and a front hatch that is opened towards the TOP. When it's opened the whole system is able to slide out to the front, including the fans, the motherboard, the PSU and the drives.
 
As of now the drawing is already outdated since i realized i will have to integrate some sort of weight in order to prevent flipping the case to the front side when i'm sliding the hardware out to the front but it will give you a general understanding of what I'm going to do.
 
 
/edit: if needed i can color the parts and give you a color key, but i think it shouldn't be needed

It's going to be used semi-professional so the L-Xeon was the better option. I mean, it's going to be used like 9-10 hours a day. While being used it's getting stressed so i went with the most powerful L-Xeon. Additionally core clock is important because SAMBA still is based on a single thread. More Power per core means more data transfer, building a machine with 12937468 cores does not give me that much of a profit.
 
I gave Lian-Li parts a thought. I most likely will go with a m-ATX Motherboard tray (I can buy it separate) but i expect to do the rest by myself. If i'm doing it myself I am not accepting a compromise. The thing is, that i am getting LOTS of HDD for a pretty small size. I did a rough design drawing and I'm going to publish it during the next minutes. I promise you will be amazed.

Hlin - Germanic god of protection
Build Log
 
 
 
 
Introduction
 
Hey guys, I'm going to do a small build focussing on network protection. I've been testing various routers, firewalls, modems (ZyXEL, FritzBox, Draytek, Asus routers, Gateprotect Business Firewalls for 2500€, Lancom UTMs, even Linux based stuff like m0n0wall and pfSense) and whatever but I never was satisfied with what  got in the past. I found something interesting last night: Sophos created a business grade UTM software suite for private use. And they give it to you for FREE. It's a complete UTM solution containing a hardware firewall, gateway antivirus (All data you receive from the internet gets checked for viruses and malware), e-mail antivirus/spam detection, VPN solutions and many more. Usually you have to pay 2000€ for a solution like this and subscribe to plans like antivirus, malware protection and all this stuff for 500€ish per year. So i went for a build instantly.
 
 
 
What not to expect
 
This won't be a fancy build. It's going to be integrated into a small HTPC case, passively cooled, external PSU, no custom configurations. Do not expect anything you usually see here.
 
 
 
What to expect
 
I don't think this kind of stuff has been here before. It's a self built Unified Threat Management solution and security is becoming really important. If you want to raise your security level at home this can be interesting. I will test the antivirus scan latency, general throughput, how it will act with games, video streaming, how good it will block threats, configuration in general. 
 
 
 
Hardware
 
CPU: Embedded Intel Avoton (Server Atom) C2550 Processor, Quad core, 2,4 Ghz/2.6 Ghz OC, 14watts
RAM: 2x 2GB Kingston 1600MHz  CL11 ECC RAM
Motherboard: SuperMicro A1SAi-2550F Embedded Server Board
Case: MS-Tech CI-57 with external 60W PSU
 
 
 
Software
 
Sophos UTM Home Edition

Hey guys. When I was looking for Linux based router/gateway solutions I stumbled over the Sophos UTM home edition.
 
http://www.sophos.com/en-us/products/free-tools/sophos-utm-home-edition.aspx
 
 
I just wanted to share this with you since you get:
 
A hardware firewall Gateway antivirus Multiple WANs Much more A business grade UTM solution All of the above for FREE! It looks pretty insane, I have no clue how they do that but this really is an impressive package and instead of looking for pfSense or m0n0wall this thing is the real deal. 
 
It's limited to 50 IPs in your network but let's be honest, for home or small business use you will never reach more than 30ish IPs. 
 
 
I ordered an SuperMicro Board with an embedded Avoton (Server-Atom) C2550, 4GB RAM and a small SSD today to test this out since it really impressed me.
 
 
What do you think.

Usually the other way around for me.
Depends what Sandisk SSD, if it's the X210, go for it - it's an excellent drive and will last longer than the Samsung (and if it doesn't, it has a longer warranty. 5 years as opposed to 3). The SanDisk Ultra SSDs aren't great performing though, so stick with the X210 or Extreme IIs.
And the Samsung isn't bad either, but probably not what I would take.

As you can see on buffalo's website, maximum speeds are 480 Mbit per sec.

http://www.buffalo-technology.de/de/produkte/speicherprodukte/nas-fuer-privat/linkstationtm/archive/linkstatio-pro-duo/

That makes 60 MB per sec max under optimal circumstances. For small files it's not surprising to see transfer speeds drop this much, especially when you realize that writing is slower than reading. So 3-25MB depending on file size is what can be expected
 
in the end you went for a budget NAS.

Yes it is most likely due to the NAS' SoC/Processor being too slow for higher throughput. I have noticed this with out WD MyBook Live over a fast network that wouldn't bottle neck it. If trasferring lots of tiny files, the speed drops below 10MB/s. With bigger file's it can do much more tho.
 
Yours does look awfully slow. There may or may not be another problem.

greens aren't that fast. copying small files (like pictures) takes a longer time since it reads data all across the drive. with 5400rpm drives it takes really long to position the head to read data. this takes its toll. If you want performance, go for a seagate barracuda (for budget) or a wd black (for performance)

that's it. You need to check for larger files. Your HDDs got a small cache storage integrated that buffers small writes.

The network connections in and around the speedtest servers might be getting hit hard.

You don't need an i7, nor an SSD to get gigabit speeds. Whenever you download something, it caches it to RAM before it writes (it only caches a certain amount). Processor is also near irrelevant since nowadays all processors are powerful enough to handle sending a command to write 200MB/s of data.

oh and I figured that it's hard to understand what it will look like in the end. here's some further detail:
 

  It's just a rough design focussing on dimensions. it's 1:1 for the parts that you see, i have to add a PSU and fans to the design sometime but it should show you the dimensions of everything

Baldur - Germanic god of wisdom
Build Log




Introduction

Hey guys, thanks for visiting at my log. First off: English isn't my native language so if somebody finds a mistake he can keep it
 
Let's start with the reasons to build a case from scratch.
My requirements for a case were:
Capacity of 12 SATA/SAS 3.5in drives HotSwap m-ATX Motherboard size 210mm/8.2in width (can be less, but 210 at maximum) Up to 550mm/21.6in maximum height Up to 600mm/23.6in maximum depth Quiet Style because it's going to be located in my living room I was searching for a case for a longer time and finally found one (iStarUSA S-919) but there are no distributors in Germany so i wasn't able to buy it here. But i really wanted it so i wrote some US friends, asked people in this forum, wrote to iStarUSA directly, talked to iSG (iStarUSA appeared to be a distributor rebranding the product), wrote to different shops (Newegg, various US located shops) ... to keep things short there was no way of getting my hands on one of those cases. So i spent some additional days looking for a proper case but didn't find one that met my requirements. Either they were too large in size or m-ITX, or they had 6 or less drive bays. There simply was no reasonable compromise, so building a case by myself is the only possible way to go.
 
 
 
What not to expect
 
Flashy inside LED, graphics power, high end gaming power, water cooling! This is going to be a Data Server. It's going to do it's job quietly and with good performance. So please don't expect stuff that usually belongs into gaming rigs. That doesn't mean though that i don't use any LEDs. Things like HDD drive activity LEDs and a bit of external designing is going to happen. There won't be a side window though.
 
 
 
What to expect
 
I'm not limiting myself to the building of the case so expect benchmarks, some guides for building a Linux NAS and so on. There will be lots of tinkering, smoldering, Aluminum welding (yes, it's difficult but I was in an apprenticeship as a toolmaker/precision engineer in GERMANY before i did my master in computer science, I learned welding Aluminum),  let's say expect craftsmanship. I've got some things i want to do that might surprise some people, including custom made hot swap drive cages (I'm producing the SAS backplanes by myself I found THESE, which do exactly what i wanted to do, just a simple backplane connector with LED pins based on the 11th pin in SAS/SATA drives which give out drive activity. Since they run on 300 micro Ampere the signal needs to be amplified and this single drive backplane does exactly that. I will come up with something else instead, stay tuned). So feel free to follow. I will update the log at least once a week.
 
 
 
Hardware
 
Cpu: Intel® Xeon® Processor E3-1285L v3 (8M Cache, 3.10 GHz)
RAM: 4x Kingston ValueRAM 8GB DDR3 PC3-12800 CL11 (KVR16E11/8) - it's a PDF
Motherboard: Asus P9D-M
SSD: 4x Samsung 840 PRO 2,5 Zoll SATA 256 GB
HDD:12x Seagate Constellation ES.3 7200RPM SATA-6Gb/s 2TB - again, it's a PDF
PSU: BeQuiet Straight Power E9 580W with Cable Management, Workstation/Server certified
Backup: Synology RS814+ with 4x4TB Seagate NAS drives (Server Housing)
Host Bus Adapter: LSI Host Bus Adapter SAS 9201-16i (16x SAS)
Network Card: Intel x540-T1
 
 
 
Software
 
It's going to run Linux. I will do a step to step documentation (expect a lot of stuff here as well)
 
 
 
Network
 
It's going to be integrated into a 10GbE network, so expect benchmarks (iSCSI transfers, general SAMBA performance, IOPS, everything in detail)

I'm a bit distracted so it's understandable if you didn't understand what I wrote, my GF is pregnant and the birth is just a question of 48 hours. She's making my live a living hell. Today we were in a movie. We said 'ok it will be the last movie for a long time'!
 
It ended with she first destroying my balls during an uterine contraction and then after I shouted out loud she started insulting me as an impotent fucker in some kind of screaming voice! In the middle of a movie! DEAD SILENCE! GOD please let this end!
 
 
I suffered so much already!