100TB in this TINY case

[sloniupl]

Do you know the model of m.2 to sata adapter they use in the video? 

[Redselk]

I am in the process of converting my old pc into a NAS. It only has gigabit LAN. At only 125MB/s, would optane used as a cache even be noticeable?

[AbydosOne]

Just now, Redselk said:

At only 125MB/s, would optane used as a cache even be noticeable?

I'd hazard to say that, in general, unless you already know you need it, never bother with caching in a NAS.

[Redselk]

5 minutes ago, AbydosOne said:

I'd hazard to say that, in general, unless you already know you need it, never bother with caching in a NAS.

In the video, Linus used a 32gb optane drive as a cache to help with transfer speeds. However, he was using 2.5Gb ethernet. I was wondering if I am only on 1Gb will the benefits of optane be noticable?

[t3ch_n1nj4]

15 hours ago, Jarsky said:

 

Cache drives / L2ARC arent needed in 99% of home use cases. When FreeNAS first became popular it used to be a big thing because memory was limited with most systems being around 4GB of ram, SSDs were still in their infancy and HDD's were often far slower than modern drives. Technology has advanced a lot since those Athlon64 & Pentium 4 days. If you're running remotely modern hardware with 16GB of memory, then the automatically allocated 8GB of ARC is enough to take care of advanced functions like scrubbing / silvering / deduplication. Cache is rarely helpful for home users where the raid is *fast enough* for throughput and latency. 

This TrueNAS install isn't for home use, but for a small office with 6 workstations that wants backups for each system, along with the capability to share select files by way of a self-hosted cloud like with Nextcloud.

I'd like to think having a L2ARC cache drive installed in a build for this type of operating environment would be of benefit, but there doesn't seem to be a clear consensus for either or based on the TrueNAS/HomeServer forums I've dug through.

So you're saying having 16GB RAM installed into a system with a decent CPU would be sufficient for its' intended purpose?

[t3ch_n1nj4]

4 hours ago, GoodBytes said:

This project is all nice and neat, however what Linus forgot to mention is that with Synology, part of the high price, is the built-in software.

If you only need a network drive, then yea, great. But if you need, say, picture gallery app that can be used remotely, via phones and PCs then sadly you are limited in options. OneCloud kinda has a gallery, but it acts more like a view mode where you see ALL pictures without categories, permissions, and so on. And the rest, to my knowledge. don't.

 

 

Nextcloud is a great open source option for hosting your own personal cloud, and works either with a self-hosted TrueNAS installation or on your own web server.

I've been using Nextcloud on my own web server for nearly a year, and a huge fan of the product. It makes for easy work to share my files and folders with specific people, and with specific rules/permissions for added control of how your data is shared.

[AbydosOne]

20 minutes ago, t3ch_n1nj4 said:

I'd like to think having a L2ARC cache drive installed in a build for this type of operating environment would be of benefit, but there doesn't seem to be a clear consensus for either or based on the TrueNAS/HomeServer forums I've dug through.

So you're saying having 16GB RAM installed into a system with a decent CPU would be sufficient for its' intended purpose?

The way that the L2ARC functions means that only data that has already been in the RAM cache and has aged/pushed out will end up there. The ARC itself is somewhat intelligent and will only cache blocks (not files, mind you) that are repeatedly accessed in a short span of time, as well as some blocks around that data. The only real workload I could see an L2ARC becoming useful is a large database server, where the full dataset is a) larger than the RAM cache and b) needs to be accessed randomly (not streaming large files). Most home users are either backing up and rarely reading, or are reading large sequential files (media) not that often. The speed of individual HDDs of today is easily over 150MB/s each, meaning an array of multiple drives is even better. Even a single drive can outstrip gigabit, and a RAID1 pair can come close to filling 2.5G during large file transfers.

 

That's why I don't recommend L2ARC.

[t3ch_n1nj4]

38 minutes ago, AbydosOne said:

The way that the L2ARC functions means that only data that has already been in the RAM cache and has aged/pushed out will end up there. The ARC itself is somewhat intelligent and will only cache blocks (not files, mind you) that are repeatedly accessed in a short span of time, as well as some blocks around that data. The only real workload I could see an L2ARC becoming useful is a large database server, where the full dataset is a) larger than the RAM cache and b) needs to be accessed randomly (not streaming large files). Most home users are either backing up and rarely reading, or are reading large sequential files (media) not that often. The speed of individual HDDs of today is easily over 150MB/s each, meaning an array of multiple drives is even better. Even a single drive can outstrip gigabit, and a RAID1 pair can come close to filling 2.5G during large file transfers.

 

That's why I don't recommend L2ARC.

This server would be for system backups and storage of/access to files. No databases or streaming of videos, but videos along with images would be part of the data being stored and intermittently accessed. Files that could/would be accessed on a daily basis are mostly office documents, so that could work with the L1ARC that's provided by way of RAM.

Well, that's two opinions against the use of a L2ARC cache drive. Looks like I'll be building it as a mirrored 10TB WD Red Pro array with a 128GB SSD boot drive.

Now, to find a good SFF case that'll support 2 x 3.5" HDDs and one 2.5" SDD, and build out the rest of the system from there. Either that, or find a good used SFF system that'll meet my needs and go the eco-friendly route.

[Nystemy]

3 hours ago, Redselk said:

In the video, Linus used a 32gb optane drive as a cache to help with transfer speeds. However, he was using 2.5Gb ethernet. I was wondering if I am only on 1Gb will the benefits of optane be noticable?

Maybe a bit for access times. HDD's aren't known to be the fastest things in the world. And perhaps one configures the NAS to spin them down if not in use to save on power. Then a cache could be even more impactful on access time for common files. (uncommon files would though still take a sweet minute since they aren't cached.)

But other than that. A single hard drive tends to bottleneck on a gigabit connection by itself. _{(at least almost all the HDDs I own can reach speed of 150-260 MB/s transfer speeds between each other.)}
A RAID array of a couple of drives tends to work on files in a bit more parallel fashion, and this can even more easily bottleneck even faster interfaces. It isn't all that hard to even make a 10 Gig connection seem slow compared to the array of drives.

But as stated, caching is often more about reducing access time rather than increasing speed.
Though, some NAS servers have a write cache so that it can write stuff out to disk later when it isn't occupied with reading, and this can noticeably increase performance.

Then there is also the question of how much bandwidth one actually needs. 125 MB/s is decent enough if all one stores is video, pictures and other documents. For a game library more can though be nice.

[Jarsky]

3 hours ago, t3ch_n1nj4 said:

This server would be for system backups and storage of/access to files. No databases or streaming of videos, but videos along with images would be part of the data being stored and intermittently accessed. Files that could/would be accessed on a daily basis are mostly office documents, so that could work with the L1ARC that's provided by way of RAM

L2ARC would be pointless in this scenario for your number of users. The most advantage would be for the images if the same images are being access constantly. The latency could possibly help with large Excel spreadsheets potentially (data driven workloads), but you'd need to have a large number of concurrent users for this to really be beneficial. ZIL/SLOG is largely in the same boat in that its very situational, you would need to be doing extensive synchronous writes for this to be beneficial. Consider LTT, they work with very large media files over 10Gbit+ networks, so it makes sense for them pushing large amounts of contigous data (large video files). 

 

The use of SSD's for L2ARC (Read cache) and ZIL/SLOG (Write cache) is a pretty well discussed topic, and the concencous that you're better to add more RAM if you can, than to default to a SSD for L2ARC. Keep in mind that if you have an L2ARC you still need room in ARC (memory) for index entries, so there is a relationship around optimal L2ARC vs ARC, typically about 5:1

 

So lets say you have 16GB of Ram, and that 8GB is allocated to ARC. Your optimal L2ARC size in that case is 32GB. But this is inefficient as far as computing, as you're burning your ARC with Indexing, and you're increasing the load on the CPU. Instead you'd be better moving to say 64GB of Ram to boost your ARC. It will greatly increase what is available to all functions of ZFS, as well as increase efficiency (lower CPU utilization). 

 

If you really do want to go with L2ARC though, then stick with lower level nand for endurance. So an Intel Optane or a Samsung Pro with SLC cache, or something with DRAM cache.

[Jarsky]

P.S this is the sort of stuff we discuss in the Servers and NAS sub forum if you ever need help. It's full of those of us that work as IT Professionals ranging from small to enterprise size companies. 

[t3ch_n1nj4]

48 minutes ago, Jarsky said:

P.S this is the sort of stuff we discuss in the Servers and NAS sub forum if you ever need help. It's full of those of us that work as IT Professionals ranging from small to enterprise size companies. 

I have since come to realize how expansive the LTT forums really are.

 

Been watching the channel for ages, but never had the need to visit the forums until yesterday.

[dbx10]

11 hours ago, GoodBytes said:

This project is all nice and neat, however what Linus forgot to mention is that with Synology, part of the high price, is the built-in software.

If you only need a network drive, then yea, great. But if you need, say, picture gallery app that can be used remotely, via phones and PCs then sadly you are limited in options. OneCloud kinda has a gallery, but it acts more like a view mode where you see ALL pictures without categories, permissions, and so on. And the rest, to my knowledge. don't.

 

 

open source software that runs on docker in a Linux VM can do so much more than Synology, but it's not just a few clicks.

 

Synology offers ease of configuration but for the same price you can do a lot more on truenas with a Linux VM and docker. The difference again being the learning curve.

 

I use photoprism on my NAS with a 30$ tensor core card passed through to the VM and it detects faces almost as well as Google photos, but is hosted and processed locally.

[Jarsky]

3 hours ago, dbx10 said:

open source software that runs on docker in a Linux VM can do so much more than Synology, but it's not just a few clicks..

DSM (Synology) is linux based, and does have Docker support: https://www.synology.com/dsm/packages/Docker

You do have more control over a general purpose linux because you can run anything you want, but DSM is pretty flexible as well. 

 

We deploy a number of Synology & QNAP appliances since they often fullfill the need for storage & basic apps, while being power efficient, quiet and easy to manage where we dont need large stack deployments, like on Ships / Transport and at small remote sites. 

[maplepants]

5 hours ago, dbx10 said:

open source software that runs on docker in a Linux VM can do so much more than Synology, but it's not just a few clicks.

 

Synology offers ease of configuration but for the same price you can do a lot more on truenas with a Linux VM and docker. The difference again being the learning curve.

 

I use photoprism on my NAS with a 30$ tensor core card passed through to the VM and it detects faces almost as well as Google photos, but is hosted and processed locally.

The difference isn't just the learning curve, it's also the upkeep. They managed to save $10 on this build, but even with the 550w power supply they targeted you'd blow that savings on electricity in a pretty short period. 

 

Synology offers calendar, contacts, photos, dns, file share, time machine, proxy, vpn, and a lot more apps with very easy setup and nearly 0 maintenance. Plus you don't need to have a spare computer for the OS install. This video actually did a lot to sell me on a Synology.

[GoodBytes]

4 hours ago, Jarsky said:

DSM (Synology) is linux based, and does have Docker support: https://www.synology.com/dsm/packages/Docker

You do have more control over a general purpose linux because you can run anything you want, but DSM is pretty flexible as well. 

 

We deploy a number of Synology & QNAP appliances since they often fullfill the need for storage & basic apps, while being power efficient, quiet and easy to manage where we dont need large stack deployments, like on Ships / Transport and at small remote sites. 

There is no docker support on non x86 CPU models of Synology. But of course, these are cheaper models

[GoodBytes]

3 hours ago, maplepants said:

The difference isn't just the learning curve, it's also the upkeep. They managed to save $10 on this build, but even with the 550w power supply they targeted you'd blow that savings on electricity in a pretty short period. 

 

Synology offers calendar, contacts, photos, dns, file share, time machine, proxy, vpn, and a lot more apps with very easy setup and nearly 0 maintenance. Plus you don't need to have a spare computer for the OS install. This video actually did a lot to sell me on a Synology.

The only upside he presented is the fact that you have a higher specs device. I want to say that Synology is too greedy on its lower end models with its processor and allergic in giving user decent amount of RAM for proper experience (non-upgradable) but, unless you have an old system laying around, you'll end up paying more, so might as well get the higher end Synology drive, unless you seek for even greater system performance, or have special secondary needs with the system that Synology doesn't offer.

[Needfuldoer]

4 hours ago, maplepants said:

The difference isn't just the learning curve, it's also the upkeep. They managed to save $10 on this build, but even with the 550w power supply they targeted you'd blow that savings on electricity in a pretty short period. 

Only if the motherboard and processor they picked draw substantially more power than the target Synology box's motherboard.

 

1 hour ago, GoodBytes said:

There is no docker support on non x86 CPU models of Synology. But of course, these are cheaper models

Even then, not all x86 DiskStations support Docker. I had a DS214play that didn't get it, but the DS415+ that replaced it did.

[Ultraforce]

2 hours ago, GoodBytes said:

The only upside he presented is the fact that you have a higher specs device. I want to say that Synology is too greedy on its lower end models with its processor and allergic in giving user decent amount of RAM for proper experience (non-upgradable) but, unless you have an old system laying around, you'll end up paying more, so might as well get the higher end Synology drive, unless you seek for even greater system performance, or have special secondary needs with the system that Synology doesn't offer.

Oh yeah I had looked into getting a small prebuilt NAS and ended up with the QNAP low end NAS because the synology one didn't even have 1 GB of RAM which seemed super limiting.

[wanderingfool2]

For a video like this, I would like see more metrics in terms of things such as power consumption.

 

The reasoning being that lets say hypothetically a 4 drive QNAP uses 25W at idle and 37W during high usage [https://www.techpowerup.com/review/qnap-ts-431p2/12.html]

Now considering the current setup, if it were to use 35W at idle and 47W during high usage [this is just a complete guess].  So the delta being 10W.

At $0.10 per kWh, that would mean in 1 year it would consume $8.76 more than a QNAP.  Assuming a 5 year lifespan that's an extra $44 operating cost

 

Overall I think that power draw of devices is really overlooked and I wish more outlets such as LMG actually listed the power units were taking (when there is practical builds). 

 

On 3/5/2022 at 10:14 AM, Needfuldoer said:

Or you could go full stupid like I did and build up a retired enterprise server for even less! My R720 with dual 10 gig Ethernet, 40 gig Infiniband, 256 gigs of RAM, Quadro P400, redundant 750 watt 80 Plus Platinum power supplies, and dual E5-2650 V2s cost less than that machine (minus storage). I've got eight WD White 12 TB drives in a RAIDz2, because RAIDz1/RAID5 sketches me out on large-capacity drives.

 

You just have to not care about looks. Or chassis size. Or noise. And not too much about power consumption. (It idles at 170 watts, half of which is the drives.)

Curious, what is the noise and the space that it takes up? [Like did you repurpose it into a more friendly size...I just have a giant server case sitting in the corner of my house, and it makes enough noise that I only keep it on when needed]

 

Is it actually idling at 170w or is it during load?  Although I can't find WD white drives data sheets, the worst I found (purple drive) only used 5.6w at idle [50w for 8 drives].  Based on what I've looked up (ts-832px) fully loaded draws about 50 watts in average workflow (so lets assume idle) [likely red drives or similar].  At $0.10, that would be $105.12 per year.

[Needfuldoer]

6 hours ago, wanderingfool2 said:

Curious, what is the noise and the space that it takes up? [Like did you repurpose it into a more friendly size...I just have a giant server case sitting in the corner of my house, and it makes enough noise that I only keep it on when needed]

 

Is it actually idling at 170w or is it during load?  Although I can't find WD white drives data sheets, the worst I found (purple drive) only used 5.6w at idle [50w for 8 drives].  Based on what I've looked up (ts-832px) fully loaded draws about 50 watts in average workflow (so lets assume idle) [likely red drives or similar].  At $0.10, that would be $105.12 per year.

Noise surprisingly isn't too bad. Once the blowymatron fans calm down, it's about as loud as a refrigerator. If it starts working hard and the fans ramp up, it gets louder than a desktop but not as loud as a 1RU server.

 

It's a proprietary motherboard, so there's no putting it in a generic chassis. It's doomed to be a 2RU slab forever. (I'm eyeing a 12-bay T620 tower to migrate the parts into, but that would mean buying more drives...)

 

I'm just using some 2TB Seagate Constellation ES ST32000644NS drives as a placeholder until I shuck the externals and lifeboat everything of the Synology four of them are currently in. With eight of those, running TrueNAS Scale, it idles at about 170 watts (as measured with a Kill-A-Watt at the wall). Without the drives, I think it idles at around 90 watts. The newer drives are probably a little more efficient.

 

I thought about going for a 13th gen server (R530, R730, etc) but the potential energy efficiency savings would've taken years to pay off the cost difference between the platforms. (Seriously, Ivy Bridge Xeons and ECC DDR3 are stupid cheap right now! That's why I'm building my Proxmox homelab around that platform, but that's a story for another thread.)

 

It's definitely the ruthless economy PC approach to a home server, and certainly not for everyone. 

 

EDIT: The Synology DS1821+ is awfully tempting, but again I could run this thing for years before the power savings would add up to the additional purchase price. (And it certainly wouldn't have 256 gigs of RAM!)

 

Another edit: I did the math, and if I bought a $1,000 Synology (and extra RAM and a couple cache SSDs to go with it), It would take over seven years of 24/7 operation for the power savings to pay for its purchase price. 

[dbx10]

15 hours ago, maplepants said:

The difference isn't just the learning curve, it's also the upkeep. They managed to save $10 on this build, but even with the 550w power supply they targeted you'd blow that savings on electricity in a pretty short period.

The upkeep on TrueNAS is nonexistent after initial setup. The learning curve in regards to the multiple ways you can opt to monitor system usage is higher, but it's worth the hassle if you're savvy.

For example my NAS runs TrueNAS Core, it runs an Ubuntu VM for Docker, Portainer, and a script automatically updating about 20 containers. The only downtime was me manually taking it offline to add drives and an HBA card, and the odd settings change on the docker-compose file. These OSes are validated and very well maintained by now.
 

 

15 hours ago, maplepants said:

Synology offers calendar, contacts, photos, dns, file share, time machine, proxy, vpn, and a lot more apps with very easy setup and nearly 0 maintenance. Plus you don't need to have a spare computer for the OS install. This video actually did a lot to sell me on a Synology.

The maintenance is similar with Portainer-ce, with better control over software and the ability to run anything on docker hub. Also I reject the notion that anyone installs a NAS / home server and doesn't own a computer to save data from. People with phones and tablets don't buy NAS boxes.

And to elaborate about 0 maintenance, the problem with Synology in particular is SHR is just not a backup solution. They say so themselves. If the logic board in your NAS box dies a slow death, your data is likely to be corrupted beyond repair. Even if it's not, you're basically vendor locked to Synology.

This is very hard to achieve with OpenZFS, unless you're @LinusTech and you never configure scrubs on a petabyte of data.

[AbydosOne]

47 minutes ago, alifarshad275 said:

so how do you gain access to that storage from your phone, laptop and your main pc? because in no video i see someone mention you have to connect via cable or lan to access those files. 

Network != Internet. You don't need Internet access to make a router/switch/access point function and provide a network.

[cmndr]

On 3/5/2022 at 10:07 AM, Muffxd said:

What is the point in using Non ECC RAM with ZFS? I think BTRFS or just EXT4 would probably be better with the RAM that is used here.

 

https://serverfault.com/questions/454736/non-ecc-memory-with-zfs-a-stupid-idea

ZFS doesn't need ECC any more than BTRFS or EXT4 does.

This is mainly a meme from 10 years ago (check the age of your post) where a handful of nerds went overboard on the worst case scenario (which has a near 0% probability of occurring - you're likely 1000x more likely to suffer a fire, power surge, flood, etc.)

I've been using ZFS without ECC just fine for 2ish years now. I will probably swap out to a budget ECC kit off ebay in the near future, since my NAS is "borrowing" 32GB RAM from my desktop.

 

You should probably still use some form of ECC on your NAS if you're running a business but that's regardless of file system.

[Needfuldoer]

5 hours ago, alifarshad275 said:

so how do you gain access to that storage from your phone, laptop and your main pc? because in no video i see someone mention you have to connect via cable or lan to access those files. 

Your local WiFi.

 

You can set up your network for remote access, but that's not a prerequisite.