Here's How to Save $45,000

[CPotter]

 

[Devryd]

why is it unlisted?

[WhitetailAni]

12 minutes ago, Devryd said:

why is it unlisted?

Forum members get early access, notwithstanding Floatplane.

[Devryd]

didnt know that, thanks

1 minute ago, FakeKGB said:

Forum members get early access.

 

[Dillpickle23422]

ayy! was just about to go watch some ltt anyways! Welp, no more rewatching videos for now!

 

[Streetguru]

2 minutes ago, FakeKGB said:

Forum members get early access.

Doesn't matter if you aren't the person to get the first comment though

[Dutch_Master]

Goodie, just in time for diner

[WhitetailAni]

1 minute ago, Streetguru said:

Doesn't matter if you aren't the person to get the first comment though

I was the person to get the first comment though.

[Aereldor]

7 minutes ago, FakeKGB said:

Forum members get early access.

For 5 minutes? Haha

[ProjectBox153]

10 minutes ago, FakeKGB said:

Forum members get early access.

No, not really. Even then there are lots of people who have already seen the video, myself included. 

[Dutch_Master]

Linus: -wait, HOW MUCH is Pulseway paying us this time?!?-

[Nystemy]

To be fair here.
A lot of people buy turnkey solutions even if it costs 2-20x more. Simply because they are turnkey solutions. Especially in enterprise/professional environments.

Like even rack mount servers can technically be out built for a fraction of the cost. (And Google for an example used to do that.... Literally having motherboards on storage shelves.)

Downside with building something in house is that it requires more technically adept employees, something that frankly is a limited and expensive resource.

Or just tons of one's own time building said systems.... Not to mention that 

 

But even a turnkey solution can require a lot of additional work at times. Since "turnkey" can be anything from just a prebuilt system without OS, to a system with software and everything. Depening on what exactly one buys.

If a "more expensive" turnkey solution is better or not can depend on many factors.
And at times, it could be cheaper.

Though, these Jelly fishes seems honestly fairly expensive almost regardless of how one looks at it....

[looney]

Fun video, but the ZIL/SLOG part is flawed and might misinform people.

First of all 2TB is insanely overkill, the ZIL gets flushed roughly every 5 seconds, therefore it can never have more then 5 seconds worth of ingress data on it (I highly doubt you are getting 400gigaBYTES per second of ingress  ).

On top of that it is only used for true synchronous writes which are not common in most workloads.

 

All the SLOG does is store the synchronous writes for a few seconds so it can immediately say with certainty to the sender "hey, I have received your file and I have stored it (non volatile), please send me the next bit of data".

At the same time the data is also being written to the actual storage in parallel, the SLOG is just there so that in case of a power cut the data that is still on the HDD cache is also in the SLOG.

 

In normal operation, the SLOG only gets written to, the only time it gets read form is after for example a power cut, at this time ZFS will call on the SLOG to get the data that was still in cache/ram on the actual non-volatile array.

 

So without a SLOG, in case of synchronous writes, ZFS will have to save the file directly to disk, bypassing cache, as it has to assure the sender that it has stored the data in a non-volatile state,

 

With async data the sender will just go BRRRRRR and wont care if the receiver has saved it volatile (HDD cache) or non volatile (actually on disk).

This is faster but also has more risk involved as the sender wont know if its stored volatile or non-volatile.

And this is also where the second fault in the SLOG config lies, the NVMe drive uses still has VOLATILE storage on it in the from of highspeed cache.
So you could still lose/corrupt data in case of a power cut or something comparable, defeating the sole purpose of SLOG.
You are telling the sender that the file is safe while it might not be, defeating the entire point of sync data transfer.

For a proper SLOG (which 99% of the time you don't need as you wont be writing sync) you still need proper DC grade SSD's those have built in supercaps to make sure that no data should ever be lost no matter what.
And yes, SSD'!s!, you would want two in raid 1 just to be sure.

SLOG is 100% a anti-dataloss thing, It will never give you a speed boost, the drives still have to handle the same ingress regardless of having a SLOG.

The only way to speed up writes in ZFS is with more VDEV's that can be written to in parallel.

In your case the write IOPS will be limited to two drives as it can only write to two VDEV's at the same time.

[Velcade]

Where is the part list?

 

Found it:

[iLikeBananas]

Couldn't you guys make a bunch of those and sell them on ebay?

[Stupiduser]

37 minutes ago, Velcade said:

Where is the part list?

 

Found it:

Approximately $13K for one of these. Submitting PO to boss now.

[nullbit]

So after seeing that video I was very surprised to see something I built on the screen years ago (from what I can tell 2015/2016). I didn't even know this jellyfish company existed or if it even did at the time for that matter.

The case is a Lian Li PC-A79 (not sold anymore but I'm guessing they have a hookup) and the drive bays are iStarUSA 3x5.25 to 5x3.5 hot swap bays (https://www.amazon.com/gp/product/B008RRZI14/ref=ppx_yo_dt_b_asin_title_o09_s00?ie=UTF8&psc=1).

Some things about my build that were a problem that I'd like to talk about and not sure how they are circumventing them if at all. The lian li case on the top 3 5.25 bays it has inward facing right angle fins to hold something like a cd drive in place. I had to bend those fins flat (as you can see by the janky radiator stuff on the top of the case I've learned a lot from LTT). Also with the drive bays, I constantly had a problem with one of them randomly disconnecting the drive, I finally got tired of it and took the bay apart to find the sata pins having not been soldered to the board... I soldered them myself and have since not had a problem but even though my sample size is small at  4 and 1 out of 4 being bad I wonder how many systems were still shipped with an issue like that (from jellyfish). 

 

The rest is some supermicro board that doesn't fit in the case properly (hence the radiators mounted outside the case). some intel xeon x5570. 59 gb of ddr2 ram (should really be in the 70s but some slots have died from the board flexing too much due to the mounting holes didn't line up with the motherboard). 3 flashed LSI 9211-8i raid cards.  Its all really garbage but I put it together for probably around $800 ($400 of which was spent on an initial ebay purchase of a server with the ram and cpu's which I gutted and put into this case to make quieter).

Personally I really like the build with it running unraid. I don't do any professional work on it and even with all those drives its only 34 tb of usable space (got the HDD's for free when I still worked IT).

I was just really chuffed to see something I designed (but obviously they use better internals) be used as a companies 1 of 3 main products and wanted to share.

[Sauron]

Unfortunately even if you saved 100K on it ultimately the problem is always the same; is Linus going to answer a call at 5AM on Sunday for direct support on a server he built? Or even on a regular workday at a reasonable time considering he has a full time job? Does he have a replacement just sitting there so he can ship it to the customer as a placeholder while he fixes the machine in case something goes terribly wrong? If you can't afford much downtime and you don't have specialized staff who can sort out issues very quickly you're kind of forced to buy off the shelf from an enterprise OEM.

 

Now granted, it's kind of absurd that Lumaforge doesn't offer redundant power supplies so maybe they aren't really delivering on that front either.

[Arika]

51 minutes ago, Sauron said:

Unfortunately even if you saved 100K on it ultimately the problem is always the same; is Linus going to answer a call at 5AM on Sunday for direct support on a server he built? Or even on a regular workday at a reasonable time considering he has a full time job? Does he have a replacement just sitting there so he can ship it to the customer as a placeholder while he fixes the machine in case something goes terribly wrong? If you can't afford much downtime and you don't have specialized staff who can sort out issues very quickly you're kind of forced to buy off the shelf from an enterprise OEM.

 

Now granted, it's kind of absurd that Lumaforge doesn't offer redundant power supplies so maybe they aren't really delivering on that front either.

i certainly agree with this, but i can only hope lumaforge isn't charging 400% per unit purely for support

[Windows7ge]

6 hours ago, looney said:

-- snip --

Wanted to comment with something similar but I was 6 hours too late.

 

I don't have evidence of this but I have been told in synchronous write operations tasks like database (MySQL perhaps?) or VM's can still see a write performance gain. Presumably writing to cache instead of directly to disk.

 

For what 99% of people at home or even work would use this for (SMB/NFS/SSH/iSCSI) yeah, ZIL does basically nothing for you performance wise.

[Electronics Wizardy]

59 minutes ago, lexusgamer05 said:

Actually TrueNAS didn't replace FreeNAS. They are two seperate but similar projects by iXsystems.

 

FreeNAS is for homelabs, home projects etc

 

TrueNAS is for, pretty much LTT's situation. 

 

IX systems rebranded it now, so what used to be freenas is now truenas core. Truenas is for ixsystems products, and can't be put on a diy system like what linus made here. The freenas branded oses are dead now.

 

 

Look at the popup here

 

https://www.freenas.org/

[JBertoli]

Awesome build! Few questions for the more experienced here:
1. What are the alternatives to iStarUSA cases that people recommenced for this kind of build? This one's not available in Australia.

2. Seems pretty straightforward but worth asking anyway, would it be fine/wise to run backup through TrueNAS?

3. Am I correct in saying that we could start with a group of 10 drives and then add the second group later to make cash-flowing the build easier?

4. I'd guess the best way to back this up offsite would be to build a similar unit (less powerful) to mirror it offsite? That way if the primary unit fails, the backup server can just be specced up and ran as the primary to get us out of trouble. Third-copy would go in the cloud, of course.

[Electronics Wizardy]

Just now, JBertoli said:

1. What are the alternatives to iStarUSA cases that people recommenced for this kind of build? This one's not available in Australia.

 

Supermicro makes a lot of cases that will work fine.

 

1 minute ago, JBertoli said:

2. Seems pretty straightforward but worth asking anyway, would it be fine/wise to run backup through TrueNAS?

 

Depends on your exact workload, but should work fine. What are you backing up to/from

 

1 minute ago, JBertoli said:

Am I correct in saying that we could start with a group of 10 drives and then add the second group later to make cash-flowing the build easier?

Yup you can, but performance will be a bit worse as the data won't be balanced optimally between the drives.

 

2 minutes ago, JBertoli said:

I'd guess the best way to back this up offsite would be to build a similar unit (less powerful) to mirror it offsite? That way if the primary unit fails, the backup server can just be specced up and ran as the primary to get us out of trouble. Third-copy would go in the cloud, of course.

Yea you can do a offsite backup copy. You can also colocate the second nas if you want it in a nice secure datacenter.

 

You can also backup to tape if you want to deal with tapes.

[indrora]

Once upon a time I heard the following phrase:

Quote

Open Source is Free, so long as you don't value your time

(This also implanted upon me the concept of "Open Sores", what you encounter when dealing with Open Source software)

 

While you can go and do this with TrueNAS Core and the like, unless you're good at it, you're gonna spend a lot of time figuring out what you want and need. It is 100% true that I can build a storage server with more density for the same price... If my time is worth nothing. At my salary, that thing would cost over $30K more, simply from the time I would have to put into it. 

 

The fact of the matter is that Jellyfish devices come with three things that are worth the cost:

• An out of box experience that starts with "plug it in" and ends under 5 steps later with "dump content onto it"
• Support for when things inevitably go south with warranty service
• Out of the gate integration with the common tools without hassle.

And that's where you're paying for the decade-and-some years of experience that the team that puts together the Jellyfish hardware, software, and integrations has. In the wise words of a consultant friend of mine:

Quote

You're not paying me $1000 for turning a wrench, you're paying me $5 for turning a wrench and $995 for knowing which wrench and how much to turn it after turning the wrong wrench a lot and costing a lot more than $1000 to unfuck it after the fact.

 

[JBertoli]

2 hours ago, Electronics Wizardy said:

Supermicro makes a lot of cases that will work fine.

 

Depends on your exact workload, but should work fine. What are you backing up to/from

 

Yup you can, but performance will be a bit worse as the data won't be balanced optimally between the drives.

 

Yea you can do a offsite backup copy. You can also colocate the second nas if you want it in a nice secure datacenter.

 

You can also backup to tape if you want to deal with tapes.

Thanks for your very detailed answer mate!