Short vs long term storage strategy

[porina]

After a long think about short term needs, I ordered two 8TB HDs with the intent of finally setting up an always-on file server/NAS. I have a separate server dedicated for backups, so that doesn't need to be part of the discussion here. Well, not entirely. I only turn on the backup server when I want to do a backup. On average, that's maybe once a month or so. It will be off and disconnected otherwise. So that needs to be taken into consideration. I'd want the new server to have some redundancy in case of disk failure between backups. I'm less worried about malware or similar.

 

The backup server runs unraid, and I'm ok with that. But the file server I am creating in addition will be for the primary purpose to be a single location store for videos. These may be either game capture files or from a camera. I want something good enough to edit straight off, although I will still be limited to gigabit ethernet until I separately upgrade that. Unraid has pretty slow writes, but video editing is mostly read-only, so I should get the read speed of whatever drive it is on. Slow writes just means the original move to the server may take longer.

 

As said, I have two 8TB drives on order. I debated getting 3 or 4 of them from the start, but 8TB of storage should last a while so I can delay the ordering of the additional disks until it is really needed. But here's the question. I could start an unraid with one parity, one data drive. I'll have slow writes, ok reads. But with only 2 disks, I could use Windows for example, and just do a mirror of it. Better write speed, with the disadvantage that when it comes to adding a 3rd disk, I'd have to juggle the data somehow. e.g. Use 3rd disk as parity. Break mirror, use one of the disks as new data disk. Create unraid. Move files over from remaining broken mirror, then add that to storage pool.

 

So, what would you do? Go straight to unraid to make future expansion easier, at the cost of lower short term performance. Or go mirror first for short term performance, and have the juggle when time comes to add 3rd disk? I'd end up at the lower write performance anyway once I get the 3rd disk, so the performance benefit would only last while I'm on 2.

 

Why 8TB? Of 7200rpm drives it was the sweet spot in per-GB pricing. A bonus was that the Toshiba Enterprise drives were practically same price as the consumer NAS ones. The initial hardware I'm thinking of using is only 4 bay (HP microserver) so I didn't want to use small drives.

[samcool55]

I would go straight for unraid. If you need to work on a video project or something I would recommend getting a local SSD, copy the video files you need from unraid to your local SSD, do whatever you need to do and dump the final file onto the unraid server.

After that you can delete everything on the SSD and move on with the next project.

[Windows7ge]

So the upgrade path would be Mirror -> RAID5.

 

I'm a proponent for long term goals. If the speed boost you'd get for the short term would be lost once you get the 3rd disk I wouldn't waste my time with the speed boost knowing I'm just going to lose it down the road. Can't miss what you never had.

 

I've been told UNRAID can be setup with an SSD as a type of tiered storage. Have you done this to help speed up write/some reads?

[samcool55]

2 minutes ago, Windows7ge said:

I've been told UNRAID can be setup with an SSD as a type of tiered storage. Have you done this to help speed up write/some reads?

That's also an option, forgot about that.

You have 4 options (I copied this straight from my unraid)

 

 

Specify whether new files and directories written on the share can be written onto the Cache disk/pool if present.

 

No prohibits new files and subdirectories from being written onto the Cache disk/pool.

 

Yes indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the Cache disk/pool and onto the array.

 

Only indicates that all new files and subdirectories must be writen to the Cache disk/pool. If there is insufficient free space on the Cache disk/pool, create operations will fail with out of space status

 

Prefer indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the array and onto Cache disk/pool.

[porina]

17 minutes ago, samcool55 said:

I would go straight for unraid. If you need to work on a video project or something I would recommend getting a local SSD, copy the video files you need from unraid to your local SSD, do whatever you need to do and dump the final file onto the unraid server.

It's kinda annoying to manage to copy locally, and references in the project file will break once the local is gone.

 

16 minutes ago, Windows7ge said:

So the upgrade path would be Mirror -> RAID5.

Unraid isn't raid5, even if there is a raid5-like element to it.

 

16 minutes ago, Windows7ge said:

I'm a proponent for long term goals. If the speed boost you'd get for the short term would be lost once you get the 3rd disk I wouldn't waste my time with the speed boost knowing I'm just going to lose it down the road. Can't miss what you never had.

That's in part what I'm thinking also.

 

Going with a Windows mirror also means I can delay buying a new unraid licence, although the cost isn't significant anyway. It's not a major factor.

 

16 minutes ago, Windows7ge said:

I've been told UNRAID can be setup with an SSD as a type of tiered storage. Have you done this to help speed up write/some reads?

I don't know if it speeds up reads, which would be actually more important to me than speeding up writes. The amount of write is going to be relatively small in comparison. Only to move the raw files over. I'd probably render output locally, and once happy, also move it to the server.

 

There's another pain point with using a SSD cache. To my understanding, cache drives count towards your licence. The basic one is 6 drives, so with 1 parity, 3 data, I could still fit two cache in that. The other bigger problem is... there is no physical space for two cache drives in the microserver. From memory there's 5 SATA connectors in total, the 5th originally being provided for an optical drive. To get a 6th I'd need a PCIe card, but I'd want to use that for something faster than gigabit ethernet. Oh, in case you wonder why two cache drives, they need to be mirrored otherwise there is no redundancy.

[Windows7ge]

1 minute ago, porina said:

Unraid isn't raid5, even if there is a raid5-like element to it.

I'm aware of this I just wanted to keep it simple.

 

2 minutes ago, porina said:

That's in part what I'm thinking also.

Going with a Windows mirror also means I can delay buying a new unraid licence, although the cost isn't significant anyway. It's not a major factor.

 

There's another pain point with using a SSD cache. To my understanding, cache drives count towards your licence. The basic one is 6 drives, so with 1 parity, 3 data, I could still fit two cache in that.

Do you have a particular reason you aren't interested in using free distributions of Linux?

 

4 minutes ago, porina said:

I don't know if it speeds up reads, which would be actually more important to me than speeding up writes. The amount of write is going to be relatively small in comparison. Only to move the raw files over. I'd probably render output locally, and once happy, also move it to the server.

I can't say for certain how the cache drives work. I've never used UNRAID.

 

7 minutes ago, porina said:

The other bigger problem is... there is no physical space for two cache drives in the microserver. From memory there's 5 SATA connectors in total, the 5th originally being provided for an optical drive. To get a 6th I'd need a PCIe card, but I'd want to use that for something faster than gigabit ethernet. Oh, in case you wonder why two cache drives, they need to be mirrored otherwise there is no redundancy.

So UNRAID uses tiered storage similarly to Windows Storage Spaces where you have to mirror the cache drive if the pool is mirrored as well.

 

The nice thing about SSD's is they don't necessarily have to be mounted properly. You could find a craves they fit in and let them sit. I had to do this in my VM server when the GPU I installed forced me to remove the bracket the two boot SSD's sat on.

[porina]

2 minutes ago, Windows7ge said:

Do you have a particular reason you aren't interested in using free distributions of Linux?

Can't be bothered to learn it. For a file server, I want to keep it simple. I can cope with the Unraid interface. I can cope with Windows disk manager. I like the sound of ZFS but its complexity scares me at the same time. I tried FreeNAS before but for whatever reason I never got it working for me in the past.

 

2 minutes ago, Windows7ge said:

So UNRAID uses tiered storage similarly to Windows Storage Spaces where you have to mirror the cache drive if the pool is mirrored as well.

You don't have to mirror cache in unraid, just that it makes sense to.

 

2 minutes ago, Windows7ge said:

The nice thing about SSD's is they don't necessarily have to be mounted properly. You could find a craves they fit in and let them sit. I had to do this in my VM server when the GPU I installed forced me to remove the bracket the two boot SSD's sat on.

Been there, done that. I'm still short a SATA connector and would need a power splitter. Does that sound like I insulted myself?  

 

There is one other option: new build!

[Windows7ge]

2 minutes ago, porina said:

Can't be bothered to learn it. For a file server, I want to keep it simple. I can cope with the Unraid interface. I can cope with Windows disk manager. I like the sound of ZFS but its complexity scares me at the same time. I tried FreeNAS before but for whatever reason I never got it working for me in the past.

Well I won't pressure you with it but consider throwing it in a VM and give setting it up a shot:

4 minutes ago, porina said:

You don't have to mirror cache in unraid, just that it makes sense to.

+1 informative

7 minutes ago, porina said:

Been there, done that. I'm still short a SATA connector and would need a power splitter. Does that sound like I insulted myself?  

 

There is one other option: new build!

Nah, you're good.

 

If your budget permits it that's an option. Buy yourself a 4U with 24 3.5" SAS bays.

[Kilrah]

Do you have a particular reason to use network storage?

If you're always going to access it from the same machine, directly attached storage is faster than GigE.

[porina]

8 minutes ago, Windows7ge said:

If your budget permits it that's an option. Buy yourself a 4U with 24 3.5" SAS bays.

Now you mention it, I was looking at some older servers with a ton of 3.5" bays in them for another project I never did implement... still, I'm not sure that's something I'd want running 24/7.

 

5 minutes ago, Kilrah said:

Do you have a particular reason to use network storage?

If you're always going to access it from the same machine, directly attached storage is faster than GigE.

For now my capture system is my editing system, but I do intend to make the capture system more "pure" for gaming and I have faster/higher spec systems for editing. I suppose I could add the storage to the editing system. The current case can only hold two 3.5" disks so it would need to be rehoused.

 

While gigabit is going to limit sustained transfers, I was thinking moving up to even 2.5gig would probably put me close to the limits of what a single HD can do. It seems an easier step up than going to 10gig.

[Kilrah]

Indeed, with single HD you won't get more.

For editing and managing large video files I always go at least 2 drives in RAID0 so it's not holding me back. I somewhat regularly work with files that simply can't be read fast enough from a single HDD (or gigE), but 2-drive RAID works. Drobo has nice directly attached enclosures that support redundancy as long as you have 3 drives or more in them, I have one of those with 5 drives too, this gets my old drives when I upgrade.

[Windows7ge]

2 minutes ago, porina said:

Now you mention it, I was looking at some older servers with a ton of 3.5" bays in them for another project I never did implement... still, I'm not sure that's something I'd want running 24/7.

The NORCO-4224 is a 24 SAS bay 4U enclosure. It uses a 3x120mm fan wall and two rear 80mm fans. You could use low RPM fans to control the noise. Temp wise the drives would be fine. Assuming you aren't trying to cool a 16 core Xeon the internals should be fine too.

 

...just an idea for you...

[porina]

I might have over-estimated my storage needs...

 

Anyway, I got lazy and ran out of time. Tomorrow is a major patch day in FFXIV and I needed to clear some space from my gaming system for new recordings. Quickest way for me to do that was to create a mirror under Windows and set a share.

 

Since I was looking at my existing recordings, I noticed it took over a year to generate almost 1TB worth. 8TB of storage, at the same rate, could last over 8 years. Even if I double that, it's 4 years. Needing expansion is far away unless I seriously increase the amount of recording I do...

 

I also found something mildly amusing. Remember how much hate I had for gigabit limiting my transfer speeds? Well, it turned out the 2.5" drive I was using for video storage could only manage around 80MB/s at the end of the drive, so it wasn't even limiting my network at that point.

[Kilrah]

Just added a bit of fresh air...

[porina]

8 minutes ago, Kilrah said:

Just added a bit of fresh air...

29TB? 

 

My gaming system local storage. I haven't mapped network stuff yet. Video recordings go on G, which I partially emptied. In case anyone wonders why I have 1TB HDs and not bigger, they're 2.5" for lower noise than 3.5" ones.

 

 

Servers go up in 15 minutes, time to fill that drive with new videos!