Jump to content

What would you do,

with 48 SAS2/SATAIII ports?

IMAG0362.thumb.jpg.47d3aaf9c5303bfb8d8d718ed96da8d0.jpg

  1. leadeater
  2. PianoPlayer88Key

    PianoPlayer88Key

    What HBAs are those?  (I've been thinking of getting an X7DWE, X8DTH-6F or X9DRH-7TF or similar with some 9211-8i or 9200-16e HBAs, but...well see below about not wanting rack.)

     

    As for what I'd do ... I'd hook these up. :) (I'd also need PATA/IDE ports for some of the older drives.)

     

    5b0f55e67a2ec_hs0101HardDrivesSSDsetc2018-04-14-resize1288x1232.thumb.jpg.47f114c38d4824ed31e99ae92125e537.jpg

     

    But... what case would I PUT those drives in? And I do NOT want rackmount! 

    (I do have an 80s vintage or so audio cabinet that's 19" wide internally (side to side, no mounting rails/hardware other than the doweled shelves the existing equipment sits on), but is only 16" deep, with a glass door front and an open back.  It's about 36" tall I think, give or take a few inches.  I don't think it would work for housing rack cases due to the limited depth, though.

    Found a couple pics of the cabinet...

    Spoiler

     

    IMG_20180722_202934181.thumb.jpg.5479b1a3310d77cc0c647657044dd7b5.jpg

     

    IMG_20180721_133739598.thumb.jpg.615c136fc831ff07feea4e766c45b01f.jpg

     

     

    Also silence would be important - under load, all fans, etc. combined should be quieter than a single 5400rpm 2.5" HDD or a Samsung 860 EVO when ready but idle.

     

    One concept I had for a case would be in the spoiler - basically a dual-chamber cube design with drive bays in one side & up to a quad-socket mobo in the other side, with other parts put where they fit.

    Spoiler

     

    P1930551b2.thumb.JPG.e3892df44baf0515048cdb6dce53f001.JPG

     

    P1930552b2.thumb.JPG.185fcc19e26ddeb4f23b98d5b6a2e7e2.JPG

     

    P1930553b2.thumb.JPG.39e8ea178a30db2f497228e0b984031d.JPG

     

    5b08fb93a3b2d_MotherboardFormFactors(largestSupermicroX11QPH)-2018-05-25.thumb.png.ab233babfe01e241716f7af0c669f97d.png

     

     

     

    My current case is a FD Define R5.

    5a3b22f19f0f9_2017-12-201834-FDDefineR5needmoreHDDcagestrays.thumb.JPG.a4e750567d6d14e84dd4f391a1722df3.JPG

     

    Spoiler

    I don't want this type of case.

    sc946ed_ad_pull-out_new_20150727.jpg.d13d826a5498c4e4a2657ca260de7bbd.jpg

     

    Also I'd want video out to a display I already have that only has DisplayPort & HDMI (so onboard VGA out wouldn't work).  I don't want to put a GPU in any PCIe slot longer than x1 - prefer leaving the x4, x8 & x16 slots for other cards like HBA, analog media capture, etc.

     

  3. Windows7ge

    Windows7ge

    Quote

    48 SSDs obviously

    ...you're on to me aren't you?...

    I have plans that involve the SSD array. I'm going to explore a different file system. BTRFS, maybe some other. Something more performance oriented. Then I'll have it backup over the 10Gbit network to a ZFS box with all the data integrity goodness.

     

    @PianoPlayer88Key

    They are the LSI 9201-16i

    That's quite the project. I have to ask though why so many miscellaneous drives? What's the use case? There are drives 15TB in size nowadays you could scale back how many you need.

  4. PianoPlayer88Key

    PianoPlayer88Key

    Ah @Windows7ge, those are a bit far out of my price range - cheapest one I see on eBay is $130 US.  Otoh, the 9211-8i / SAS2008 can be had as low as about $25 or so, and the 9200-16e also starts around $25.  (They seem to be similar price now; when I looked several months ago the 9211-8i started north of $40 and the 9200-16e was closer to $18-20 or so I think.)

     

    As for the many drives, well, a lot of them have been accumulated over the years. :) Many of the older ones are pretty much ready for retirement, just need to make sure I have everything I need copied off of the still-functioning drives.  I think I've done that already, just it's been so long since I powered them up that I forgot what's what for now.  I do know some drive still have stuff on them, but I think it's mostly clones, or stuff I was testing.

     

    As for what the drives are...

     

    3.5" IDE HDD:

    Spoiler

    These were freebies last year (2018) from a church friend who no longer wanted them.  I DBANned them all except the dead one.

    • 8.4GB IBM Deskstar 14GXP & 16GP – DTTA-350840 - #WE0WEFR8284 – 1998-10
    • 8.4GB IBM Deskstar 14GXP & 16GP – DTTA-350840 - #WE089704 – 1998-11 – This one has the click of death.  Based on the sound it makes, I don't think it will ever resurrect itself, unlike the WD800BB below.  This one sounds like I hear the head scraping across the platters on each pass across between clicking against the stops, whereas the WD was silent between the clicks.)
    • 20GB Maxtor DiamondMax 40 VL – 32049H2 - #L2R0189C – 2001-03-08.
    • 20GB Maxtor DiamondMax 40 VL – 32049H2 - #L24KGCXC – 2001-06-11
    • 40GB IBM Deskstar 120GXP – IC35L040AVVN07-0 (07N8450) - #VNP200B2GRNNWA – 2002-05-18
    • 40GB IBM Deskstar 120GXP – IC35L040AVVN07-0 (07N8450) - #VNP200B2GRR12A – 2002-05-18.
    • 40GB Maxtor DiamondMax Plus 8 – 6E040L0 - #E1T7976E – 2004-05-20
    • 40GB Maxtor DiamondMax Plus 8 – 6E040L0 - #E1T7LRHE – 2004-05-20
    • 40GB Maxtor DiamondMax Plus 8 – 6E040L0 - #E1TDCPME – 2004-05-24

    These are ones I bought myself a long time ago.

    • 80GB Western Digital Caviar – WD800BB-32CAA0 - #WMA8E1004969 – 2001-09-24  (This is the first HDD I personally ever bought.  Interestingly, it had the click of death when I first plugged it in after several years of sitting on the shelf, then several months later after a few more attempts, spontaneously started working again.  It struggled getting going a bit, but I haven't had any major issues with it since then.) .........  It actually only has 1 reallocated sector.  (I have a screenshot of some Linux SMART utility showing that somewhere but idk where it is, & it's probably not labeled properly so I'd have to look through my photos to find it.  Windows 10 can't see SMART stats on my IDE drives for some reason, idk if it's my ByteCC BT-PESAPA controller I'm using attached to my ASRock Z97 Extreme6.) 
    • 250GB IDE 3p5 in HDD – Western Digital Caviar (Recertified) – WD2500JB-53EVA0 - #WMAEH1268512 – 2004-07-22

     

    3.5" SATA HDD ≤ 2 TB:

    Spoiler
    • 750GB Western Digital Caviar SE16 – WD7500AAKS-00RBA0 - #WCAPT0467717 – 2007-09-26 (I think this was the first SATA drive I bought back in 2007 or 2008 or so.  I remember it being the largest capacity I could get at the time, AND the best price to capacity, which is something I don't think we see happen simultaneously very often.)
    • 750GB Western Digital Caviar Black – WD7501AALS-00J7B0 - #WMAU00194112 – 2009-07-28
    • 1TB Western Digital Caviar Green – WD10EADS-00L5B1 - #WCAU49210342 – 2009-04-09
    • 1.5TB Western Digital Caviar Green – WD15EADS-00P8B0 - #WMAVU0170855 – 2009-09-07
    • 1.5TB Western Digital Caviar Green – WD15EADS-00P8B0 - #WMAVU0188395 – 2009-09-07
    • 2TB Western Digital Caviar Green (Recertified) – WD20EADS-00S2B0 - #WCAVY1252756 – 2012-09-28
    • 2TB Western Digital Caviar Green – WD20EZRX-00D8PB0 – #WCC4ML3ZN5HX – 2014-08-06

     

    3.5" SATA HDD > 2 TB:

    Spoiler
    • 4TB HGST Deskstar NAS – HDN724040ALE640 (0F22408) - #PCJGJSXS – 2014-11
    • 4TB HGST Deskstar NAS – HDN724040ALE640 (0F22408) - #PCJGK38S – 2014-11
    • 4TB HGST Deskstar NAS – HDN724040ALE640 (0F22408) - #PCJGK3BS – 2014-11
    • 5TB HGST Deskstar NAS – HDN726050ALE610 (0F23073) - #NAG226TK – 2015-06  (The $100 I paid each for this and #NAG5VZXN below in November 2016 is still the cheapest I've gotten a HDD per TB.  I know some external 8TB drives have been as low as $120-130 or so recently, but I'd prefer to keep the warranty; don't think shucking them allows that.)
    • 5TB HGST Deskstar NAS – HDN726050ALE610 (0F23073) - #NAG5VZXN – 2015-06
    • 5TB HGST Deskstar NAS – HDN726050ALE610 (0F23073) - #NCG1VDLL – 2015-06
    • 8TB HGST Deskstar NAS – HDN728080ALE604 (0F29801) - #VLKW1R6Z – 2017-01
    • 8TB HGST Deskstar NAS – HDN728080ALE604 (0F29801) - #R6G4Z6SZ – 2017-03
    • 8TB HGST Deskstar NAS – HDN728080ALE604 (0F29801) - #R6G5049Z – 2017-03
    • 10TB HGST Deskstar NAS – HDN721010ALE604 (0F27620) - #1SGHVXTZ – 2017-10
    • 10TB HGST Deskstar NAS – HDN721010ALE604 (0F27620) - #1SGLSEJZ – 2017-10

     

    2.5" SATA HDD:

    Spoiler
    • 2TB Samsung (Seagate) Spinpoint M9T – ST2000LM003 - #S377J9BGA00474 – 2015-10  (This was in my laptop for about a year or a year and a half or so, until one day explorer froze while I was accessing it.  Anyway, it still powers up, but last I checked had 38 pending sector reallocations.  I copied the data to one of the 4+ TB drives (except a couple files which were probably in bad sector areas), then retired the drive, in favor of one of the 1TB Crucial MX300 SSDs mentioned below.)

    These were in the batch from the friend, mentioned above in the 3.5" IDE section.

    • 1TB Toshiba MQ01ABD100 - #63T4FHDWS – 2013-06-29
    • 1TB HGST Travelstar 5K1000 – HTS541010A9E680 (0J22413) - #JA1003BVJP99WM - 2014-04
    • 1TB HGST Travelstar 5K1000 – HTS541010A9E680 (0J22413) - #JA1003BVJPA7MM – 2014-04

     

    2.5" SATA SSD:

    Spoiler
    • 256GB Crucial M550 – CT256M550SSD1 - #14500E09AE8B (This is the current boot drive in my desktop.)
    • 1050GB Crucial MX300 – CT1050MX300SSD1 - #171816E6CFD2 (This is in my laptop - replaced the 2TB Samsung HDD that started developing bad sectors.  Interestingly, I had already bought it, intending to replace the 256GB M550 SSD, and was going to give that to my dad.  His Dell D830 laptop at the time had a 500GB WD Black HDD (as well as 2GB DDR2 RAM, Core 2 Duo CPU, Intel GMA GPU).  He ended up getting a 500GB Samsung 850 Evo, then around March 2018 bought a newer laptop.  It has an M.2 boot drive (as well as an i7-8550U, Radeon 530, 16GB DDR4), and we took the 2TB HDD out and put the 850 EVO in so he'd have his old data.)
    • 1050GB Crucial MX300 – CT1050MX300SSD1 - #1742194F936C  (Also in my laptop, bought it when it was on "sale" although now they're quite a lot cheaper.)
    • (not pictured) 240GB Crucial BX200 (I put this in my dad's old D830 laptop, been using it occasionally for testing purposes, with Windows 10.  Yes, that includes benchmarks, games, etc.  Cinebench gets about 80-90 single core and about 45 or so single core, Ice Storm gets about 4500 or so overall score,  TF2 runs about 20 or 30 fps at 640x480 low I think, and CS:GO launches but crashes a minute afterwards (system runs out of RAM.))

    These were given to me by the same friend, on a slightly earlier occasion.  He'd been having problems with them.  It happened to be around Black Friday 2016, so I suggested he get some Samsung 850 EVOs, and he got a couple 1TB drives for $250 each.  (They were normally $350 when not on sale at the time.)  Ever since then, he as absolutely LOVED the Samsung drives, they've never given him any trouble (except for a bit of growing / birthing pains - he's not all THAT computer literate, and was having trouble with cloning them - actually returned them to the store, then bought them back when he learned how he could clone / back up stuff with them.)  I think he's bought a couple more since then.  I've also told him about the M.2 drives (and showed him my 970 Evo), and while his current laptops don't support M.2, his next one most likely will.  (He's about 84 I think, and was pretty impressed with the M.2 form factor compared to the 2.5" ones.)

    • 256GB Transcend TS256GSSD25S-M - #425257-0046
    • 256GB Transcend TS256GSSD25S-M - #A02560-0182

     

    2.5" M.2 SSD:

    Spoiler
    • 250GB SATA Crucial MX200 – CT250MX200SSD6 - #1601116CC7B2 (This has been the boot drive in my laptop.  Technically still is, I just have to F7 on startup to choose it, otherwise it defaults to the Samsung.)
    • (not pictured) 1TB NVME Samsung 970 EVO (I have Windows installed on this, but haven't fully set it up.  It's supposed to be the boot drive in my laptop when I can get around to finishing setting it up (installing apps, setting things up, importing settings, VMs, etc), and in fact by default the laptop boots to this if I don't hit F7 during POST.)

     

     

    Scattered across the drives (mostly the 4TB and larger HDDs, with a few things on the smaller HDDs) is data accumulated over the years.  (A year or two ago, I was coming across stuff on one of the drives that had originated from our very first 286 PC, which had a 40 MB (not GB) Seagate MFM hard drive and was bought in January 1989.  We still have the invoice.)

     

    I'd like to see if I can get things organized, although that would be a TON of work to do.

    Spoiler

     

    (A lot of my several (possibly 10s) of TB of photos, audio files, videos I've recorded, etc, are essentially unlabeled - in folders named with the date I copied them from the device I recorded with, and filenames like IMG_3721.jpg, MVI_8332.mp4, STE-003.wav, etc.)

     

    Also I think some form of deduplication is in order, although in thinking about it, I don't think the ZFS definition of "deduplication" is quite what I want.  There are multiple copies of files and folder structures strewn about, and some have subtle differences (like one duplicate folder might be missing one file, and its duplicate has that file but misses ANOTHER one that the other folder has).  Also I might have multiple encodings of audio & video files - basically the same content, but different codecs, bitrates, other parameters, among other things.

     

    I'd like to be able to go in, get rid of the disorganized duplicate stuff (and on the multiple encodes, keep only the highest quality plus maybe a couple others, like for audio, uncompressed WAV plus maybe 192kbps and 48kbps mp3 or something like that), pool it all together, put it on however many drives it would fit on.  My guess is that it would probably be somewhere around 20 to 25 TB, maybe 30 TB, but I'm not totally sure yet.

     

     

    Some photo albums of SMART data, WinDirStat, etc: (not really organized though):

    Then, I'd want to make a full-image backup of absolutely everything, then have some way to set up incremental / differential ongoing backups.

     

    My idea was to have the main copy of the data (on multiple 8 or 10 TB HDDs) in my main desktop PC readily accessible, then a server would house my backup copy in some kind of RAID / ZFS or something with multiple drive redundancy.  I'm not sure if I want unraid or FreeNAS/ZFS though.

    Spoiler

    I like the integrity checks, snapshots, etc. that ZFS uses, but don't like its recommendation for lots of ECC RAM.  (I'm willing to use ECC, but would rather use a lot less than they say.)  I also like how Unraid puts its data across disks in such a way so that if more drives die than you have parity for, you don't lose everything, you just lose the data on the extra dead drives.  Also from what I"ve heard you can add drives 1 at a time with Unraid, but not with FreeNAS.)

    As for the incremental backups... one idea would be to have another HDD plugged into the desktop (either internal or external) onto which the automatic incremental backups would go.  Then when it gets full, I'd fire up the NAS system, plug in that HDD, dump the stuff into the NAS, then hook the newly emptied drive back up to the desktop.

     

     

    And, besides not really wanting to use noisy nor rackmount hardware, doing it at a not-too-expensive cost is turning out to be quite a challenge.  For right now, I have two possible scenarios in mind, depending on the cost per TB of the storage media relative to normal consumer HDDs (or SSDs if/when they overtake 5400rpm 3.5" SMR HDDs in cheap $/TB.)

    Spoiler

     

    • My preferred option, if I'm able to get storage media at a tiny fraction of the cost per TB compared to consumer-grade HDDs, would be to spend about $800-1K on the NAS setup, not including the storage media.  (I think back in the early/mid 1990s, you could get backup tape media for about 5-10% or so of the cost of a HDD, and the tape drives cost about half or so as much as a HDD back then.)
    • Or, if HDDs is the cheapest media I can get for backup purposes, then the total combined cost of the NAS server hardware (except the HDDs and, if I use LGA1151, the i3-6100 I already have) has to be less than the cost of a single 6TB or 8TB Seagate Barracuda/Ironwolf, WD Blue/Red, HGST Deskstar NAS, etc.

     

     

    If I go with the first option, I'd also use the server for other purposes, like ingesting and converting analog media (like VHS & 8mm video, cassette and reel-to-reel tape, vinyl records, etc).  If I went with the 2nd option, it would strictly be for backup purposes only.

     

    A couple motherboards I've been considering would only be usable with the 1st option, as their price on their own comes close to or exceeds my projected budget for the 2nd option.

    Spoiler

     

    They are:

    • Supermicro X9DRH-7TF ($300 on eBay, or $250 for the -7F or -iTF),
    • X8DTH-6F ($125 on eBay), or
    • ASRock Z270 Taichi ($160 at Amazon / Newegg). 

    Cheaper boards under consideration include:

    • ASRock E3V5 WS ($84 on Newegg but no iGPU support),
    • Asus B150M-C/CSM (also $84, supports iGPU but no ECC),
    • Supermicro X7DCL-i ($51 on eBay) 
    • Supermicro X7DWE ($40 on eBay)

     

     

     

    Anyway .... I haven't really been doing much with the project in the last few months, been working on other things.  That includes, but isn't limited to, getting a CompTIA A+ certification which I have now, working on updating my resume which is still a work in progress, and finding work so I can move on with life, move out of my parents' house, etc. which is still to come.  I'm hoping to find some employment basically building / repairing PCs - basically the kind of things you do at the workbench in PC Building Simulator.  I haven't even powered on the desktop or any of the 3.5" HDDs since probably late summer 2018, or early fall at the latest.

    Also, as far as plugging things in ... the house I'm in now (my parents moved here November 1978, I was born April 1981) was built in 1962 and thereby has some older electrical designs.  Many of the 3-prong outlets have an open ground, and my bedroom only has 2-prong outlets, among other issues.

     

    Anyway I'm sure this will be quite a bit.  I attempted to make it appear shorter with the generous use of spoilers, but I wouldn't be surprised if it won't all fit on an 8K display when your browser is running in full screen, OS scaling is set to 100% and the page scaling is set to 50%. :P 

  5. Windows7ge

    Windows7ge

    @PianoPlayer88Key That's quite the extensive situation you've got going there and $130 for 16 SAS2 ports is a pretty good deal buying new.

     

    Quote

    I like the integrity checks, snapshots, etc. that ZFS uses, but don't like its recommendation for lots of ECC RAM.  (I'm willing to use ECC, but would rather use a lot less than they say.)  I also like how Unraid puts its data across disks in such a way so that if more drives die than you have parity for, you don't lose everything, you just lose the data on the extra dead drives.  Also from what I"ve heard you can add drives 1 at a time with Unraid, but not with FreeNAS.)

    The RAM issue with ZFS is a misconception. (link, link) ECC is recommended but you don't require vast quantities of it. There's also DDR3 16GB ECC RDIMMs you can buy pretty cheaply you just need a motherboard that supports them.

     

    UNRAID does let you mix and match drives because you can set an SSD as a cache drive which you can write to and then UNRAID will flush the data to the pool later. Read performance takes a bit of a hit for non-frequently accessed files though. On ZFS or any other type of RAID for that matter if you setup your pools correctly w/ parity or striping w/ mirrors you can have the benefit of the read/write performance bump along with the redundancy. Not to mention FreeNAS is free, UNRAID is not. I will say though if you need functionality beyond a file server then FreeNAS isn't your best option.

     

    I can't really comment on deduplication. I haven't used it on FreeNAS or elsewhere but I do have some plans to try it after I build my backup server. I don't know how FreeNAS's dedup functions compared to others so again I can't comment on that.

     

    Also the only real way you can plug a NTFS formatted drive into a FreeNAS box in via SATA and even then you can only read data from the disk. It'll dump the data to whatever directory you specify from the WebUI and from there you can organize it.

     

    If you have any questions or questions I may have missed go ahead and ask (again).

  6. PianoPlayer88Key

    PianoPlayer88Key

    Ahh yeah @Windows7ge buying them new, I think I've seen them a fair bit more expensive.  I'd most likely be buying mine used.  (I'd be surprised if the $30-40 cards with 16 external SAS ports on eBay are new.)

     

    I've heard that about ZFS not needing nearly as much RAM as people think.  I'm hoping to be able to build a NAS system with low RAM without too much issues.  For example, I'd like to be able to do what I need (if I only use it as a cold backup storage which was my original plan) with only 256MB or 512MB or maybe 1GB of physical RAM installed.

    I might consider DDR4, only because I already have an i3-6100 that was harvested from my laptop when I upgraded to a 6700K, but if so I'd only use maybe 4GB RAM, which is the smallest DDR4 stick I know of that's available.  At least I haven't heard of any 2GB or even 1GB DDR4 DIMMs.)  The only way I might even remotely consider coming close to an X GB per Y TB ratio is if I used DDR2.  I've seen that stuff for like 70-90¢/GB on ebay, a far cry from the $10-15 or whatever DDR4 commands, and still cheaper than the $2-4/GB I last remember seeing DDR3 a couple months ago or so.

     

    Although, I'm concerned that even older server motherboards that might be cheap enough to be in my price range for that use case (that is, if the trend in parts getting cheaper the older they are continues farther back than LGA 771/775), like Intel Socket 603, Slot 2, Socket 8, or AMD Socket 940 or Slot A, wouldn't be able to support expansion cards that could connect 3.5" SATA HDDs > 2TB.  So far the oldest I've seen that *might* support something like that (via having PCIe slots) is Intel Socket 604 or LGA 771, or AMD Socket F.

     

    Quote

    you can set an SSD as a cache drive which you can write to and then UNRAID will flush the data to the pool later. 

    Another thing I'd like is some kind of safety so that if I lose electrical power in the middle of things (or the OS crashes like the BSOD I got on my laptop a little under an hour ago), it doesn't get corrupted, I don't lose the cached data, etc.  Also I remember quite a few years ago when I'd have some external storage device connected, and transferring files between them... Even if the Windows file copy dialog hit 100% and went off the screen, it wasn't actually done copying.  (I unplugged the drive, and it said data got corrupted / didn't all get saved.)

    Also I'd like to see a fundamental change in how RAM volatility works, etc.  If a system loses power suddenly, then upon restoration of power you should be able to resume where you left off.  (Might Optane DIMMs or something similar in the hopefully near future allow that?)

     

    I don't care too much about read/write performance, so long as it's not slower than a single (non-RAID) mechanical HDD when doing incremental backups, and it doesn't take more than a day or two to do a full image backup of everything.

     

    Yeah ... I"m not sure if FreeNAS is the best option either, still need to learn more about that, as well as UnRAID, OpenMediaVault, NAS4Free, maybe others that I've heard of but forgotten their names for now.

     

    One major thing is that I want to do things as cheaply as possible.  If I can get backup media for 1/15th the cost per TB as a 5400rpm HDD, I'd be willing to spend up to $700-800 on the rest of the hardware.  But, if HDDs are the cheapest media I can get per TB, then my limit on the rest of the server, accessories, etc. is about $150-200 or so.

     

     

    I might have to look through more to see if there's stuff even I might have missed. :)  Anyway though ... it's probably better i I can get SOMETHING started with a backup, cause otherwise I HAVE pretty much no backup.  (I have a partial backup of a few things, but it's all older stuff mostly, and so disorganized & strewn all over the place haphazardly, that i don't even think it's right to CALL it a backup.)

    Budget issues, knowing how to set things up, etc. are pretty much my biggest holdups - ESPECIALLY knowing that in the 1990s I could get a tape drive for about 1/2 the cost of a hard drive, and media for about 1/10th to 1/15th the cost or thereabouts, but I can't find anything remotely like that now.  (Take a look at the prices in the magazine ad below from January 1994, comparing cost per MB - yes they were in megabytes back then.)

    25664186_IMG_20180119_131518692-SGComputers-tapebackupmuchcheaperthanHDDs.thumb.jpg.5457f8c961b803a075b6800837658454.jpg

     

  7. Windows7ge

    Windows7ge

    @PianoPlayer88Key

    You would want to use more than 1GB of RAM. If you can get a server board that supports DDR3 Registered memory you can buy one 16GB stick for $50. Of course it will have to be somewhat modern to support that density of memory. If you're really strapped for cash you know you can buy super micro servers off eBay pretty cheap. You could hack it up so as to fit all of your drives.

     

    Quote

    Also I'd like to see a fundamental change in how RAM volatility works, etc.  If a system loses power suddenly, then upon restoration of power you should be able to resume where you left off.  (Might Optane DIMMs or something similar in the hopefully near future allow that?)

    @leadeater You might be able to help here because I can't recall the specifics.

     

    This exists and has existed for quite some time now. It's called (I believe) NVRAM and it acts as a form of permanent storage even in the event of shutdown or power loss. However this is rather special memory which you will really only find in high-end or application specific servers. A con to this type of memory is that it isn't as fast as VRAM (I believe, it's been a long time since I read up on this I can't remember.) and comes with a much higher price tag which makes it not consumer friendly and then there's the matter of the rest of the system has to support it and your use case has to leverage it it's just not ready to be put into PC's.

     

    As for data-loss during file transfers you're only going to lose data if the drive fails during transfer or before it gets flushed to the pool. I wouldn't be surprised though if UnRAID let you setup a pair of SSD's in RAID 1 to acts as the cache. It's also helpful if you do copy/paste not cut/paste to verify the file got there safely before you delete it locally.

     

    I've seen some very interesting tech from the old ages not just how small hard drives use to be or tape storage. For example, here's some RAM:

    plane.jpg

     

    Here's a close up:

    cores2.jpg

     

    Ferrite core memory, one "stick" is equivalent to 4kB.

  8. leadeater

    leadeater

    Quote

    @leadeater You might be able to help here because I can't recall the specifics.

    @PianoPlayer88Key

    NVDIMMs are a thing, most storage array controllers have them so write cache data is written to disk on power restore. In HA setups writes aren't committed until both controllers have acknowledged the data is in NVDIMM. Controller failover with data loss would be bad if the writing client was told it had been written then actually lost out of cache.

     

    A lot of software storage solutions read cache more than they write cache and sometimes have an option to tell the system it is power protected which enables write-back cache rather than write-through.

     

    Hardware RAID handles this situation the easiest and very well, write-through default unless BBU present then write-back. Simple, automatic and easy to understand and no diving in to configuration options and they all handle it the same way unlike software solutions.

  9. Windows7ge

    Windows7ge

    Of course like most servers weather you're using software or hardware RAID a UPS is highly recommended in the event of power failure. This will reduce the likelihood of corrupt data in the pool.

     

    ZFS though does handle spontaneous power failure quite well. (My memory is foggy again, someone told me this like a year ago) I believe the way it handles it is it retains a copy of the cached data until it checks that it has written it to the pool. If a power failure occurs mid-write it can rewrite the corrupt block(s) when power is reapplied.

     

    A downside to hardware RAID (I don't know if this is true with newer controllers) is it doesn't know which blocks failed to write so it delete everything (either in the sector or-I should really re-look this up.) and rewrites it all. ZFS will only rewrite the single blocks that failed. (The idea is this causes performance loss during recovery and downtime which isn't really a big deal for a home gamer so it's actually not a big deal if you go this route)

     

    If you need a mass of ports but raid controllers are the cheapest you could do two things with them:

     

    1. Use them for hardware RAID

    2. Flash their firmware to IT mode. (Turn it into an HBA)

    3. Just run the drives in JBOD mode.

     

    I use to use the Dell PERC H310. I flashed it's firmware to IT mode. The card cost me $42.

  10. leadeater

    leadeater

    Quote

    A downside to hardware RAID (I don't know if this is true with newer controllers) is it doesn't know which blocks failed to write so it delete everything (either in the sector or-I should really re-look this up.) 

    As long as you have a BBU it'll have everything, unless it's so long the BBU drains but that's a really long ass time. Danger only comes from forcing write-back on with no BBU then you will lose data and may corrupt it, every power loss.

  11. Windows7ge

    Windows7ge

    That's not it. Maybe it was when a drive fails and it has to re-build the pool. Something about deleting all the data and rebuilding it from scratch (parity data maybe) I should really re-lookup where I read this. It's a function specific to ZFS that I cannot fully recall but to be honest it isn't that important that I go though the hassle of finding it. 

     

    Related to my update though I spoke with Electronics Wizardry about ZFS alternatives which is why I mentioned BTRFS. I told him I did get that Windows Server 2016 key for free and he was pretty adamant about me using it natively on the server as it would have the highest chance of giving me the performance I've been looking for.

     

    I'd have to use storage spaces as I have no RAID card (that's what the original picture above is about) which isn't a problem for me so long as it works reliably.

     

    I have mixed feeling about it though. I don't really want to run Windows natively. I want to play with Linux but at the same time I can see Windows making it much much easier to run and operate any additional software on the server that I may want to.

     

    Also apparently it comes with hyper-v?

     

    Everything says I should just use it considering what I want to do. The only serious question is how to setup the pool as he says parity will tank the performance but RAID10 would cut the usable storage by 50% so I really don't want to do that.

     

    As un-advisable as it may sound I'm thinking in this case I might just stripe it. I'd have it backup to another server every 30min or hour or so so I could see it working fine.

     

    Opinions? On anything above?

  12. leadeater

    leadeater

    Quote

    Everything says I should just use it considering what I want to do. The only serious question is how to setup the pool as he says parity will tank the performance but RAID10 would cut the usable storage by 50% so I really don't want to do that.

    Setup a Tiered Multi-Resilient Virtual Disk with Two-Way Mirror SSD and Parity HDD, that should give you all the performance you need. All new data and modifications of existing data happen in the fast tier. You can actually in fact for both Tiers have as SSD or HDD if you wanted to.

     

    Storage Spaces is also a bit more flexible in how you allocate storage from within a pool, the pool itself does not define the resiliency and is only a pool of disks. The virtual disks you create within the pool define the resiliency so you can have a smaller 400GB Virtual Disk pure SSD Two-Way Mirror for hosting VMs and have a larger Virtual Disk as pure Parity or if you want higher performance Tiered.

     

    Virtual Disks consume capacity from within the Pool so SSDs and HDDs can/will be used for the underlying hardware of any Virtual Disks created within that pool of storage devices. The only caveat to that is setting an SSD to Journal Mode which you need to do if you want to use pure Parity with no Tiering, Journal devices are write-back cache and storage devices can only be for Journal or for data in the Pool not both. If you have enough SSDs you can just add 4-6 of them in the Pool and set only 1 or 2 as Journal Mode but I always use Tiering myself.

     

     

    Quote

    Also apparently it comes with hyper-v?

    Yep so you can run any VMs you like for Linux. You can just use the host Windows OS only for storage to give to additional VMs, keeping a VM host clean as possible is a good idea if you're going to have it run more than 1 or 2 and just for testing purposes.

  13. Windows7ge

    Windows7ge

    Well in the event of this the objective is as fast as possible (read/write) while maintaining as much usable storage as possible. Right off the bat that suggests pure striping but a little parity would be nice even though I will have a ever updated backup on another server.

     

    Is there a configuration that allows striping with a dedicated drive for parity? (like RAID3, yes I know it's depreciated) Which also doesn't destroy write performance?

     

    I'm also only vaguely familiar with the terms you're using because I haven't used Storage Spaces before.

  14. PianoPlayer88Key

    PianoPlayer88Key

    @leadeater I think I'd heard something about NVDIMMs, could be in my extensive browsing of supermicro's site looking at their motherboards, some of which I think support them.

    @Windows7ge Yeah I like some of those ZFS features, just not entirely sure yet if it's the right platform for me for a NAS yet.  And yeah I'd probably be using some of those type of RAID controllers, like the 9200-16e or 9211-8i.  Hopefully I'd be able to get something that's already in IT mode.  (Would be afraid of bricking something by a flash gone wrong.)  Or, maybe disks in JBOD might work.

     

    My objective would be to get a backup of all my data as cheap as possible, preferably with a ratio of main HDD storage vs backup media and device pricing similar to what it was in the 1990s.  For example (from the image I posted earlier), in January 1994:
     

    Spoiler

     

    ◘ ~250 MB:

     • IDE Hard Drive = 250MB $215, 270MB $225

     • 250 MB Tape Drive = $145 - $169 (~67-81% the cost/GB vs the HDD)

     • 250 MB Tape Media = $20 (9.3-9.6% the cost/GB vs the HDD)

    ◘ ~2 GB:

     • 2.1 GB SCSI Hard Drive = $1850 (I likely wouldn't have used this back then, probably would have gone for a 340MB ($258) drive or something around there.)

     • 2.3 GB Tape Drive = $925 (~46% the cost/GB vs the HDD)

     • 2 GB Tape Media = $59 (~3.3% the cost/GB vs the 2.1 GB HDD, or ~3.9% vs the 340 MB HDD)

     

    Looking up the current pricing of some cheaper NAS HDDs, and if backup media was about 6.25% the cost/GB vs HDDs....

    • 4 TB HDD: $109.99 (Seagate IronWolf) = $27.50/TB - backup media should be $6.87 for same TB, OR 64 TB for same price.

    • 6 TB HDD: $174.99 (Toshiba N300) = $29.17/TB - backup media $10.94 OR 96 TB

    • 8 TB HDD: $219.99 (Toshiba N300) = $27.50/TB - Backup media $13.75 OR 128 TB

    • 10 TB HDD: $299.92 (Seagate IronWolf) = $29.99/TB - Backup Media $18.75 OR 160 TB

    • 12 TB HDD: $378.70 (Seagate IronWolf) = $31.56/TB - Backup Media $31.56 or 192 TB

     

    If I can get media that inexpensive per TB, I'd be willing to spend up to $800-1K on the rest of the parts.  If hard drives are the cheapest media I can get, then the rest of the parts combined would be limited to the lesser of $175-250, or such that the combined cost of server plus drives doesn't exceed what the other scenario (cheap media + backup device/server) would cost if I was able to use that option.  I would really prefer not using rackmount if it's at all possible to avoid it, and have the system be quiet enough so that a single HDD reading sequentially would be louder than everything else combined that's not a HDD, including fans under load, etc.

     

    Hey I wonder if the fine people over at level1techs forum might have some ideas of how to do a many-drive backup NAS or something like that on a tight budget....  Should I maybe ask over there?

  15. leadeater

    leadeater

    Quote

    Well in the event of this the objective is as fast as possible (read/write) while maintaining as much usable storage as possible. Right off the bat that suggests pure striping but a little parity would be nice even though I will have a ever updated backup on another server.

    Two-way Mirror is stripped, fully for reads and n/2 for writes. So if you have 6 SSDs you get 6x read and 3x write. Tier cold data down to Parity HDD and you'll have an array that can do 2-2.5GB/s read and 1/1.5GB/s write. That's my setup anyway.

  16. leadeater

    leadeater

    Also Storage Spaces supports single disk additions to the pool, only nice for home setups though.

  17. Windows7ge

    Windows7ge

    @PianoPlayer88Key

    Quote

    just not entirely sure yet if it's the right platform for me for a NAS yet

    If you want it to do anything other than be a NAS I wouldn't recommend it. It does allow for VM's but the software (Bhyve) isn't very good if you need them to do anything workload intensive.

     

    Quote

    Hopefully I'd be able to get something that's already in IT mode

    Although it's possible to buy pre-flashed RAID controllers they're not all that widely found and you'd be better off doing it yourself but you can just run them in JBOD mode.

     

    Quote

    If hard drives are the cheapest media I can get

    You can get tape storage: Example

    Cheaper than mechanical storage if you're looking to archive. I don't know how much a read/write player would cost.

     

    Quote

    Hey I wonder if the fine people over at level1techs forum might have some ideas of how to do a many-drive backup NAS or something like that on a tight budget....  Should I maybe ask over there?

    You can if you want.

     

    @leadeater

    Looking into it more I could create a high teir simple volume and a slow tier parity. If I can add 1 disk at a time I'll probably do 2-way mirrors until usable storage becomes more important that redundancy (run out of ports or bays).

     

    An important thing Electronics Wizardy mentioned is the re-distribution of the data when a new disk is added. Apparently it doesn't do that automatically but only a single powershell command is needed to take care of that.

  18. leadeater

    leadeater

    Quote

    An important thing Electronics Wizardy mentioned is the re-distribution of the data when a new disk is added. Apparently it doesn't do that automatically but only a single powershell command is needed to take care of that.

    Yep, Optimize-StoragePool will do that. Can just task schedule that to run every night and you'll never have to worry about it.

  19. PianoPlayer88Key

    PianoPlayer88Key

    Spoiler

    If you want it to do anything other than be a NAS I wouldn't recommend it. It does allow for VM's but the software (Bhyve) isn't very good if you need them to do anything workload intensive.

    Yeah @Windows7ge that depends on how much I spend on the various parts, especially the storage.

    • If I spend like $150 to $250 or so on the NAS hardware as a group, then use normal HDDs as storage, it would strictly be a cold-backup NAS.  (Every once in a while, power it on, do a backup, then shut it off and unplug it; also powering on as necessary to do a restore.)
    • If I could do the tape option like I've described, then I might spend closer to $700 or $800 or so.  Then, I'd want the option of ingesting analog media (like 16mm, super-8, VHS film, reel-to-reel & cassette tapes, vinyl records, etc) and transcoding to a modern format.  (And I have preferences on that, for example getting the best possible quality out of the analog media before / without any digital post processing, to the extent that several years ago I was eyeing some $500-1K pro-grade cassette decks on ebay, but that's a topic better saved for a future different post somewhere.)  However ... see below about my observation of tape media vs drive vs HDD relative prices now vs 25 years ago.

     

    Spoiler

    Although it's possible to buy pre-flashed RAID controllers they're not all that widely found and you'd be better off doing it yourself but you can just run them in JBOD mode.

    Ahh, yeah I haven't seen a lot of them around.  Would JBOD mode still at least have redundancy or parity or something like that, and not have any worse performance than a single drive?

    Also another feature of UnRAID I like that I was just remembering ... if you lose more drives than you have parity for, you only lose the data on the however many drives over the limit you lost, you don't lose the entire array.  I'm not sure if ZFS / FreeNAS can do that... or can it?

     

    Spoiler

    You can get tape storage: Example

    Cheaper than mechanical storage if you're looking to archive. I don't know how much a read/write player would cost.

    I looked up the prices on some of those things on Newegg, and, yeah, they're not anywhere near the price I'd be willing to pay.  (See my earlier posts for what tape vs HDD prices were like in January 1994, 25 years ago.)


     

    Spoiler

     

    Tape drives:

    Tape media:

     

    I added up the total media capacity I have, and it comes to somewhat over 91.5 TB or so.  (Some if it is empty, a few pieces are dead, and some has redundant but disorganized storage.)  I'm thinking I might have closer to 30 or 40 TB or so of actual data to back up, but let's allow for growing into my storage, so to speak.

     

     

     

    If I had, for example, 96 TB of data to back up, the maximum I could spend would be about $1200, including the media AND the drives / NAS.  (96 TB of HDDs would run $3K for the drives I'd likely buy for my main system, or a bit over $1600 if I got some ones that I'd consider questionable.)

     

     

    Spoiler

    Modern HDD starting prices according to pcpartpicker:

    • 14TB = $428.99 ($34.64/TB) = Seagate IronWolf
    • 12TB = $378.70 ($31.56/TB) = Seagate IronWolf
    • 10TB = $289.99 ($29/TB) = WD Purple
      • more likely to buy: $299.92 Seagate IronWolf, $304.92 WD Red or $319.99 HGST Deskstar NAS
    • 8TB = $214.96 ($26.87/TB) = Toshiba X300
      • more likely to buy: $219.99 Toshiba N300, $237.78 Seagate IronWolf, $248.89 HGST Deskstar NAS, or $250.20 WD Red
    • 6TB = $122.22 ($20.37/TB) = Seagate Enterprise (older drive)
      • more likely to buy: $174.99 Toshiba N300, $183.97 Seagate Ironwolf, $194.50 WD Red, $199.00 HGST Deskstar NAS
    • 4TB = $69.99 Seagate Terascale (older drive)
      • more likely to buy: $109.99 Seagate IronWolf, $118.97 Toshiba N300, $119.99 WD Red, or $126.09 Hitachi Deskstar NAS

     

    If I bought my equivalent capacity in HDDs now, 23 of the 4TB Seagate (Terascale) drives at $70 each wound run about $1610, totalling about 92TB capacity.  I'd be much more likely to buy the 8TB Toshiba N300 or HGST Deskstar NAS drives for my main system, though.  They run about $220-250 apiece, and 12 of them (96TB) would run me about $2700 to $3K.

     

    Tape media 25 years ago was about 6% or so the cost per capacity compared to hard drives.  6% of $3,000 is $180.

    So, if I spent about $1,000 on either a tape drive, or a NAS that could interface with my backup media, and another $200 or so on the media ... that's how I arrived at the $1200 total budget.

     

    Anyway .... maybe I should take this over to l1t or someplace else, where hopefully there's a higher concentration of nas / server knowledgeable people there, & maybe someone has ideas on how to do a backup for the same ratio of cost of backup vs cost of storage as we had 25 years ago.  (Also, if my budget was to go up, then 90-95% of that would go toward main computer storage, with the remaining 5-10% of the $ being for backups.)

     

    Single disk additions like what @leadeater said about storage spaces is actually one of my requirements.  I don't want to have to buy 3 or so drives at a time and make them their own separate RAID.  I could be wrong, though, but I think just the enterprise / server Windows license by itself (or is it available in Pro or Home? although even that would take a big chunk especially since I don't want to use one of those cheap keys for this type of project) would exceed what I could spend on everything combined by several times.

     

    Part of the reason I haven't even powered ON my desktop system at all since like summer 2018 or so, nor been doing hardly any video recording or other things, is because my backup situation is not resolved, and I don't currently have a solution that works for me.  (I feel like the less I use those drives, the less they're likely to fail for a while, and the less video & other stuff I record, the less I'll have need for backing it up.)

     

     

  20. leadeater

    leadeater

    Quote

    I could be wrong, though, but I think just the enterprise / server Windows license by itself (or is it available in Pro or Home? although even that would take a big chunk especially since I don't want to use one of those cheap keys for this type of project) would exceed what I could spend on everything combined by several times.

    Windows 10 Pro will do Storage Spaces, Home does too I think but I wouldn't use that. Some options you can do in Windows Server GUI have to be done in PowerShell though but it is actually easier to setup Storage Spaces correctly using PowerShell anyway.

  21. Windows7ge

    Windows7ge

    @PianoPlayer88Key

    Quote

    Would JBOD mode still at least have redundancy or parity or something like that, and not have any worse performance than a single drive?

    Today there's two definitions of the term JBOD. Originally it operated as a form of RAID but as it's acronym implies it was Just a Bunch Of Disks. This allowed you to mix drifferent drives of different capacities and even different RPM. However it was crap performance and it just just not recommended in general.

     

    Today JBOD means no RAID at all. You're not configuring RAID and you're operating each drive independently. Using a RAID controller in JBOD mode means you're not going to configure the controller and you're going to the the OS handle the drives. In the event of FreeNAS that means you'd be leaving ZFS to handle the disks.

     

    Quote

    I looked up the prices on some of those things on Newegg, and, yeah, they're not anywhere near the price I'd be willing to pay.  (See my earlier posts for what tape vs HDD prices were like in January 1994, 25 years ago.)

    If you're just looking to archive it's a lot cheaper than today's hard drives even if tape itself has become more expensive. Although that higher expense comes with much higher capacities (even if inflation has offset the value it use to have). I do however see that the players are incredibly expensive which make this a less financially viable option. Of course a little bit of digging might reveal a 3rd party player in the affordable range. 

     

    Quote

    Anyway .... maybe I should take this over to l1t or someplace else, where hopefully there's a higher concentration of nas / server knowledgeable people there, & maybe someone has ideas on how to do a backup for the same ratio of cost of backup vs cost of storage as we had 25 years ago.  (Also, if my budget was to go up, then 90-95% of that would go toward main computer storage, with the remaining 5-10% of the $ being for backups.)

    With files getting bigger and bigger and inflation making everything more expensive I'm afraid you may never find that much storage for the same ratio you had back then. Your best option here would be to explore both compression and deduplication. This may help. Also eliminating duplicate data and finding large files you know you no longer need.

     

    @leadeater

    Quote

    Yep, Optimize-StoragePool will do that. Can just task schedule that to run every night and you'll never have to worry about it.

    I only need it when I add a disk. Unless Storage Spaces has a tendency to start favoring certain disks during normal operation making it desirable to have it run daily.

  22. leadeater

    leadeater

    Quote

    I only need it when I add a disk. Unless Storage Spaces has a tendency to start favoring certain disks during normal operation making it desirable to have it run daily.

    Nope it won't actually do anything unless a disk has been added or removed, just a could do if you don't want to have to remember.

  23. Windows7ge

    Windows7ge

    Oh, I do have one question you might be able to answer. The product key I got, I did use it once in a VM about a year and a half ago (the vm has long since been inactive because I ended up having no use for the VM).

    Do you know if I should expect to have activation problems when I go to use it for the 2nd time?

  24. leadeater

    leadeater

    Should be fine

×