Bigun

I got a few 3d printers, this should work.

low-key if it was open-source and use any gpu it would be more appealing because you won't run the risk of them just dropping this project and making these setups unusable.

Hmn...I suddenly feel compelled to add a regular scrub job to crontab...

Thank you!  "Back me up" on this.
 
[/dad joke]

One minor correction, compressed video is hard to compress.  So no advantage there, but everything else stands.

Petabytes of data, Seagate drives, ZFS, no hot spares and no scrubs.  For anybody who's been in the storage industry for any length of time, this was a disaster waiting to happen.
 
Hard drives are consumables.  They're mechanical, which means they wear out, and at petabyte scale using cheap drives you can expect to be seeing drive errors on a regular basis.  If those drives aren't being managed by a software stack that expects that, corrects the faults, and manages the health of the drives constantly then it's a question of when, not if you'll be losing data.  ZFS has some good points, but unfortunately hardware management and fault recovery are two major weaknesses and there are very, very few enterprise storage companies who have made it robust enough to be a viable solution.  It doesn't re-map bad blocks well, doesn't handle large amounts of errors on drives well, and only scrubs your data, not the entire mechanical platter on the disks.  It's very good at telling you that you've lost data, but unfortunately not so good at preventing it.
 
I would strongly recommend Linus and the team reach out to Nexsan and take a look at their petabyte scale Unity range.  Those are ZFS fronted appliances that use Nexsan's high-density E-Series systems for the disk management, and that makes them probably the most bulletproof ZFS storage solution in the industry.
 
I've no connection to Nexsan these days, but I know their hardware well:  Many years ago I worked there, both in their customer service team and as a solutions engineer, and their E-Series and Beast range are to this day some of the most resilient pieces of IT infrastructure I've ever worked with.  I've seen them survive datacentre fires, floods and numerous power cuts.  Heck, back in the day some of their devices were installed in tanks.  They've been managing large scale environments with hundreds of disks for decades, and the arrays scrub every single block of every drive in the background, and automatically re-map bad blocks.  They also report cable or SAS link errors as well as drive bit errors or drive failures.  They're very, very thorough and I had customers back in the day with arrays that were 9 years old but which refused to die.
 
You can also quite literally destroy nearly the entire array and still not lose data.  You can lose the chassis, the power supplies, both controllers, and a chunk of your disks and still not lose data.  Provided you had enough disks surviving to be a valid (partial) raid set they were always able to recover.  You can literally move drives into another Nexsan array (in any order, shuffle the drives as you wish), and that array will boot up, ask if you'd like it to load your previous settings, and bring all your data back online, with all your data, volumes, IP addresses, the works restored.

Heck, even after doing all that if you were still having problems the support team would still have enough logging and disgnostic data to tell you which of your drives were healthiest, which order they failed in, what your best options are for getting data back, and they can literally force faulting drives back online while you copy data off.

Now the E-Series and Beast are block storage devices so aren't ideal for Linus Tech Tips, but the Nexsan Unity range takes those bomb proof pieces of hardware as the foundation of the storage system, and then adds all the filesharing, caching, snapshots and other goodies of ZFS on top.

It's by far and away the most reliable ZFS solution I've seen in more than a decade in the storage industry. 

1)  That's what the proposal is for, and after all the information is gathered (which you and I really don't have), that's when the decision is made

2)  That's why you cycle tapes

3)  You didn't work the math:  50 tapes @ 12TB each per library.  Each library would have 50 tapes and 3 decks each, writing simultaneously @ 300 MB/s per deck.

Again, I need more info, and I'm itching to see if this could work for them.

Why not use LTO as a cold backup option? Like the latest revision of LTO, LTO-9.
It can store 18TB of raw data without compression per tape and cost approx. $200 each. With your reputation and order size even cheaper. 
780TB will fit on ~44 drives and will cost you about 44 * 200 = 8800USD, plus you need to invest some money into LTO drive (a few thousand USD).
But this storage is like forever. It could be even cheaper if it can compress your type of data. 
 

This stuff always was black magic to me, it could be grounding issues, electrical wiring issues, shielding issues, a bad capacitor - who knows.  If this just started, I would start looking over your capacitors on the board and see if any of their tops are starting to dome out - they should be completely flat.  If you find any, they will need to be replaced.
 
If all looks good, then no telling.  A solution that should work is an external USB DAC.  Good luck!

This stuff always was black magic to me, it could be grounding issues, electrical wiring issues, shielding issues, a bad capacitor - who knows.  If this just started, I would start looking over your capacitors on the board and see if any of their tops are starting to dome out - they should be completely flat.  If you find any, they will need to be replaced.
 
If all looks good, then no telling.  A solution that should work is an external USB DAC.  Good luck!

On a side note - this is going into my post signature:
 
 

So, I have two identical setups for me and the Mrs.  We both have a Radeon R9 380 in each of our machines.  They are a little older, they are a little slower, but for the gaming we do, it's enough.... up until yesterday afternoon.
 
My wife was watching some Netflix when she said her screen went black.  When I got home I saw some BSOD screen (see attachment).  For the purpose of googling it was a Stop code: VIDEO TDR FAILURE.  What failed:  amdkmdag.sys.  
 
It kept rebooting into a cycle of black screen for a while, BSOD, then an option to either troubleshoot or reboot.
 
Going into BIOS I saw a yellow stripe artifact across both screens.

To put the nail into the coffin, I swapped the cards between me and my wife's machine - at that point mine started exhibiting the same behavior, while hers booted and started working as expected.

So now, I'm left with a choice.  Pay scalper prices for a new GPU or not game... right?  Oh hell no, I'm trying to fix this, besides, I'm single income with kids and still married - I have no extra money to speak of.

I removed the front cooler, backplate, heatsinks on the top, and anything insulating the card.  Cranked the oven to 370F (convection on), propped the card on aluminum balls inside of a cast iron skillet, and put it in for 20 minutes.
 
After baking I pulled it out, put everything back together, and put it in the PC - it worked.
 
Then this morning, I boot it up and it stopped working again - same behavior as before.
 
Story isn't over - since I have nothing to lose, I'm going to do 390F for 22 minutes.  Will report back.

Just documenting progress just in case someone else goes through this and wishes to fix their Radeon card of a similar model.  I'm hoping for success.
 
edit:
(SPOILER WARNING - it's dead)
 

Glad it was good enough as a stop gap solution!! Cheers. 

Really, you have nothing to lose at this point eh?
 
Worst case Ontario, I have an old AMD 280x I think it is? 2GB, that is just collecting dust. If you want it and pay the shipping, she's yours.

If it's only going to be used as a router, you can install OS's that are tailored to be routers.
 
As long as you meet the requirements, pfsense is a good starting place.

I assume it's running system.d since it's Ubuntu?
 
How about creating a system.d service?  http://tuxgraphics.org/npa/systemd-scripts/

Job security

Sorry about that, I just learned how to read yesterday

Sorry about that, I just learned how to read yesterday

Yep, if it's important always read it back and write to new medium to keep it on current technology and readable. Never count on the ability of a storage medium to be readable after 30+ years e.g. tape (LTO-x). Write it to LTO-5 then when LTO-7 comes out read it back and write to LTO-7, when LTO-9 comes out read it back and write to LTO-9 etc etc
 
If it's application specific data and a proprietary format then keep a copy of the software with the data.

Just know I'm not arguing with you on this because in most cases you are correct BUT when I started I had a soldering station with temp control.
I agree this kind of work needs this kind of equipment and that's why I too say a $5 soldering pen just won't cut it.
It's a simple job but you need the right tool for it.

I wasn't clear about it so that's on me.
 

Looks like a solderable part.  Are you or someone you know experienced with soldering?  

This is a ticking time-bomb.  They need to invest in personnel and should have started years ago.

Sorry about that, I just learned how to read yesterday

Sorry about that, I just learned how to read yesterday

Sorry about that, I just learned how to read yesterday