Jump to content

16GB optane for proxmox boot drives?

 

So it seems proxmox does not play  well with standard cheap SSDs.

I have a had another that looks like failing.

Presently I have  2 x intel optane 16GB as  deduplicaton devices on a truenas scale virtual machine.

 

I will maybe back up EVERYTHING, recreate the NAS with no de-duplicaton special devices.

They are about 4.5GB used each.  It works well.

Instead it just use the memory instead - for deduplicaton data.  It plays nice as it is for 1MB block size ISCSI devices

Those two 16GB optane  I wish to use as proxmox boot drives.

As I understand they have high endurance & very cheap. read & write speeds are not so great.

 

The server is for entertainment  related purposes & not for  livelihood related.

 

So  the plan is  for proxmox.

 2 x 16GB intel optane mirrored as boot drive.

 3 x 500GB standard sata SSDs in raidZ1 for virtual machines.

 

Also a sata SSD adapter for hard drives - to pass through to truenas scale.

 

The above hardware I have already.

 

So proxmox users - a question - is 16GB enough for a boot drive?

 

If practical , will buy some more very cheap 16GB o instead of failures.

Also more of the cheapest 500GB sata SSDs when one fails & replace.

 

Thanks

 

 

 

 

 

Main Machine: CPU: 5800X3D  RAM: 32GB  GPU: RTX 3080  M/B: ASUS B550-E Storage: 2 x 256GB NVME boot, 1/2 TB NVME OS: Windows 10, Ubuntu 22.04

Server1:  M92p micro  CPU: i5-3470T  RAM: 8GB OS: Proxmox  Virtual Machines: Opnsense router, LXC containers: netboot server, download manager

Server2: CPU: 3600X  RAM: 64GB M/B MSI B450 Tomahawk  OS: Proxmox  Virtual machines: Windows 10, 3 x Ubuntu Linux, Truenas scale (16TB logical storage)

Link to comment
Share on other sites

Link to post
Share on other sites

I have done this exact setup, and it works fine.

 

But the 16GB optane drive really don't have that much endurace ~180TB, so a normally ssd will be pretty simmilar. And the optane drives will go read only at about 10% more than the rated writes. 

 

Proxmox only writes about 10TB a year on the boot drives, so a standard ssd should work fine. 

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, Electronics Wizardy said:

I have done this exact setup, and it works fine.

 

But the 16GB optane drive really don't have that much endurace ~180TB, so a normally ssd will be pretty simmilar. And the optane drives will go read only at about 10% more than the rated writes. 

 

Proxmox only writes about 10TB a year on the boot drives, so a standard ssd should work fine. 

Interesting.

The 240GB proxmox sata SSD I got about 6 months ago is beginning to fail now.

I had a 240GB before that, for about 2 years, which failed.

They are getting worse and worse in terms of quality as time goes on.

It is not a surprise, using 4 bits/ cell and  no DRAM cache.

 

SSDs seem to have gone up in price recently which is annoying.

 

Intel 16GB optane are very cheap.


As stated is for a home server geared to entertainment, so  do not wish to spend much.

 

How long were you using 16GB optane as boot drives?

Did they fail?

 

Thanks

 

 

 

Main Machine: CPU: 5800X3D  RAM: 32GB  GPU: RTX 3080  M/B: ASUS B550-E Storage: 2 x 256GB NVME boot, 1/2 TB NVME OS: Windows 10, Ubuntu 22.04

Server1:  M92p micro  CPU: i5-3470T  RAM: 8GB OS: Proxmox  Virtual Machines: Opnsense router, LXC containers: netboot server, download manager

Server2: CPU: 3600X  RAM: 64GB M/B MSI B450 Tomahawk  OS: Proxmox  Virtual machines: Windows 10, 3 x Ubuntu Linux, Truenas scale (16TB logical storage)

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, ianm_ozzy said:

How long were you using 16GB optane as boot drives?

Did they fail?

I have utilized 3 sticks of Optane 16GB in my server, 1 of which as boot drive for TrueNAS Scale, and 2 as cache drives, since early January. They have done pretty well so far.

All electronics are probable to fail, including Optane. It is said that 2nd-gen Optane M10 series may be more likely to have early defects, but they have double durability and lower power consumption, compared to first-gen Optane. Even with 182.5TB of durability, Optane sticks can sustain over 10K cycles, equivalent to SLC Flash chips.🤔

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, ianm_ozzy said:

Interesting.

The 240GB proxmox sata SSD I got about 6 months ago is beginning to fail now.

I had a 240GB before that, for about 2 years, which failed.

They are getting worse and worse in terms of quality as time goes on.

It is not a surprise, using 4 bits/ cell and  no DRAM cache.

 

SSDs seem to have gone up in price recently which is annoying.

 

Intel 16GB optane are very cheap.


As stated is for a home server geared to entertainment, so  do not wish to spend much.

 

How long were you using 16GB optane as boot drives?

Did they fail?

 

Thanks

 

 

 

How many writes did those drives have reported? What issues do they have? Could just be your drives, as I have deployed a lot of SSDs with a much lower failure rate than that.

 

OPtane drives should work fine if you can get them cheap, but other cheap larger drives should also be fine. I've bought a lot of used server ssds without issues.

 

1 hour ago, Bersella AI said:

All electronics are probable to fail, including Optane. It is said that 2nd-gen Optane M10 series may be more likely to have early defects, but they have double durability and lower power consumption, compared to first-gen Optane. Even with 182.5TB of durability, Optane sticks can sustain over 10K cycles, equivalent to SLC Flash chips.🤔

The 10k cycle count doesn't matter cause they basically self destruct after 110% of the rated write limit.

Link to comment
Share on other sites

Link to post
Share on other sites

I have the boot drive 'behaving' for now. I reduced the memory in the VMs, and turned of trivial ones.
I had various errors, like the boot turning into a read only filesystem, or kernel panic.

Now there is 0 swap space used, so no need to rapid reads/writes.
It it those cheap SSDs struggling with proxmox.

What I really need to know is a 16GB drive enough for the boot drive,.
I doubt it will have any I/O delay issues.


So has anyone actually been using 16GB optane as boot drives for proxmox - long term?

Main Machine: CPU: 5800X3D  RAM: 32GB  GPU: RTX 3080  M/B: ASUS B550-E Storage: 2 x 256GB NVME boot, 1/2 TB NVME OS: Windows 10, Ubuntu 22.04

Server1:  M92p micro  CPU: i5-3470T  RAM: 8GB OS: Proxmox  Virtual Machines: Opnsense router, LXC containers: netboot server, download manager

Server2: CPU: 3600X  RAM: 64GB M/B MSI B450 Tomahawk  OS: Proxmox  Virtual machines: Windows 10, 3 x Ubuntu Linux, Truenas scale (16TB logical storage)

Link to comment
Share on other sites

Link to post
Share on other sites

21 hours ago, ianm_ozzy said:

 

So it seems proxmox does not play  well with standard cheap SSDs.

I have a had another that looks like failing.

Presently I have  2 x intel optane 16GB as  deduplicaton devices on a truenas scale virtual machine.

 

I will maybe back up EVERYTHING, recreate the NAS with no de-duplicaton special devices.

They are about 4.5GB used each.  It works well.

Instead it just use the memory instead - for deduplicaton data.  It plays nice as it is for 1MB block size ISCSI devices

Those two 16GB optane  I wish to use as proxmox boot drives.

As I understand they have high endurance & very cheap. read & write speeds are not so great.

 

The server is for entertainment  related purposes & not for  livelihood related.

 

So  the plan is  for proxmox.

 2 x 16GB intel optane mirrored as boot drive.

 3 x 500GB standard sata SSDs in raidZ1 for virtual machines.

 

Also a sata SSD adapter for hard drives - to pass through to truenas scale.

 

The above hardware I have already.

 

So proxmox users - a question - is 16GB enough for a boot drive?

 

If practical , will buy some more very cheap 16GB o instead of failures.

Also more of the cheapest 500GB sata SSDs when one fails & replace.

 

Thanks

 

 

 

 

 

As a boot drive they will most likely be fine if all other storage are on other volumes and you don't have heavy logging on. 

The thing you might have an issue with is the USB hard drives. Getting some of those to pass through is not easy and I couldn't get my 5 bay enclosure to pass through as it would only randomly see 2 of the 5 drives. 

Link to comment
Share on other sites

Link to post
Share on other sites

I’m running proxmox on a pair of Samsung 980’s…. It’s been fine for 1.5 years, as has my friends running for the same time on a 980 as well, and another buddy for about a year running on a SATA Samsung drive. 

Rig: i7 13700k - - Asus Z790-P Wifi - - RTX 4080 - - 4x16GB 6000MHz - - Samsung 990 Pro 2TB NVMe Boot + Main Programs - - Assorted SATA SSD's for Photo Work - - Corsair RM850x - - Sound BlasterX EA-5 - - Corsair XC8 JTC Edition - - Corsair GPU Full Cover GPU Block - - XT45 X-Flow 420 + UT60 280 rads - - EK XRES RGB PWM - - Fractal Define S2 - - Acer Predator X34 -- Logitech G502 - - Logitech G710+ - - Logitech Z5500 - - LTT Deskpad

 

Headphones/amp/dac: Schiit Lyr 3 - - Fostex TR-X00 - - Sennheiser HD 6xx

 

Homelab/ Media Server: Proxmox VE host - - 512 NVMe Samsung 980 RAID Z1 for VM's/Proxmox boot - - Xeon e5 2660 V4- - Supermicro X10SRF-i - - 128 GB ECC 2133 - - 10x4 TB WD Red RAID Z2 - - Corsair 750D - - Corsair RM650i - - Dell H310 6Gbps SAS HBA - - Intel RES2SC240 SAS Expander - - TreuNAS + many other VM’s

 

iPhone 14 Pro - 2018 MacBook Air

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×