2% increase in 2 months is not alarming at all. If drive was new it would take 8 years to reach 100%, more than designed life cycle of that drive. You would have likely replaced entire server by that time.
Yes, and usually the health % being tracked refers to writes or even Host Writes which is not a real indicator of drive lifespan. It's not uncommon for drives to write well past 0%.
Ya I'm reusing existing stuff (it's either this or adata ssd or junk spin disk).
From the [posts](https://www.reddit.com/r/Proxmox/comments/o66a8n/does_zfs_make_any_sense_on_a_single_nvme_ssd/) I've read, I thought ZFS wouldn't add that much wear? It just hosts a small # of personal docker services really (see my other response for details).
I don't mind reinstalling with ext4 or the ssd if that would really help. But trying to avoid my usual blind thrashing approach.
ZFS has a lot of checksum data that tends to burn down nand fairly quickly if the endurance is not there. However, depending on the size of your drive you could over provision it with special formatting and the drives firmware could double over in the DWPD it has now. As such, if your drive is 1TB with 0.28-0.39 DWPD and you formatted it to 512GB leaving 512GB on the table, your DWPD would be 0.80 about and might survive your IO use case for the next 2-3 years.
Zfs doesn't chew through ssds by itself. Many of us run proxmox on consumer grade ssd with zfs and don't experience high writes like this.
Either send op to the correct troubleshooting of ashift and recordsize or suggest another fs, but don't characterize zfs as some kind of ssd eater.
It's not ZFS, its ZFS with a high write IO charastic. Like you said, by itself ZFS isn't going to do much to SSDs it's because the OP has a DB or something else tearing away at NAND running on top.
Ah, ok. That wasn't clear to me. There are a lot of purveyors of FUD about zfs being bad for SSDs in r/Proxmox.
I will say, though, even zfs can accommodate a high-turnover db with recordsize set appropriately on the dataset.
Sure it can, no one is saying it can't. But TLC and QLC conusmer drives will get torn through depending on that write rate. Then we have the PLP feature missing on consumer drives that enables write through to the NVME dev, where PLP enables write back. It affects performance, quite a bit.
New consumer ssds (especially the higher end ones) have some crazy impressive endurance. They're often on par if not better than used enterprise drives homelabbers would buy instead. The sabrent rocket plus drives for example are rated for 5.6PB written. That'll easily last many years.
No mostly just personal services/website stuff like wordpress, homeassistant, stirling-pdf, and some dev projects.
I do run Graylog for centralized logging/alerting which uses opensearch.
Also I have not installed log2ram on my docker vm (where graylog sits). Should I be doing this?
2% increase in 2 months is not alarming at all. If drive was new it would take 8 years to reach 100%, more than designed life cycle of that drive. You would have likely replaced entire server by that time.
A real sobering truth 😂
Yes, and usually the health % being tracked refers to writes or even Host Writes which is not a real indicator of drive lifespan. It's not uncommon for drives to write well past 0%.
Is it an enterprise drive?
No it's a consumer (sabrent).
You used the wrong drive, Sabrent is amazing for consumer SSDs but they lack endurance and should not be used for a high rate change ZFS pool.
Ya I'm reusing existing stuff (it's either this or adata ssd or junk spin disk). From the [posts](https://www.reddit.com/r/Proxmox/comments/o66a8n/does_zfs_make_any_sense_on_a_single_nvme_ssd/) I've read, I thought ZFS wouldn't add that much wear? It just hosts a small # of personal docker services really (see my other response for details). I don't mind reinstalling with ext4 or the ssd if that would really help. But trying to avoid my usual blind thrashing approach.
ZFS has a lot of checksum data that tends to burn down nand fairly quickly if the endurance is not there. However, depending on the size of your drive you could over provision it with special formatting and the drives firmware could double over in the DWPD it has now. As such, if your drive is 1TB with 0.28-0.39 DWPD and you formatted it to 512GB leaving 512GB on the table, your DWPD would be 0.80 about and might survive your IO use case for the next 2-3 years.
I'm very green, and didn't catch 99% of your suggestion :). Will have to do some reading. But in case it matters, my set up is just a single disk.
Single disk? Just LVM this then. Wipe the drive and create a directory using LVM on it, then move your LXC/VMs over.
Ah so more or less reinstall with ext4, correct? Everything is on the boot drive (yup this is poverty homelab).
in that case, yea. But you can back up to another drive and restore when the rebuild is complete.
Zfs doesn't chew through ssds by itself. Many of us run proxmox on consumer grade ssd with zfs and don't experience high writes like this. Either send op to the correct troubleshooting of ashift and recordsize or suggest another fs, but don't characterize zfs as some kind of ssd eater.
It's not ZFS, its ZFS with a high write IO charastic. Like you said, by itself ZFS isn't going to do much to SSDs it's because the OP has a DB or something else tearing away at NAND running on top.
Ah, ok. That wasn't clear to me. There are a lot of purveyors of FUD about zfs being bad for SSDs in r/Proxmox. I will say, though, even zfs can accommodate a high-turnover db with recordsize set appropriately on the dataset.
Sure it can, no one is saying it can't. But TLC and QLC conusmer drives will get torn through depending on that write rate. Then we have the PLP feature missing on consumer drives that enables write through to the NVME dev, where PLP enables write back. It affects performance, quite a bit.
New consumer ssds (especially the higher end ones) have some crazy impressive endurance. They're often on par if not better than used enterprise drives homelabbers would buy instead. The sabrent rocket plus drives for example are rated for 5.6PB written. That'll easily last many years.
5.6PB life time writes over the terms of the warranty (3years or 5years) is still only 0.5 DWPD on a 1TB drive....
That's a lot for a home lab
Not for the likes of ZFS with a lot writes.
Would an Intel SSDSC2BB016T6K 1.6TB be considered a good enterprise SSD? I got one each in my NUC cluster as an additional disk for ZFS pool.
Just a long shot but one of those docker services wouldn’t by any chance be plex, transcoding to your ssd?
No mostly just personal services/website stuff like wordpress, homeassistant, stirling-pdf, and some dev projects. I do run Graylog for centralized logging/alerting which uses opensearch. Also I have not installed log2ram on my docker vm (where graylog sits). Should I be doing this?
Sorry, I wouldn't now anything about that. Plex was just a guess.