Didn't read your linked study, the linked blog post threw out the actual data though. Not surprising you get nothing of use when you start with:
> Each attribute also has a feature named raw_value, but this is discarded due to inconsistent reporting standards between drive manufacturers.
Sure, raw_value is inconsistent between manufacturers and sometimes models, but it's the most real data.
Edit to add: reread the Google survey, which says more or less bad sector counts were indicative of failure in about half the cases; in 2007.
One thing to note is drives made now may perform differently than those in the 2007 survey. Different firmware, different recording, better soldering (200x is full of poor soldering because of RoHS mandating lead free solder in most applications, and lack of experience with lead free solder leading to failing connections).
I found, in 2013ish, with a couple thousand WD enterprise branded drives that just looking at the sum of all the sector counts and thresholding that was enough to catch drives that would soon fail. If your fleet is large enough, and you have things setup so failed disks is not a disaster (which you should!), it's pretty easy to run the collection and figure out your thresholds. IIRC, we would replace at 100+ damaged sectors, or if growth was rapid. Some drives in service before collecting smart data did seem to be working fine at around 1000 damaged sectors though, so you could make a case for a higher threshold.
SSDs on the otherhand seemed to fail spectacularly (no longer visible from the OS), with no prefailure symptoms that I could see. Thankfully the failure rates were much lower than spinning disks, but a lot more of a pain.
Hmm, one could encode manufacturer and maybe even model as a categorical or one-hot encoded variable. Then again, the small number of failed drives in their dataset would probably make this hard.
> Each attribute also has a feature named raw_value, but this is discarded due to inconsistent reporting standards between drive manufacturers.
Sure, raw_value is inconsistent between manufacturers and sometimes models, but it's the most real data.
Edit to add: reread the Google survey, which says more or less bad sector counts were indicative of failure in about half the cases; in 2007.
One thing to note is drives made now may perform differently than those in the 2007 survey. Different firmware, different recording, better soldering (200x is full of poor soldering because of RoHS mandating lead free solder in most applications, and lack of experience with lead free solder leading to failing connections).
I found, in 2013ish, with a couple thousand WD enterprise branded drives that just looking at the sum of all the sector counts and thresholding that was enough to catch drives that would soon fail. If your fleet is large enough, and you have things setup so failed disks is not a disaster (which you should!), it's pretty easy to run the collection and figure out your thresholds. IIRC, we would replace at 100+ damaged sectors, or if growth was rapid. Some drives in service before collecting smart data did seem to be working fine at around 1000 damaged sectors though, so you could make a case for a higher threshold.
SSDs on the otherhand seemed to fail spectacularly (no longer visible from the OS), with no prefailure symptoms that I could see. Thankfully the failure rates were much lower than spinning disks, but a lot more of a pain.