Checksum Collision
Note that any form of integrity testing will not be able to detect all forms of bit rot corruption. There are cases where two different content can yield the same checksum. This effect is known as a hash collision. An acceptable trade off. While there will be instances where bit rot detection is missed, it is still a good tool to have because eventually the errors will be detected.
Ironically you’d not be able to hear a problem if you are using WAV file. As there are 44100 samples per channel per second, so every second of audio is represented by a total of 88,200 numbers. This is like trying to spot of a few bad pixels from a twenty megapixel picture. Even harder with music since a single bit error is unlikely to be resolved physically by your Hifi system during playback. It is only apparent with FLAC files because it drop the corrupted frame, leaving enough time to note the skip.
About 18 months ago I swapped out my N7700 NAS to a different machine and running the FreeNAS OS. This new machine has 16 GB of ECC RAM and is now using the more modern ZFS file system. This file system is supposed to guard against bit rot, using a process called “scrubbing“. Scrubbing is a system process that is run about once a month to detect and correct any bit errors it discover.
Two years is still too early for me to equivocally state my problem with bit rot is now completely resolved. Will continue to check the scrubbing output, as well as monitoring for corruption of any newly created files on the new NAS.
The bit rot phenomenon is very rare issue to begin with, adding to the difficulty in detection and diagnostics. It is assuring to know that scrubbing did detect (and corrupt) a single bit error once. However it is still way too early for me to ascertain just how reliable scrubbing (and it’s error correcting mechanism) is. In the worst case scenario bit correcting apparently may well corrupt my entire file system!
About The Author
4 comments
Comment from: Frank Collins Visitor
Comment from: Agent Kith Member
It does a four-weekly scrub which detects and corrects bit rot on the whole file system. While bit rot isn’t a problem for me, it never will be.+1. FreeNAS for the win.
Comment from: Agent Kith Member
Maybe it’s Cosmic Rays: https://science.slashdot.org/story/17/02/19/2330251/serious-computer-glitches-can-be-caused-by-cosmic-rays
A "single-event upset" was also blamed for an electronic voting error in Schaerbeekm, Belgium, back in 2003. A bit flip in the electronic voting machine added 4,096 extra votes to one candidate. The issue was noticed only because the machine gave the candidate more votes than were possible. "This is a really big problem, but it is mostly invisible to the public," said Bharat Bhuva. Bhuva is a member of Vanderbilt University’s Radiation Effects Research Group, established in 1987 to study the effects of radiation on electronic systems.
Comment from: Agent Kith Member
Here is an interesting YouTube video about bit flipping.
I don’t believe bit rot is a problem and the Red Book standard copes with missing bits, to a point.
I store all of my music (FLAC, level 8 compression) on a NAS, using FreeNAS. It does a four-weekly scrub which detects and corrects bit rot on the whole file system. While bit rot isn’t a problem for me, it never will be.