The ABX Myth
Monday, 4th July 2016
You want to know why?
Well you are familiar with the old Spot the Difference games in quiz books and newspapers, so imagine the ABX test done with pictures instead of audio. If you were shown picture A, then picture B with A hidden and asked to identify any differences between them, how do you think you would perform? Then make it even more difficult by not asking you for differences, but by showing you either A or B again and asking whether it was A or B? It gets a lot harder. It gets ridiculously hard if you then replicate the audio test by using moving pictures (video clips). I wouldn't be able to do it and I'm confident that no one could. Why would you even try, because it is so obviously impossible to do better than random?
You certainly wouldn't propose it as a valid test of whether anyone could tell two videos apart. Or if they were the same! My big gripe with ABX is that the test is unable to prove when the two items under test are identical. To illustrate this, there was once a test done with a valve amp and a transistor amp. Amp 'A' was played and then amp 'B' and the listeners were supposed to say which was which. Except they weren't different; they were the same transistor amplifier played twice. Predictably the resulting answers were no better than random, so the testers were just trying to make fools of those who said they could hear the difference between valve and transistor amps.
If it can't prove sameness, the test is worthless. What would you think of an ordinary ohmmeter that gave you a different reading every time you tested the same resistance? Yet that is what "experts" expect you to be able to do with a music passage.
That is the whole point of the ABX confidence trick. It is just to make fools of us.
It's about time we stopped falling for it and devised an objective test that works.
I'm working on it!
Written By Russ Andrews