Home > Reviews
How We Test |
|
|
Published on April 05, 2008 Comment on this |
| Our approach to testing things like sound quality is to use scientific testing methods. While many other testers rely on "golden ears" (those with good hearing who subjectively judge headphones), we use proven scientific test methods developed by industry experts. This means that we can not only quote specific numbers for tests (instead of saying that the music sounded bassy, we can quote a specific frequency response), but that we can run exactly the same tests and get the same results year by year. Audio performance is the most important thing we test on headphones, so we put a lot of time and effort into testing the quality of the sound that headphones produce. We evaluate the electroacoustic performance of headphones (meaning the details of the sound produced) objectively using a test system from Listen, Inc which is compromised of a SoundCheckTM, electroacoustic test system, and a Brüel & Kjær Head and Torso Simulator (HATS). This enables us to do precise, scientific testing of headphones, rather than the subjective testing that is often used. These components of our testing system are described in more detail below. SoundCheck  SoundCheck, from Listen, Inc. is an electroacoustic measurement and analysis package widely used for testing audio devices, both on the production line and in R&D applications. It is a PC and sound card based system which communicates with both analog and digital equipment using standard, non-proprietary interfaces.The Brüel & Kjær hardware is directly controlled through SoundCheck, with the test sounds passing from the soundcard to the high-end audio amplifier we use to drive the headphones (a Crown D45). Tests are fully programmable, enabling the sound signal, various analysis methods and result output format to be selected.  Head and Torso SimulatorThe Head and Torso Simulator (HATS) is a mannequin with built-in ear and mouth simulators that provides a realistic reproduction of the acoustic properties of an average adult human head and torso, including how it absorbs and reflects sound. It incorporates an artificial mouth and ear with pinna (the visible part of your ear that acts to gather sound). We mount the headphones on the HATS in the same way that they are mounted on a human head, and the sensitive microphones in the ears of the HATS pick up the sound from the headphones and feed this back to the SoundCheck system through two high-end microphone amplifiers. The entire system is calibrated and tested on a regular basis. Test Configuration
 What We Test Our audio quality tests are broken down into 6 sections: frequency response, distortion, tracking, sound pressure level, isolation and leakage. Below, we describe the tests we do for each of these sections in turn. Frequency Response Frequency response is how well a pair of headphones responds to sounds at different frequencies. To test this, our test system sends a frequency sweep to the headphones of 20 to 20,000Hz, which covers the entire frequency range of the human ear. The HATS picks up the sound that the headphones produce, and the SoundCheck system analyzes the sound to determine how much of the sound was output by the headphones. This produces a graph like the one below, which shows the frequency response of the headphones from low frequencies (at the left) to high frequencies (at the right). The two lines are for the left (in green) and right (in red) channels of the headphones. The response curve we feature includes a diffuse free field correction, which corrects for the effects of the head, torso, pinna and ear canal on the response curve.  There is no ideal response for headphones, so there is not a single response curve that we test against. Instead, we set a range of +/- 6 dBSPL in the frequency range of 500Hz to 9Khz, and our testing system puts these these limits against the response curve of the headphones. Our scores are then based on how much the headphones go outside these limits, so we are not scoring on the exact curve, but rather on the smoothness of the curves. Although headphones are often regarded as being a matter of personal taste, a good pair of headphones should produce a good, clean frequency response that does not overly exaggerate or diminish nearby frequencies to accurately reproduce the music you are listening to. We test the headphones frequency response at an average level of 78 dbSPL, measured at 1Khz. This represents a typical listening level for someone listening in a quiet room or other low noise environment. Distortion Distortion is a measure of how accurately the headphones reproduce the waveforms that make up the music you listen to; if they clip, compress or otherwise mangle the waveform, your music won't sound the way it should. Using the SoundCheck system, we measure something called Total Harmonic Distortion (THD), which relates to how well the headphones reproduce not only the fundamental frequency itself, but on the higher frequency harmonics that accompany it. A decent set of headphones should be able to accurately reproduce both the fundamental frequency, and the harmonics at the same level as they are present in the original sound, and that's what we set out to test. Our automated testing system measures the THD of headphones at 90 dBSPL and produces a graph like this, which shows the THD for both channels (the left in green and the right in red) for the frequency range of 20 to 20,000Hz. Big peaks in this graph indicate that the headphones are having a particular problem with the harmonics of that frequency, and that notes at that frequency may sound buzzy or rough, like a distorted guitar. THD is measured as a percentage, indicating how much the original waveform is distorted . The bottom line is that lower is better here; a headphone with less distortion will produce more accurate sound.  Our distortion score is based on the total amount of distortion across the frequency range; the higher the graph (and any peaks on it), the lower the score. However, it is worth remembering that in the complex assortment of frequencies that is music, you are very unlikely to be able to detect low levels of distortion. Tracking Headphones have two channels for your two ears; the left and the right. And both of these channels should sound the same, so we test this to make sure. Our testing system analyzes the sound both channels produce, and creates a graph that shows any differences between the two as a percentage of the frequency response.  This shows what the difference between the sound produced in the two channels is over a frequency range of 20 to 20,000Hz is. A difference of a couple of percentage points here isn't a problem, but we don't expect to see anything above 4 to 5 percent; that could make the sound feel unbalanced and unpleasant. Our score for this test is based on the maximum imbalance between the two channels; the bigger the peak or valley on the graph, the lower the score. Maximum Usable Volume In this test, we examine how high the volume on the headphones can go before they reach a peak THD of 3%. The SoundCheck test sequence gradually increases the level of the stimulus and analyzes the THD until it reaches 3 percent, or the level of the sound reaches 120 dBSPL. We don't test any higher than that because that is as high as these headphones are ever likely to be (or ever should be) used; that level of sound pressure could cause permanent hearing damage with long term exposure. Isolation You often wear headphones because you want to block out the outside world; you'd rather listen to your own music than the conversations of the people next to you. So, we test how much of these outside sounds the headphones block. We do this by placing a speaker next to the HATS and playing pink noise, which has equal power across the entire frequency range.  Our SoundCheck system measures the detected noise in the HATS with no headphones present, with headphones present (but the active noise cancellation disabled) and with the headphones on and the noise cancellation enabled. We then subtract the last two curves from the first one, which produces a graph like this:  This shows how much of the sound was blocked ; the higher the line, the more sound was prevented from reaching the ears, and the more effective the noise canceling feature was. Again, this shows the frequency range of 20 to 20,000Hz, which covers the entire range of human hearing. In the case of headphones that include active noise cancellation (such as the Bose QuietComfort2 and QuietComfort3), we feature a graph with two lines; one for the active noise cancellation disabled (the green) and one for the noise canceling enabled (the blue).  Our score is based on how well the headphones block the noise; the more they block, the better the score. With active noise canceling headphones, we use whichever test provides the best results as the basis for our score; it is invariably the active noise cancellation one. Leakage If you're trying to block out the world by listening to music, it's only polite to return the favor; the person sitting next to you on the bus wants to hear their own conversation, not your music. So, we also test the leakage of headphones; how much of the sound they output escapes into the air around your head. To do this, we set up a senstive microphone to measure the sound level from a few inches from the HATS with no sound coming out of the headphones, then we measure the sound level with about 90 dBSPL of pink noise playing back over the headphones. We then calculate the difference between the two sound levels; the bigger the difference, the more the headphones are contributing to the aural pollution of the world. Our scoring for this is based on this difference; the more sound the headphones leak, the lower the score.  We feel that this comprehensive set of tests is the most thorough and comprehensive in the industry. It also allows us to do direct comparisons of headphones, even if they are not tested at the same time.
|
|
||||
