We created our own headlamp testing standards to address this issue. Here are the key elements in our testing philosophy:
- Do not rely on manufacturers' claims.
- Test all the finalist products ourselves, hands-on, and in a consistent manner.
- Perform field tests as well as lab tests to assure fair ratings.
- Field tests include trail finding tests, close-proximity tests, gloved tests, and ease-of-use tests.
- Test brightness levels with industrial light meters, measured in the same way for every product.
- Provide photos of beam distance and beam diameter in spot and close proximity mode.
- Provide graphs of battery run-time performance degradation using our proprietary “light coffin” which measures brightness each minute with a data logging light meter, to reveal how beam distance degrades as batteries get weaker.
Beam Distance Testing
We measured each headlamp's maximum brightness using the same industrial light meter (Extech LT300) in units of lux measured at a distance of four meters. Tests were conducted in a controlled environment to eliminate variance due to changing temperature and humidity levels. We used fresh alkaline batteries from the same battery manufacturer (Energizer Alkaline AA and AAA batteries). Brightness was measured one minute after turning on each light with fresh batteries.
To convert these brightness measurements to beam distance, we used a similar calculation method as described on the Black Diamond website in 2010 (our calculation differed very slightly since we measured Lux at four meters distance instead of two meters).
Below is an excerpt from an article on the Black Diamond website blog in 2010:30 Mar 2010, 1:31PM
…Our distance measurements are calculated by determining the maximum “lux” level at two meters. We then use the inverse proportion law (rule) to extrapolate the distance the light beam will provide a 0.25 lux measurement. This 0.25 lux is the equivalent of the amount of light that is produced by a full moon on a clear night, which is a light level most people have experienced. The distance measurements and the manner in which we calculate them are the standard followed by all major headlamp and lantern manufacturers. Black Diamond also calculates battery life (burn-time) by running the headlamp or lantern until a light level of 0.25 lux at two meters is reached.
If you really care, here is the math for distance calculation:
E=i/(D²). [The D² is D "squared"]
E is the illuminance in “lux”, “i” is the luminous intensity (in candela) and D is the distance in meters, from which the measurement was taken.
We measure an Icon [2010] headlamp to read 651 lux at 2 meters. Therefore the luminous intensity is: 651=i/(2²) or i=651 x 4 = 2604 candela.
To calculate the distance the light beam will read 0.25 lux, we substitute these values into the equation:0.25 = 2604 / D² or D² = 2604 / 0.25 or D = SQRT(10,416), thus D = 102.05 meters.
There will a quiz on this tomorrow.Being skeptics by nature, we were unwilling to use this calculation based on brightness to determine beam distance until we could verify the math in practice. To do so, we walked into an empty field with our light meter on a moonless night and tested a handful of headlamps manually to be sure the predicted beam distance based on our measurements at four meters matched what we observed in the field. It did.
Battery Run-Time Tests
To measure battery run-time, we built a light-proof box we call the OutdoorGearLab Light Coffin which is instrumented with a data logging light meter. Each headlamp is placed at a fixed distance from the light meter, which is in turn connected to a laptop to record data. Fresh batteries are used for every test. We close the box and begin measurement one minute after the light is turned on. This setup allows us to measure brightness once a minute and see how performance degrades as batteries drain. Using similar math as described above, we convert the recorded data into a graph of beam distance over time (see below).
The data from our Light Coffin tests also allows us to determine battery run-time as defined by the American National Standards Institute protocol (ANSI) FL1 Flashlight basic performance standard.
The ANSI FL1 standard calls for battery run-time duration to last until the brightness level diminishes to 10% of the initial maximum value. As noted in our accompanying article, Why Headlamp Claims Are Deceptive, the manufacturers have refused to use the ANSI FL1 standard for battery run-time (that they helped develop) and instead use their own method, which we believe overstates battery run-time. We think you'll get a better picture of how lights compare in battery run-time by relying on our test results.
Field Testing
The most fun part of this review was getting to use all these different headlamps in the field over a months-long period. We scored trail finding, close proximity, ease-of-use, and gloved-use ratings the old fashioned way. We donned lights and used them on trails, around the campsite, and in tents. Field test ratings involve a subjective judgement of performance, and to increase reliability, we deployed a team of reviewers to score each light in these different hands-on test scenarios. We also received input from the Marin County Search and Rescue Team who graciously gave us feedback on light performance on one of their training exercises. Lastly, we put all the headlamps in a backpack and headed out for a l hike on a moonless night, alternating between the lights and making notes on each.
Beam Photos
To help you see how each headlamp compares, we took photos of each shining down the same bike path on a moonless night. These photos allow you to see the relative performance of each beam.
In addition, we took photos of the beam diameter pattern in both the high mode of the light (using the spotlight, if available), and the close-proximity mode using the low intensity flood light. These photos allow you to see if there are aberrations in the light pattern, as well as the relative beam width. These photos were taken with the light one meter from the target screen. The scale on the screen is in centimeters. Camera settings were fixed for each type of photo test, but a different setting standard was used for the very bright spotlight photos versus the dimly lit floodlight photos to allow beam diameter and quality to be visible in both cases.
Weight Comparison
Rather than rely on manufacturers' weight measurements, we measured the weight of each product on the same scale, with batteries installed.
