How Good Are Activity Trackers at Counting Calories Burned?

People who wear activity trackers to count the calories they’re burning are probably not getting accurate estimates, suggests a new study.

Researchers who tested seven popular activity trackers found that while heart rate measurements were generally accurate, none of the devices provided a reliable calorie count.

“At this point with this level of error, I would be wary of using that estimate to alter a calorie-controlled diet,” said senior author Dr. Euan Ashley, of Stanford University in California.

Patients “have been bringing data from these devices to us and some of us were using these devices ourselves,” Ashley told Reuters Health.

Because so little is known about the data’s accuracy, “We realized that we had to do our own study,” he said.

Ashley’s team recruited 60 healthy adults to test seven popular wrist-worn activity trackers: the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, MIO Alpha 2, PulseOn and Samsung Gear S2.

Participants wore up to four devices at a time, and they also wore laboratory devices to measure heart rate and calories burned while sitting, walking, running and cycling.

All of the devices but one had an average heart rate error rate below 5 percent. The exception was the Samsung Gear S2, which had an error rate of 5.1 percent.

But for calculating energy expenditure – or calories burned – all of the devices had error rates above 25 percent. The Fitbit Surge had the lowest average error rate for calories burned at about 27 percent. The PulseOn had the highest at about 93 percent, according to a report in the Journal of Personalized Medicine.

Overall, the Apple Watch had the lowest error rates while the Samsung Gear S2 had the highest.

The researchers were surprised at the unreliability of the calorie counts.

The devices “were literally all over the map with error rates,” Ashley said.

Data tended to be less accurate for men, people with higher body weights and darker skin tones, and while walking.

Ashley’s team hopes the devices’ calorie counting technology will improve. “I think we’re all hopeful that as we move forward they will get better,” he said.

In a statement to Reuters Health, Fitbit said its trackers show an estimated total number of calories. “Fitbit uses a scientifically validated estimate of (basal metabolic rate) based on height, weight, age, and gender information that users provide when setting up their Fitbit account,” said the statement, which added that the measure also takes into account people’s heart rates.

“While the Mio ALPHA 2 was designed for the individuals focused on heart rate zone training, and not for all-day activity tracking, we agree that more accurate calorie estimation is important for the industry as a whole, since most individuals are monitoring calorie deficits for weight loss,” said Mark Gorelick, chief science officer at Mio Global, in a statement.

Markku Lankinen, who is head of operations for PulseOn Oy, said in an email that the researchers may not have adjusted the device specifically for each participant. “With PulseOn device, you would need to apply these user parameters in the application before exercising, and this seems not to have been done,” said Lankinen. “This causes the (energy expenditure) estimates to be badly off.”

Apple, Microsoft and Samsung did not provide comments for publication. All Basis Peak watches were recalled in 2016 due to overheating, according to its website.

Ashley’s team is currently conducting a study to test the accuracy of devices in the real world.