New Smartphone App Made by Ex-Google Engineer Calculates Calories in Any Meal by Scanning a Photo
Anyone who’s eaten out in the past few years has felt and maybe caved to the urge to snap a pic of a particularly well-plated or photogenic entrée for social media. But now, there might finally be a practical and legitimately constructive reason to photograph all those perfectly seared veggies, juicy burgers, or towering desserts with your phone.
Former engineers at Google and defence contractor Raytheon have created a cellphone app that can count the calories in any meal by simply taking a photo. Its makers say SnapCalorie — which is already available for download on Apple’s App Store and Google Play, for free, with a $29.00-per-month premium option — is better at eyeballing the calorie content of a dish than ‘professional nutritionists.’
In a quick test at the office, however, the source noticed that SnapCalorie underestimated a side of broccoli on a lunch plate, recording a photo of the vegetable, which was lightly buttered, as only steamed.
‘Human beings are terrible at visually estimating the portion size of a plate of food,’ according to SnapCalorie’s co-founder, who hopes the app will find wide adoption among dieters too afraid to eat out in case they go over their calorie limit.
As of this month, Index Ventures, Y Combinator and even CrossFit CEO Eric Roza himself, have collectively invested a total of $2 million into the team’s new start-up. The funders have seen something in the company’s co-founders, former Google AI engineer Wade Norris and ex-Raytheon engineer Scott Baron, as well as their calorie-counting AI, Roboflow.
But SnapCalorie’s true secret is its special dataset, Nutrition5k, which the company produced itself by taking nutritional data on 5,000 popular, real-world meals and pairing that data with thousands of photos and videos of those meals, taken via a robotic rig.
Nutrition5k was used to train a computer vision algorithm to accurately guess the caloric content of everything from soups and burritos to trickier ingredients like oils and ‘mystery sauces.’
In an academic paper on training their algorithm with the Nutrition5k dataset, Wade and his team estimate that their AI’s average combined error for calories, food mass and macronutrient mass is about 16.5 per cent. For comparison, their research found that the equivalent average error by a professional nutritionist was 41 per cent, and 53 per cent for an ordinary person.
Before SnapCalorie, the company’s co-founder Wade Norris had pioneered Google’s Google Lens project — another computer vision algorithm-powered tool, one which pulled up public information on objects identified out in the world. With SnapCalorie, Norris said he was looking for a way to leverage that tech to improve people’s lives.
Accuracy is very much SnapCalorie’s core goal, with its AI aided in measuring portion size, not just by modern phone camera’s unique depth sensors, but also by a team of expert human reviewers for ‘an added layer of quality.’
‘On average, the team can reduce the caloric error to under 20%,’ Norris told TechCrunch this week. There are other apps capable of using AI to do photo-based meal tracking,’ he noted, ‘but none of them helps with portion size estimation — the most important part to reduce error.’
SnapCalorie’s promises of unheard-of accuracy will have to prove, if its makers wish to make a name for themselves in the currently crowded market: apps deploying calorie-counting AI are already widely available to fitness-conscious consumers, including Lose It, Foodadviser, Bite.AI, and Calorie Mama, although their results have been called into questions by nutrition experts.
One 2020 study in the Journal of Medical Internet Research’s Formative Research compared the industry’s leading AI-calorie counting apps to dismal results. Calorie Mama, for example, was only right about 63 per cent of the time.
Right now, the results from SnapCalorie can be logged in either the app’s native ‘food journal’ or exported to popular fitness trackers like Apple Health. At its current price point, users will probably want and expect that the app’s AI-generated calorie estimations will only get better and better with time.
Today, SnapCalorie’s premium model costs either $29.00-per-month, or nearly $200 cheaper annually at $149.00-per-year. The company’s recent $2 million cash infusion follows a previously raised angel investment of $125,000 given by unidentified funders in what TechCrunch described as a pre-seed round.
But, it remains an open question whether any automated, photo-logging and calorie-counting app is more effective than the physical and emotional investment of health journaling. A study published last year by Yale marketing professor Gal Zauberman and others found that people found the physical practice of manually logging what they ate, in practice, is more effective and more satisfying.
‘This is a classic case of what behavioural researchers call ‘misprediction,” Zauberman said of their study, published in the Journal of the Association for Consumer Research.
‘People are drawn to the apparent ease of taking pictures of their food,’ he said. ‘But we find, in fact, that they follow through less when they use the photo tool instead of the text tool.’
