How to evaluate a food recognition API.

Build a test set from your users' meals

Collect 50 to 100 meals that look like what your users eat, with their consent. Aim for a spread rather than easy cases:

• Mixed dishes such as rice dishes, stews, salads and sandwiches, where ingredients overlap.
• Home cooking, where recipes vary from one family to another.
• Regional dishes from the markets you serve.
• Drinks, snacks and small portions, which are easy to miss.
• Some difficult photos: low light, a plate partly out of frame, two plates in one picture.

If your app supports text or voice logging, write or record a description of each meal too, in the languages your users speak.

Get reference values you trust

An estimate can only be scored against a known answer. For each meal in the set, weigh the ingredients before serving and calculate calories and macronutrients from a national food-composition database. This is slow, so a smaller set with real reference values beats a large set with guesses.

Record the reference dish name, each ingredient with its weight, and the totals. Keep the set private to your team, so no vendor can tune for it.

Measure each step separately

A calorie total can look right because two errors cancel out. Score each step on its own so you know where a service is weak:

• Identification: did it name the dish, or its main ingredients, correctly? Have one reviewer score this without knowing which vendor produced the result.
• Portion: the difference between estimated and actual weight, in grams and as a percentage.
• Calories and macronutrients: the average absolute difference from the reference, and the share of meals within a margin your product can accept.
• Missing items: how often an oil, sauce, drink or side was left out.

Decide the acceptable margin before testing, based on how your product uses the numbers. A general wellness diary can accept more error than a programme where a dietitian reviews carbohydrate intake.

Test every input your users will use

If people will log by voice or text, test those too, using the descriptions you recorded. Check whether the service understands the way your users name foods, including local names and, where it matters, other languages. Then check how corrections work: can the user's edit be sent back and recalculated, or does your app have to do it?

Check where the nutrient values come from

• Source: are values taken from food-composition records, or generated by a model? Records can be traced and checked.
• Regional coverage: does the food data include the countries and dishes your users eat?
• Breadth: which nutrients are returned beyond calories and macronutrients, such as sugars, salt, fibre, vitamins and minerals?
• Mixed dishes: does the service break a dish into ingredients, or match it to a single database entry?

Check the integration, not only the results

• Response time for a typical photo, and how the service behaves when it is slow.
• Response shape: are units included, are amounts numbers or strings, and how are errors reported?
• Failure cases: a non-food photo, a blurry photo, an empty description.
• Documentation and sandbox access for your developers.

Ask the commercial and compliance questions early

Ask every vendor the same questions in writing: pricing and how it scales, usage and rate limits, uptime commitments, hosting region, encryption, data retention, whether they offer a data processing agreement, and whether customer data trains their models.

For BiteRight, the API is in private preview and sandbox access is by request. Data is encrypted in transit and at rest, hosting can be regional and a DPA is available. Pricing and usage limits have not been published yet, so ask the team for current terms.

Compare fairly and decide

1. Run the same test set, in the same order, through every service.
2. Score identification without knowing which vendor produced each result.
3. Compare portion, calorie and macronutrient error against the margin you set.
4. Weigh results against integration effort, data handling and cost.
5. Re-run a smaller check before launch, because services change.