Keeping Data Updated

If your application serves Apicbase data to end users (for example, displaying a recipe when a customer taps on it) you must not query the Apicbase API in real-time for each of those requests. The API is not designed for live, high-frequency access of this kind and will quickly hit rate limits.

The correct approach is to maintain a local database copy of the data your application needs, and keep it up to date by periodically polling the Apicbase API for changes.

Polling for changes with `modified_date__gt`

Most list endpoints support a modified_date__gt filter that returns only items modified after a given date. Use this to fetch only what has changed since your last sync, rather than re-fetching your entire dataset each time. On large libraries, fetching full recipe details without this filter can take hours.

The sync process follows these steps:

Query the list endpoint with modified_date__gt set to the timestamp of your last sync.
For each item returned, fetch its detail endpoint (available in the item's url field).
Update your local copy with the data from the detail endpoint.

Here's a sample script that performs a periodic sync for recipes:

import requests
import datetime

# Set up API authentication
access_token = 'your_access_token'
headers = {'Authorization': f'Bearer {access_token}'}

# Set up last sync date
last_sync_date = '2023-04-30'  # replace with your actual last sync date

# Define API endpoints
recipe_list_url = 'https://api.apicbase.com/v2/products/recipes/'

# Query the API recipe list endpoint with the modified_date__gt query param
params = {'modified_date__gt': last_sync_date}
response = requests.get(recipe_list_url, headers=headers, params=params)
recipe_list = response.json()['results']

# Query the detail endpoint of each item in the filtered list
for recipe in recipe_list:
    recipe_url = recipe['url']
    response = requests.get(recipe_url, headers=headers)
    recipe_detail = response.json()

    # Update local database copy of each recipe with the data returned by the API
    # replace the print statement with your actual database update code
    print(f'Updating recipe with ID {recipe_id}')
    print(recipe_detail) # this is just an example, replace with your actual update code

const fetch = require('node-fetch');

// Set up API authentication
const accessToken = 'your_access_token';
const headers = { 'Authorization': `Bearer ${accessToken}` };

// Set up last sync date
const lastSyncDate = '2023-04-30'; // replace with your actual last sync date

// Define API endpoints
const recipeListUrl = 'https://api.apicbase.com/v2/products/recipes/';

// Query the API recipe list endpoint with the modified_date__gt query param
const params = { 'modified_date__gt': lastSyncDate };
fetch(`${recipeListUrl}?${new URLSearchParams(params)}`, { headers })
  .then(response => response.json())
  .then(responseJson => {
    const recipeList = responseJson.results;

    // Query the detail endpoint of each item in the filtered list
    recipeList.forEach(recipe => {
      const recipeUrl = recipe.url;
      fetch(recipeUrl, { headers })
        .then(response => response.json())
        .then(recipeDetail => {
          // Update local database copy of each recipe with the data returned by the API
          // replace the console.log statement with your actual database update code
          console.log(`Updating recipe with ID ${recipeDetail.id}`);
          console.log(recipeDetail); // this is just an example, replace with your actual update code
        });
    });
  });

<?php

// Set up API authentication
$access_token = 'your_access_token';
$headers = array('Authorization' => 'Bearer ' . $access_token);

// Set up last sync date
$last_sync_date = '2023-04-30'; // replace with your actual last sync date

// Define API endpoints
$recipe_list_url = 'https://api.apicbase.com/v2/products/recipes/';

// Query the API recipe list endpoint with the modified_date__gt query param
$params = array('modified_date__gt' => $last_sync_date);
$response = Requests::get($recipe_list_url, $headers, $params);
$recipe_list = json_decode($response->body)->results;

// Query the detail endpoint of each item in the filtered list
foreach ($recipe_list as $recipe) {
    $recipe_url = $recipe->url;
    $response = Requests::get($recipe_url, $headers);
    $recipe_detail = json_decode($response->body);

    // Update local database copy of each recipe with the data returned by the API
    // replace the print statement with your actual database update code
    echo 'Updating recipe with ID ' . $recipe_detail->id . PHP_EOL;
    var_dump($recipe_detail); // this is just an example, replace with your actual update code
}

Calculated fields and the modification date

The modified_date on an object only reflects changes to its own native attributes. It does not update when a calculated field changes due to a change in a related object. For example, updating the price of an ingredient will affect the cost price of every recipe that uses it, but those recipes' modified_date values will not change.

For fields of this kind, you need to track the modification date of the source object instead. Ingredients and recipes both have a used_in_recipes field that lets you navigate this relationship upward. When an ingredient changes, you can find all affected recipes and re-fetch them. Make sure to also check the used_in_recipes field on recipes themselves, since a recipe can be used as a subrecipe inside another recipe.

❗️

This applies to allergens and nutritional data too.

Allergens and nutritional values on a recipe are calculated from its ingredients. If an ingredient changes, the recipe's own modified_date will not reflect this. To keep allergen and nutritional copies up to date, poll ingredients by modified_date__gt and use used_in_recipes to identify which recipes need to be re-fetched.

Here's a sample script that handles this recursively:

import requests

def update_recipes(last_sync):
    # Get list of ingredients changed since the last sync
    ingredients_changed = request_api('products/ingredients', {"modified_date__gt": last_sync})
 
    for ingredient in ingredients_changed:
        recipes_that_use_it = ingredient['used_in_recipes']
        
        # Update every recipe that uses this ingredient
        for recipe_url in recipes_that_use_it:
            recipe_detail = request_api(recipe_url)
            update_recipe_detail(recipe_detail)
  
    # Now, get a list of recipes themselves that changed since the last sync
    # Because allergens can also be defined on the recipe itself
    recipes_changed = request_api('products/recipes', {"modified_date__gt": last_sync})
    for recipe in recipes_changed:
        recipe_detail = request_api(recipe['url'])
        update_recipe_detail(recipe_detail)

def update_recipe_detail(recipe):
    # Update recipe copy
    my_local_recipe = get_recipe_from_db(recipe['id'])
    my_local_recipe.allergens = recipe['allergens']
    
    # Also update every recipe that uses it as a subrecipe
    recipes_that_use_it = recipe['used_in_recipes']
    for recipe_url in recipes_that_use_it:
        recipe_detail = request_api(recipe_url)
        update_recipe_detail(recipe_detail)

def request_api(endpoint, params=None):
    url = f"https://app.apicbase.com/api/v2/{endpoint}/"
    headers = {'Authorization': 'Bearer your-api-token'}
    response = requests.get(url, headers=headers, params=params)
    return response.json()

const requestApi = (endpoint, params) => {
  const url = `https://app.apicbase.com/api/v2/${endpoint}/`;
  const headers = { Authorization: 'Bearer your-api-token' };
  const response = await fetch(new URL(url), { headers, params });
  return await response.json();
};

const updateRecipeDetail = (recipe) => {
  // Update recipe copy
  const myLocalRecipe = getRecipeFromDb(recipe.id);
  myLocalRecipe.allergens = recipe.allergens;

  // Also update every recipe that uses it as a subrecipe
  const recipesThatUseIt = recipe.used_in_recipes;
  for (const recipeUrl of recipesThatUseIt) {
    const recipeDetail = requestApi(recipeUrl);
    updateRecipeDetail(recipeDetail);
  }
};

const updateRecipes = async (lastSync) => {
  // Get list of ingredients changed since the last sync
  const ingredientsChanged = await requestApi('products/ingredients', {
    modified_date__gt: lastSync,
  });

  for (const ingredient of ingredientsChanged) {
    const recipesThatUseIt = ingredient.used_in_recipes;

    // Update every recipe that uses this ingredient
    for (const recipeUrl of recipesThatUseIt) {
      const recipeDetail = await requestApi(recipeUrl);
      updateRecipeDetail(recipeDetail);
    }
  }

  // Now, get a list of recipes themselves that changed since the last sync
  // Because allergens can also be defined on the recipe itself
  const recipesChanged = await requestApi('products/recipes', {
    modified_date__gt: lastSync,
  });
  for (const recipe of recipesChanged) {
    const recipeDetail = await requestApi(recipe.url);
    updateRecipeDetail(recipeDetail);
  }
};

<?php

function update_recipes($last_sync) {
  // Get list of ingredients changed since the last sync
  $ingredients_changed = request_api('products/ingredients', array('modified_date__gt' => $last_sync));
 
  foreach ($ingredients_changed as $ingredient) {
    $recipes_that_use_it = $ingredient['used_in_recipes'];
    
    // Update every recipe that uses this ingredient
    foreach ($recipes_that_use_it as $recipe_url) {
      $recipe_detail = request_api($recipe_url);
      update_recipe_detail($recipe_detail);
    }
  }
  
  // Now, get a list of recipes themselves that changed since the last sync
  // Because allergens can also be defined on the recipe itself
  $recipes_changed = request_api('products/recipes', array('modified_date__gt' => $last_sync));
  foreach ($recipes_changed as $recipe) {
    $recipe_detail = request_api($recipe['url']);
    update_recipe_detail($recipe_detail);
  }
}

function update_recipe_detail($recipe) {
  // Update recipe copy
  $my_local_recipe = get_recipe_from_db($recipe['id']);
  $my_local_recipe['allergens'] = $recipe['allergens'];
  
  // Also update every recipe that uses it as a subrecipe
  $recipes_that_use_it = $recipe['used_in_recipes'];
  foreach ($recipes_that_use_it as $recipe_url) {
    $recipe_detail = request_api($recipe_url);
    update_recipe_detail($recipe_detail);
  }
}

function request_api($endpoint, $params = null) {
  $url = "https://app.apicbase.com/api/v2/$endpoint/";
  $headers = array('Authorization' => 'Bearer your-api-token');
  $response = requests\get($url, array('headers' => $headers, 'query' => $params));
  return json_decode($response->getBody(), true);
}

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;

namespace YourNamespace.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class RecipeController : ControllerBase
    {
        private readonly HttpClient _httpClient;

        public RecipeController(IHttpClientFactory httpClientFactory)
        {
            _httpClient = httpClientFactory.CreateClient();
            _httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer your-access-token");
        }

        [HttpGet("UpdateRecipes/{lastSyncDate}")]
        public async Task<IActionResult> UpdateRecipes(DateTime lastSyncDate)
        {
            var ingredientsChanged = await RequestApiAsync<List<object>>("products/ingredients", new Dictionary<string, string>
            {
                {"modified_date__gt", lastSyncDate.ToString("yyyy-MM-dd")}
            });

            foreach (var ingredient in ingredientsChanged)
            {
                var recipesThatUseIt = (List<string>)ingredient["used_in_recipes"];

                foreach (var recipeUrl in recipesThatUseIt)
                {
                    var recipeDetail = await RequestApiAsync<object>(recipeUrl);
                    await UpdateRecipeDetailAsync(recipeDetail);
                }
            }

            var recipesChanged = await RequestApiAsync<List<object>>("products/recipes", new Dictionary<string, string>
            {
                {"modified_date__gt", lastSyncDate.ToString("yyyy-MM-dd")}
            });

            foreach (var recipe in recipesChanged)
            {
                var recipeDetail = await RequestApiAsync<object>((string)recipe["url"]);
                await UpdateRecipeDetailAsync(recipeDetail);
            }

            return Ok();
        }

        private async Task<T> RequestApiAsync<T>(string endpoint, IDictionary<string, string> queryParams = null)
        {
            var queryString = "";
            if (queryParams != null)
            {
                foreach (var kvp in queryParams)
                {
                    queryString += $"{kvp.Key}={kvp.Value}&";
                }
            }

            var url = $"https://app.apicbase.com/api/v2/{endpoint}/?{queryString.TrimEnd('&')}";
            var response = await _httpClient.GetAsync(url);
            response.EnsureSuccessStatusCode();
            var json = await response.Content.ReadAsStringAsync();
            return Newtonsoft.Json.JsonConvert.DeserializeObject<T>(json);
        }

        private async Task UpdateRecipeDetailAsync(object recipe)
        {
            // Update recipe copy
            var myLocalRecipe = await GetRecipeFromDbAsync((string)recipe["id"]);
            myLocalRecipe.allergens = (List<string>)recipe["allergens"];

            // Also update every recipe that uses it as a subrecipe
            var recipesThatUseIt = (List<string>)recipe["used_in_recipes"];
            foreach (var recipeUrl in recipesThatUseIt)
            {
                var recipeDetail = await RequestApiAsync<object>(recipeUrl);
                await UpdateRecipeDetailAsync(recipeDetail);
            }
        }

        private async Task<object> GetRecipeFromDbAsync(string id)
        {
            // Replace with actual database call
            await Task.CompletedTask;
            return null;
        }
    }
}

Polling for changes with modified_date__gt

Calculated fields and the modification date

This applies to allergens and nutritional data too.

Polling for changes with `modified_date__gt`