Cache product lookups for 60 seconds

harbor-tools/catalog-api#88·by jun-w·+57 −6 · 3 files·TypeScript · Node 22 service

Precomputed example. This review was run once on a real pull request and stored. Nothing on this page talks to GitHub or Claude: no review runs, and nothing posts.

What the agent said

2 things to fix before merge

2 should fix1 nit
  • Invalidation deletes `id` but entries are keyed `tenant:id`: updates read stale for 60 s.
  • Entry count is bounded by FIFO eviction; worth confirming the bound is right for this service.
  • The TTL is a literal in the service module.
Full summary and the reviewer's notes

Puts a 60 s TTL cache in front of getProduct, keyed by tenant and id, with FIFO eviction at 5,000 entries. The cache class is small and correct. The wiring has one bug: updateProduct invalidates by id alone while the entries are stored under tenant:id, so the delete never matches and a product reads stale for up to a minute after it is edited, the exact case the one new test does not cover.

What this PR does. Product lookups are the hottest query in the catalog API. This PR remembers each product for a minute after it is first read so repeated lookups skip the database, and forgets a product when it is updated so the change shows immediately.

What was checked, and what was dropped. Checked: expiry is lazy on read (fine for this size); `set` evicts the oldest insertion when full, `Map` preserves insertion order so `keys().next()` is the oldest; a miss on a null row is not cached (every lookup of a missing product still hits the DB, acceptable, flagged nowhere). Dropped: a suggestion to cache nulls (a product decision, not a review finding); a note that `hit` would skip a product whose value is falsy (a `Product` is an object, never falsy).

  1. Finding 1 of 3 Should fix src/catalog/service.ts:18

    Invalidation uses the wrong key: updated products read stale for up to 60 s

    Why it mattersAn edit to a product (price, availability) is invisible to the API for up to a minute, on every instance that has the old entry cached.

    src/catalog/service.ts
    @@ -1,18 +1,33 @@
    import { db } from "../db";
    import { TtlCache } from "./cache";
    import type { Product, ProductPatch } from "./types";
    
    const cache = new TtlCache<Product>(60_000);
    const key = (tenant: string, id: string) => `${tenant}:${id}`;
    
    export async function getProduct(tenant: string, id: string): Promise<Product | null> {
      return db.products.findOne({ tenant, id });
      const hit = cache.get(key(tenant, id));
      if (hit) return hit;
      const row = await db.products.findOne({ tenant, id });
      if (row) cache.set(key(tenant, id), row);
      return row;
    }
    
    export async function updateProduct(tenant: string, id: string, patch: ProductPatch): Promise<Product> {
      const row = await db.products.update({ tenant, id }, patch);
      cache.delete(id);
      return row;
    }
    

    The comment as drafted

    Entries go in under key(tenant, id), that is "acme:p1", but updateProduct calls cache.delete(id), which looks for "p1" and deletes nothing. Use the same key builder so the update really evicts the entry.

    Suggested change: the author could apply this in one click on GitHub
    cache.delete(key(tenant, id));

    Evidence and confidence

    High confidence Lands inline on the diff

    How to verify. Read a product, update its price, read it again inside 60 s: the old price comes back. The new test passes because it never updates.

    Kept by the reviewer

    Confirmed by reading the two lines side by side: `set` goes through `key(tenant, id)`, `delete` does not. The one-line suggestion is the whole fix. Posted as written.

  2. Finding 2 of 3 Should fix src/catalog/cache.ts:20

    Unbounded growth: entries are never evicted, only expired on read

    Why it mattersA catalog with millions of products and a scan-style client (a crawler, an export job) fills the map until the process runs out of memory; expired entries are only removed if the same key is read again.

    src/catalog/cache.ts
    @@ -0,0 +1,31 @@
    const MAX_ENTRIES = 5_000;
    
    interface Entry<T> { value: T; expires: number }
    
    export class TtlCache<T> {
      private readonly map = new Map<string, Entry<T>>();
    
      constructor(private readonly ttlMs: number) {}
    
      get(key: string): T | undefined {
        const e = this.map.get(key);
        if (!e) return undefined;
        if (e.expires <= Date.now()) {
          this.map.delete(key);
          return undefined;
        }
        return e.value;
      }
    
      set(key: string, value: T): void {
        if (this.map.size >= MAX_ENTRIES) {
          const oldest = this.map.keys().next().value;
          if (oldest !== undefined) this.map.delete(oldest);
        }
        this.map.set(key, { value, expires: Date.now() + this.ttlMs });
      }
    
      delete(key: string): void {
        this.map.delete(key);
      }
    }
    

    The comment as drafted

    set() adds to the Map without any size check, and get() only removes the entry it is asked for. Under a wide key space (a crawler walking the catalog) the map grows without bound until the process is OOM-killed. Add a maximum entry count and evict when it is reached, or sweep expired entries on a timer.

    Evidence and confidence

    Medium confidence Lands inline on the diff

    How to verify. Call `set` with 1,000,000 distinct keys and watch heap usage.

    Dropped by the reviewer

    Not true: `cache.ts:20-23` evicts the oldest entry once the map reaches `MAX_ENTRIES` (5,000), so the bound exists. The reviewer saw the lazy expiry in `get` and stopped reading. Whether 5,000 is the right number is a fair question, but that is not what this finding says.

  3. Finding 3 of 3 Nit src/catalog/service.ts:5

    TTL is a literal in the service module

    Why it mattersTuning the cache lifetime, the first thing ops will want to do once it is live, needs a code change and a deploy.

    src/catalog/service.ts
    @@ -1,18 +1,33 @@
    import { db } from "../db";
    import { TtlCache } from "./cache";
    import type { Product, ProductPatch } from "./types";
    
    const cache = new TtlCache<Product>(60_000);
    const key = (tenant: string, id: string) => `${tenant}:${id}`;
    
    export async function getProduct(tenant: string, id: string): Promise<Product | null> {
      return db.products.findOne({ tenant, id });
      const hit = cache.get(key(tenant, id));
      if (hit) return hit;
      const row = await db.products.findOne({ tenant, id });
      if (row) cache.set(key(tenant, id), row);
      return row;
    }
    
    export async function updateProduct(tenant: string, id: string, patch: ProductPatch): Promise<Product> {
      const row = await db.products.update({ tenant, id }, patch);
      cache.delete(id);
      return row;
    }
    

    The comment as drafted

    Why is the TTL hardcoded? Magic numbers everywhere. 60_000 should obviously be in config like everything else.

    Evidence and confidence

    High confidence Lands inline on the diff

    How to verify. Search `src/config.ts` for the other catalog settings; the TTL is the only one not there.

    Edited by the reviewer

    A real nit, the other catalog settings live in config, but "magic numbers everywhere" is both rude and false (there is one). Said what to do and where.

    Posted instead

    Could the 60 s TTL move to `src/config.ts` next to the other catalog settings? It is the first value ops will want to tune once this is live, and today that needs a deploy.

The review that posts: 2 of 3 comments, as one comment review, under the reviewer's own name

  1. Should fixsrc/catalog/service.ts:18inline

    Entries go in under key(tenant, id), that is "acme:p1", but updateProduct calls cache.delete(id), which looks for "p1" and deletes nothing. Use the same key builder so the update really evicts the entry.

    cache.delete(key(tenant, id));
  2. Nitsrc/catalog/service.ts:5inline· edited

    Could the 60 s TTL move to src/config.ts next to the other catalog settings? It is the first value ops will want to tune once this is live, and today that needs a deploy.

Nothing was sent from this page: the review above is what the reviewer's click would post from their own ReviewStage, with their own GitHub account.

Run this on your PR

One command on your laptop. Your GitHub account, your Claude subscription, your call on every comment: nothing posts until you click.

Install guide Try it live