nda improvements

.claude/settings.local.json

@@ -63,7 +63,10 @@
       "Bash(bash -n acceptance6b.sh)",
       "Bash(curl -s -m 3 http://127.0.0.1:8099/__stats)",
       "Bash(node *)",
-      "Bash(bash /tmp/claude-1000/-home-aknuth-git-bizmatch-app/0d38fdce-4b18-4159-86e7-135dadf9386d/scratchpad/acceptance6c.sh)"
+      "Bash(bash /tmp/claude-1000/-home-aknuth-git-bizmatch-app/0d38fdce-4b18-4159-86e7-135dadf9386d/scratchpad/acceptance6c.sh)",
+      "Bash(bash /tmp/claude-1000/-home-aknuth-git-bizmatch-app/0d38fdce-4b18-4159-86e7-135dadf9386d/scratchpad/acceptance6d.sh)",
+      "Bash(bash /tmp/claude-1000/-home-aknuth-git-bizmatch-app/0d38fdce-4b18-4159-86e7-135dadf9386d/scratchpad/acceptance6e.sh)",
+      "Bash(awk '{print \"  size:\", $5, \"bytes  modified:\", $6, $7, $8}')"
     ]
   }
 }

.env.example

@@ -12,6 +12,8 @@ NDA_REMINDER_DAYS=14
 # Dropbox Sign: the NDA inbox routes answer 503 until this is set.
 # Keep the real key in .env only — .env is gitignored and the key is never logged.
 DROPBOX_SIGN_API_KEY=
+# Pending signature requests older than this are no longer polled by the sync
+SYNC_PENDING_MAX_AGE_DAYS=60
 # Where signed NDAs are filed (one directory per first letter of the last name)
 NDA_ROOT=/mnt/bizmatch-nas/AA Buyers NDA's/Buyers NDA's A-Z
 

README.md

@@ -199,7 +199,7 @@ directory aborts the scan with an error naming the path.
 | GET    | /api/nda-inbox              | mirrored signature requests, `?since=<iso date>` | yes     |
 | POST   | /api/nda-inbox/refresh      | start the background walk (202 / 409 if running) | yes     |
 | POST   | /api/nda-inbox/import       | import one request into buyer/contact/nda(/deal) | yes     |
-| POST   | /api/nda-inbox/sync         | refresh the status of every pending imported NDA | yes     |
+| POST   | /api/nda-inbox/sync         | background: re-check pending NDAs + stale mirror rows | yes |
 | POST   | /api/nda-inbox/backfill-fields | retrofit form fields onto signed rounds       | yes     |
 | GET    | /api/ndas/:id/file          | stream the filed NDA PDF (Range + ETag)          | yes     |
 
@@ -318,15 +318,42 @@ are mirrored into `ds_request` by a background task, and
   (status, signer, `imported`, `known_buyer` by exact normalised e-mail, up to
   five `business_suggestions` by word overlap with the title remainder) plus
   `last_refresh_at` and `refresh_state`.
-* `POST /api/nda-inbox/refresh?since=` starts the walk and returns `202`
+* `POST /api/nda-inbox/refresh` starts the walk and returns `202` immediately,
-  immediately, or `409` when one is already running — the slot is claimed with
+  or `409` when one is already running — the slot is claimed with a conditional
-  a conditional upsert on `app_meta`, so two clicks cannot start two walks. A
+  upsert on `app_meta`, so two clicks cannot start two walks. A `running` state
-  `running` state left behind by a killed process is reset at startup.
+  left behind by a killed process is reset at startup.
+* **`?since=` means "re-read that whole window"**, so the UI only sends it when
+  the user actually moved the date picker, once; a plain Refresh posts without
+  it and catches up incrementally. Sending it on every click was what made the
+  refresh feel slow in production — the button then said Refresh but did a full
+  reload every time. The button reflects this: *Refresh* vs *Reload window*.
 
-The task walks the pages newest-first with a 500 ms pause between calls and
+The task **searches server-side** rather than paging through everything and
-upserts every NDA request into `ds_request`; re-walking is how a row that was
+discarding most of it. It sends
-pending last time is picked up as signed or declined. It stops at the first
+
-page whose oldest entry predates the window. On `429`/`409` it honours
+```
+query=title:"Buyer Forms - NDA" AND created:{<cutoff-date> TO *}
+```
+
+which turns a 13k-request account into the ~360 that are ours. The two
+client-side filters are kept as safety nets: a title that should not have come
+back is filtered out *and logged as a warning*, and the exact-timestamp check
+catches rows from the cutoff day itself, since the API's date clause is only
+day-granular. If the query were ever ignored, the mirror would still be correct
+— just slow again, and the log would say so.
+
+It pages through that filtered set with a 500 ms pause between calls; re-walking
+is how a row that was pending last time is picked up as signed or declined.
+
+Measured against the live account:
+
+| | pages | seen | stored | duration |
+| --- | --- | --- | --- | --- |
+| full 90-day reload | 4 | 359 | 359 | 33 s |
+| incremental, seconds later | 1 | 43 | 43 | 5 s |
+
+`seen` and `stored` now match. The old gap (≈1000 seen, ≈360 stored, 10 pages,
+a minute) *was* the problem: 97% of what it fetched was thrown away. On `429`/`409` it honours
 `Retry-After` but waits at least 10s, retries a page up to three times, and
 logs the response body once per run at warn level — we still do not know what
 Dropbox means by the `409` it sometimes sends. `app_meta` holds
@@ -462,10 +489,33 @@ status change itself always stands. The same holds for the import: a PDF that
 cannot be downloaded or written returns a `warning` instead of rolling back an
 import that already succeeded.
 
-`POST /api/nda-inbox/sync` re-checks every NDA that still has status `SENT`, a
+`POST /api/nda-inbox/sync` runs in the background exactly like the refresh —
-`dropbox_sign_id` and `declined = false`; it applies the signed rule and files
+`202` when started, `409` when one is already running, state and result in
-the PDF for the ones that came in, flags the ones that were declined, and
+`app_meta` under `ds_sync_state` / `ds_last_sync_at` / `ds_last_sync_result`,
-answers with `{checked, signed, declined, failed, warnings}`.
+and the UI polls until it reports the counts. It does two things:
+
+1. Re-checks every NDA that still has status `SENT`, a `dropbox_sign_id` and
+   `declined = false`, applying the signed rule and filing the PDF for the ones
+   that came in, and flagging the ones that were declined.
+2. Re-fetches mirror rows the incremental walk can no longer reach — but only
+   those worth asking about, which is what keeps the run finite:
+
+| Bound | Why |
+| --- | --- |
+| `created_at` older than the refresh reaches | anything newer was just re-read by the walk |
+| `fetched_at` older than 12 h | without it every run re-fetches the same few hundred rows |
+| `created_at` within `SYNC_PENDING_MAX_AGE_DAYS` (env, default 60) | a request pending that long is realistically dead |
+
+A request past the age cutoff is *not* deleted or hidden: it stays in the inbox
+as pending and can still be imported by hand. We simply stop asking Dropbox
+about it. The answer carries both halves apart —
+`{checked, signed, declined, failed, mirror_candidates, mirror_rechecked,
+mirror_changed, mirror_failed, warnings, warnings_omitted}` — because a mirror
+row that cannot be re-read is a stale cache entry, not an NDA that failed.
+
+Both background jobs share `src/background-task.ts`: the same atomic slot
+claim, the same `idle` / `running` / `error:<msg>` state, the same
+stale-state reset at startup.
 
 `DROPBOX_SIGN_BASE_URL` exists so the whole flow can be exercised against a
 local stub; leave it unset in production.
@@ -575,6 +625,7 @@ src/
   business-scan.ts    NAS scan, recursive listing, safe file path resolution
   dropbox-sign.ts     thin Dropbox Sign REST client (list, get, download)
   nda-files.ts        naming, filing and active/inactive moves of NDA PDFs
+  background-task.ts  slot claim + idle/running/error state for the two jobs
   nda-refresh.ts      the background walk that mirrors requests into ds_request
   nda-fields.ts       response_data -> buyer/contact/nda, fill-only-what-is-empty
   server.ts           Fastify app (health, staff, login, businesses, file, static)

docker-compose.yml

@@ -25,6 +25,7 @@ services:
       NAS_ROOT: ${NAS_ROOT:-/mnt/bizmatch-nas}
       NDA_REMINDER_DAYS: ${NDA_REMINDER_DAYS:-14}
       DROPBOX_SIGN_API_KEY: ${DROPBOX_SIGN_API_KEY:-}
+      SYNC_PENDING_MAX_AGE_DAYS: ${SYNC_PENDING_MAX_AGE_DAYS:-60}
       NDA_ROOT: ${NDA_ROOT:-/mnt/bizmatch-nas/AA Buyers NDA's/Buyers NDA's A-Z}
       NAS_DIR_ACTIVE: ${NAS_DIR_ACTIVE:-AAA = ACTIVE}
       NAS_DIR_SOLD: ${NAS_DIR_SOLD:-AAA = SOLD}

src/background-task.ts

@@ -0,0 +1,117 @@
+import type { FastifyBaseLogger } from 'fastify';
+import { query, queryOne } from './db.js';
+
+/**
+ * Long-running jobs that must not sit in front of an HTTP request: the NDA
+ * refresh walk and the signature sync. Both talk to a rate-limited API for
+ * minutes at a time, so the route starts them and returns, and the UI polls
+ * the state out of app_meta.
+ *
+ * At most one run of each at a time, claimed atomically so two clicks on the
+ * same button cannot start two jobs.
+ */
+
+export interface TaskKeys {
+  /** idle | running | error:<message> */
+  state: string;
+  /** ISO timestamp of the last completed run. */
+  lastAt: string;
+  /** JSON result of the last completed run. */
+  lastResult: string;
+}
+
+export async function getMeta(key: string): Promise<string | null> {
+  const row = await queryOne<{ value: string | null }>(
+    'SELECT value FROM app_meta WHERE key = $1',
+    [key],
+  );
+  return row?.value ?? null;
+}
+
+export async function setMeta(key: string, value: string): Promise<void> {
+  await query(
+    `INSERT INTO app_meta (key, value) VALUES ($1, $2)
+     ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`,
+    [key, value],
+  );
+}
+
+/**
+ * The conditional upsert *is* the lock: it returns nothing when somebody else
+ * already holds the slot.
+ */
+async function claim(keys: TaskKeys): Promise<boolean> {
+  const claimed = await queryOne<{ key: string }>(
+    `INSERT INTO app_meta (key, value) VALUES ($1, 'running')
+     ON CONFLICT (key) DO UPDATE SET value = 'running', updated_at = now()
+       WHERE app_meta.value <> 'running'
+     RETURNING key`,
+    [keys.state],
+  );
+  return Boolean(claimed);
+}
+
+/**
+ * Tasks live in this process, so a "running" found at startup can only be the
+ * remains of a killed one.
+ */
+export async function clearStaleTask(
+  keys: TaskKeys,
+  label: string,
+  log: FastifyBaseLogger,
+): Promise<void> {
+  const stale = await queryOne<{ key: string }>(
+    `UPDATE app_meta SET value = 'idle', updated_at = now()
+      WHERE key = $1 AND value = 'running' RETURNING key`,
+    [keys.state],
+  );
+  if (stale) log.warn(`[${label}] found a stale "running" state at startup, reset to idle`);
+}
+
+/**
+ * Claims the slot and runs `job` detached. `false` means somebody else is
+ * already running it — the caller answers 409.
+ */
+export async function startTask<T>(
+  keys: TaskKeys,
+  label: string,
+  log: FastifyBaseLogger,
+  job: () => Promise<T>,
+): Promise<boolean> {
+  if (!(await claim(keys))) return false;
+
+  void (async () => {
+    try {
+      const result = await job();
+      await setMeta(keys.lastResult, JSON.stringify(result));
+      await setMeta(keys.lastAt, new Date().toISOString());
+      await setMeta(keys.state, 'idle');
+    } catch (err) {
+      const message = (err as Error).message;
+      log.error(`[${label}] failed: ${message}`);
+      // Kept in the state so the UI can show why, rather than a silent idle.
+      await setMeta(keys.state, `error:${message}`.slice(0, 400)).catch(() => {});
+    }
+  })();
+
+  return true;
+}
+
+export interface TaskState {
+  state: string;
+  last_at: string | null;
+  last_result: unknown;
+}
+
+export async function readTaskState(keys: TaskKeys): Promise<TaskState> {
+  const [state, lastAt, lastResult] = await Promise.all([
+    getMeta(keys.state),
+    getMeta(keys.lastAt),
+    getMeta(keys.lastResult),
+  ]);
+  return {
+    state: state ?? 'idle',
+    last_at: lastAt,
+    last_result: lastResult ? (JSON.parse(lastResult) as unknown) : null,
+  };
+}

src/config.ts

@@ -16,6 +16,12 @@ export const config = {
   dropboxSignApiKey: process.env.DROPBOX_SIGN_API_KEY ?? '',
   /** Overridable so the inbox can be exercised against a local stub in tests. */
   dropboxSignBaseUrl: process.env.DROPBOX_SIGN_BASE_URL ?? 'https://api.hellosign.com',
+  /**
+   * A request pending longer than this is realistically dead: the sync stops
+   * polling Dropbox for it. It stays visible in the inbox and can still be
+   * imported by hand.
+   */
+  syncPendingMaxAgeDays: Number(process.env.SYNC_PENDING_MAX_AGE_DAYS ?? 60),
   /** Where signed NDAs are filed, one directory per first letter of the last name */
   ndaRoot: process.env.NDA_ROOT ?? "/mnt/bizmatch-nas/AA Buyers NDA's/Buyers NDA's A-Z",
   /** Directory names directly below NAS_ROOT, one per business status */

src/dropbox-sign.ts

@@ -95,29 +95,51 @@ async function getJson<T>(path: string): Promise<T> {
 }
 
 export interface SignatureRequestPage {
-  /** Everything on the page, unfiltered — the caller needs the oldest entry. */
+  /** Everything on the page as returned. */
   requests: SignatureRequest[];
-  /** Only the ones this app cares about. */
+  /** Only the ones this app cares about — the safety net over the query. */
   ndaRequests: SignatureRequest[];
   page: number;
   numPages: number;
+  numResults: number;
+}
+
+/**
+ * The search the refresh runs against the list endpoint. Filtering server-side
+ * is the difference between reading 13k requests and reading ~330: without it
+ * the walk pages through every signature request the account ever had and
+ * throws away the 97% that are not ours.
+ *
+ * `created:{<date> TO *}` is the API's range syntax; the date is a plain
+ * calendar day, which is as precise as the filter goes.
+ */
+export function ndaSearchQuery(since: Date): string {
+  const day = since.toISOString().slice(0, 10);
+  return `title:"${NDA_TITLE_PREFIX}" AND created:{${day} TO *}`;
 }
 
 /**
  * One page of the list endpoint, newest first. Paging is driven by the caller
  * (see nda-refresh.ts) so it can pace the calls and handle throttling.
  */
-export async function listPage(page: number, pageSize = 100): Promise<SignatureRequestPage> {
+export async function listPage(
+  page: number,
+  pageSize = 100,
+  query?: string,
+): Promise<SignatureRequestPage> {
+  const params = new URLSearchParams({ page: String(page), page_size: String(pageSize) });
+  if (query) params.set('query', query);
   const body = await getJson<{
     signature_requests?: SignatureRequest[];
-    list_info?: { num_pages?: number; page?: number };
+    list_info?: { num_pages?: number; page?: number; num_results?: number };
-  }>(`/v3/signature_request/list?page=${page}&page_size=${pageSize}`);
+  }>(`/v3/signature_request/list?${params.toString()}`);
   const requests = body.signature_requests ?? [];
   return {
     requests,
     ndaRequests: requests.filter((request) => request.title?.startsWith(NDA_TITLE_PREFIX)),
     page: body.list_info?.page ?? page,
     numPages: body.list_info?.num_pages ?? 1,
+    numResults: body.list_info?.num_results ?? requests.length,
   };
 }
 

src/nda-inbox-routes.ts

@@ -14,11 +14,20 @@ import {
 } from './dropbox-sign.js';
 import {
   META_LAST_REFRESH,
-  META_STATE,
+  OVERLAP_MS,
+  REFRESH_KEYS,
   clearStaleRefreshState,
-  getMeta,
   startRefresh,
+  upsertRequests,
 } from './nda-refresh.js';
+import {
+  type TaskKeys,
+  clearStaleTask,
+  getMeta,
+  readTaskState,
+  startTask,
+} from './background-task.js';
+import { config } from './config.js';
 import { NdaFileError, resolveNdaFile, writeNdaFile } from './nda-files.js';
 import { applyFormData } from './nda-fields.js';
 import { UUID_RE, badId, sendFile, trimmed } from './http.js';
@@ -26,9 +35,21 @@ import { UUID_RE, badId, sendFile, trimmed } from './http.js';
 /** How far back the inbox looks when the caller does not say. */
 const DEFAULT_WINDOW_DAYS = 90;
 
-/** Between per-request fetches in the retrofit, same courtesy as the refresh. */
+/** Between per-request fetches in the sync and retrofit, as in the refresh. */
 const FETCH_PAUSE_MS = 500;
 
+/**
+ * A mirror row read this recently is not read again. Without it every sync
+ * would re-fetch the same few hundred pending requests.
+ */
+const MIRROR_RECHECK_AFTER_HOURS = 12;
+
+export const SYNC_KEYS: TaskKeys = {
+  state: 'ds_sync_state',
+  lastAt: 'ds_last_sync_at',
+  lastResult: 'ds_last_sync_result',
+};
+
 /** Words too common in a business name to say anything about a match. */
 const STOPWORDS = new Set([
   'the',
@@ -181,6 +202,136 @@ async function fileSignedPdf(
   }
 }
 
+/**
+ * Catches up with whatever happened on the Dropbox side: signatures that came
+ * in, and requests that were declined. Runs as a background task.
+ */
+async function syncSignatures(
+  app: FastifyInstance,
+  staffId: string,
+): Promise<Record<string, unknown>> {
+  // Declined rounds are excluded: they stay SENT forever but will never
+  // change state on the Dropbox side again, so re-fetching them is waste.
+  const pending = await query<{ id: string; dropbox_sign_id: string; buyer_id: string }>(
+    `SELECT id, dropbox_sign_id, buyer_id FROM nda
+      WHERE dropbox_sign_id IS NOT NULL AND status = 'SENT' AND NOT declined
+      ORDER BY sent_at`,
+  );
+
+  let signed = 0;
+  let declined = 0;
+  let failed = 0;
+  const warnings: string[] = [];
+
+  for (const nda of pending) {
+    try {
+      const request = await getSignatureRequest(nda.dropbox_sign_id);
+      const status = statusOf(request);
+      if (status === 'signed') {
+        const signer = signerOf(request);
+        await withTransaction(async (tx) => {
+          await tx.query(`UPDATE nda SET status = 'SIGNED', signed_at = $2 WHERE id = $1`, [
+            nda.id,
+            signer.signed_at,
+          ]);
+          await applySignedRule(tx, nda.id);
+          // The signature brought the form answers with it.
+          await applyFormData(tx, {
+            ndaId: nda.id,
+            buyerId: nda.buyer_id,
+            contactId: await primaryContactId(tx, nda.buyer_id),
+            signerName: signer.name,
+            signerEmail: signer.email,
+            responseData: request.response_data,
+            staffId,
+          });
+        });
+        const filed = await fileSignedPdf(app, nda.id, request);
+        if (filed.warning) warnings.push(filed.warning);
+        signed += 1;
+      } else if (status === 'declined') {
+        await withTransaction(async (tx) => {
+          const flipped = await tx.queryRow<{ buyer_id: string; sent_at: string | null }>(
+            'UPDATE nda SET declined = true WHERE id = $1 RETURNING buyer_id, sent_at',
+            [nda.id],
+          );
+          await noteDeclined(
+            tx,
+            flipped.buyer_id,
+            flipped.sent_at ?? dayOf(request.created_at),
+            staffId,
+          );
+        });
+        declined += 1;
+      }
+    } catch (err) {
+      app.log.warn(`[nda-sync] sync of ${nda.id} failed: ${(err as Error).message}`);
+      warnings.push(`${nda.dropbox_sign_id}: ${(err as Error).message}`);
+      failed += 1;
+    }
+  }
+
+  // An incremental refresh only re-reads the newest pages, so a request that
+  // has been pending for weeks would never be looked at again. Those are
+  // re-fetched here one by one, bounded three ways so the run stays finite:
+  // older than the refresh reaches, not re-read in the last few hours, and
+  // not so old that nobody is waiting for it any more.
+  const lastRefresh = await getMeta(META_LAST_REFRESH);
+  const staleBefore = new Date(
+    (lastRefresh ? new Date(lastRefresh).getTime() : Date.now()) - OVERLAP_MS,
+  );
+  const stale = await query<{ signature_request_id: string }>(
+    `SELECT signature_request_id FROM ds_request
+      WHERE status = 'pending'
+        AND created_at < $1
+        AND created_at >= now() - ($2::int * interval '1 day')
+        AND fetched_at < now() - ($3::int * interval '1 hour')
+      ORDER BY created_at DESC`,
+    [staleBefore.toISOString(), config.syncPendingMaxAgeDays, MIRROR_RECHECK_AFTER_HOURS],
+  );
+
+  let mirrored = 0;
+  let mirrorChanged = 0;
+  let mirrorFailed = 0;
+  for (const [index, row] of stale.entries()) {
+    if (index > 0) await new Promise((resolve) => setTimeout(resolve, FETCH_PAUSE_MS));
+    try {
+      const request = await getSignatureRequest(row.signature_request_id);
+      await upsertRequests([request]);
+      mirrored += 1;
+      if (statusOf(request) !== 'pending') mirrorChanged += 1;
+    } catch (err) {
+      const message = (err as Error).message;
+      app.log.warn(`[nda-sync] re-checking ${row.signature_request_id} failed: ${message}`);
+      warnings.push(`${row.signature_request_id}: ${message}`);
+      // Counted apart from `failed`: a mirror row that cannot be re-read is
+      // a stale cache entry, not an imported NDA that failed to sync.
+      mirrorFailed += 1;
+    }
+  }
+
+  app.log.info(
+    `[nda-sync] ${pending.length} NDA(s) checked, ${signed} signed, ${declined} declined; ` +
+      `${mirrored}/${stale.length} mirror row(s) re-read, ${mirrorChanged} moved on`,
+  );
+
+  // A run over a few hundred stale rows would otherwise answer with a few
+  // hundred warning strings; the count says how many were left out.
+  const MAX_WARNINGS = 10;
+  return {
+    checked: pending.length,
+    signed,
+    declined,
+    failed,
+    mirror_candidates: stale.length,
+    mirror_rechecked: mirrored,
+    mirror_changed: mirrorChanged,
+    mirror_failed: mirrorFailed,
+    warnings: warnings.slice(0, MAX_WARNINGS),
+    warnings_omitted: Math.max(0, warnings.length - MAX_WARNINGS),
+  };
+}
+
 /**
  * The NDA inbox: what Dropbox Sign has, what of it is already in the database,
  * and the one-click import that turns a signature request into buyer +
@@ -189,7 +340,10 @@ async function fileSignedPdf(
 export function registerNdaInboxRoutes(app: FastifyInstance): void {
   // The task lives in this process, so a "running" left in app_meta can only
   // be the remains of a kill.
-  app.addHook('onReady', () => clearStaleRefreshState(app.log));
+  app.addHook('onReady', async () => {
+    await clearStaleRefreshState(app.log);
+    await clearStaleTask(SYNC_KEYS, 'nda-sync', app.log);
+  });
 
   /**
    * Reads ds_request only — never Dropbox. Mounting the view is one indexed
@@ -256,7 +410,11 @@ export function registerNdaInboxRoutes(app: FastifyInstance): void {
         ORDER BY c.is_primary DESC, c.created_at`,
       [emails],
     );
-    const businesses = await query<BusinessRow>('SELECT id, name, status FROM business');
+    // Only businesses that are actually on the market can be suggested; a
+    // sold or inactive one is never the right deal for a fresh NDA.
+    const businesses = await query<BusinessRow>(
+      `SELECT id, name, status FROM business WHERE status = 'ACTIVE'`,
+    );
 
     const requests = rows.map((row) => {
       const remainder = titleRemainder(row.title ?? '', row.signer_name ?? '');
@@ -283,10 +441,19 @@ export function registerNdaInboxRoutes(app: FastifyInstance): void {
       };
     });
 
+    const [refresh, sync] = await Promise.all([
+      readTaskState(REFRESH_KEYS),
+      readTaskState(SYNC_KEYS),
+    ]);
     return {
       requests,
-      last_refresh_at: await getMeta(META_LAST_REFRESH),
+      last_refresh_at: refresh.last_at,
-      refresh_state: (await getMeta(META_STATE)) ?? 'idle',
+      refresh_state: refresh.state,
+      /** {pages, seen, stored} of the last completed walk, null before the first. */
+      last_refresh_result: refresh.last_result,
+      last_sync_at: sync.last_at,
+      sync_state: sync.state,
+      last_sync_result: sync.last_result,
     };
   });
 
@@ -305,7 +472,9 @@ export function registerNdaInboxRoutes(app: FastifyInstance): void {
       return reply.code(400).send({ error: `invalid since: ${sinceParam}` });
     }
 
-    if (!(await startRefresh(app.log, since))) {
+    // An explicit ?since= is a deliberate request for a wider re-read and
+    // therefore skips the incremental cutoff.
+    if (!(await startRefresh(app.log, since, Boolean(sinceParam)))) {
       return reply.code(409).send({ error: 'a refresh is already running', state: 'running' });
     }
     return reply.code(202).send({ state: 'running', since: since.toISOString() });
@@ -446,68 +615,13 @@ export function registerNdaInboxRoutes(app: FastifyInstance): void {
     const staffId = staffIdFromRequest(req);
     if (!staffId) return reply.code(401).send({ error: 'not signed in' });
 
-    // Declined rounds are excluded: they stay SENT forever but will never
+    // Same shape as the refresh: it fetches per id at 500ms over what can be
-    // change state on the Dropbox side again, so re-fetching them is waste.
+    // hundreds of rows, which is minutes of work and has no business sitting
-    const pending = await query<{ id: string; dropbox_sign_id: string; buyer_id: string }>(
+    // in front of an HTTP request.
-      `SELECT id, dropbox_sign_id, buyer_id FROM nda
+    if (!(await startTask(SYNC_KEYS, 'nda-sync', app.log, () => syncSignatures(app, staffId)))) {
-        WHERE dropbox_sign_id IS NOT NULL AND status = 'SENT' AND NOT declined
+      return reply.code(409).send({ error: 'a sync is already running', state: 'running' });
-        ORDER BY sent_at`,
-    );
-
-    let signed = 0;
-    let declined = 0;
-    let failed = 0;
-    const warnings: string[] = [];
-
-    for (const nda of pending) {
-      try {
-        const request = await getSignatureRequest(nda.dropbox_sign_id);
-        const status = statusOf(request);
-        if (status === 'signed') {
-          const signer = signerOf(request);
-          await withTransaction(async (tx) => {
-            await tx.query(`UPDATE nda SET status = 'SIGNED', signed_at = $2 WHERE id = $1`, [
-              nda.id,
-              signer.signed_at,
-            ]);
-            await applySignedRule(tx, nda.id);
-            // The signature brought the form answers with it.
-            await applyFormData(tx, {
-              ndaId: nda.id,
-              buyerId: nda.buyer_id,
-              contactId: await primaryContactId(tx, nda.buyer_id),
-              signerName: signer.name,
-              signerEmail: signer.email,
-              responseData: request.response_data,
-              staffId,
-            });
-          });
-          const filed = await fileSignedPdf(app, nda.id, request);
-          if (filed.warning) warnings.push(filed.warning);
-          signed += 1;
-        } else if (status === 'declined') {
-          await withTransaction(async (tx) => {
-            const flipped = await tx.queryRow<{ buyer_id: string; sent_at: string | null }>(
-              'UPDATE nda SET declined = true WHERE id = $1 RETURNING buyer_id, sent_at',
-              [nda.id],
-            );
-            await noteDeclined(
-              tx,
-              flipped.buyer_id,
-              flipped.sent_at ?? dayOf(request.created_at),
-              staffId,
-            );
-          });
-          declined += 1;
-        }
-      } catch (err) {
-        app.log.warn(`[nda-inbox] sync of ${nda.id} failed: ${(err as Error).message}`);
-        warnings.push(`${nda.dropbox_sign_id}: ${(err as Error).message}`);
-        failed += 1;
-      }
     }
-
+    return reply.code(202).send({ state: 'running' });
-    return { checked: pending.length, signed, declined, failed, warnings };
   });
 
   /**

src/nda-refresh.ts

@@ -1,6 +1,14 @@
 import type { FastifyBaseLogger } from 'fastify';
-import { query, queryOne, withTransaction } from './db.js';
+import { withTransaction } from './db.js';
-import { type SignatureRequest, isRateLimited, listPage, statusOf } from './dropbox-sign.js';
+import {
+  NDA_TITLE_PREFIX,
+  type SignatureRequest,
+  isRateLimited,
+  listPage,
+  ndaSearchQuery,
+  statusOf,
+} from './dropbox-sign.js';
+import { type TaskKeys, clearStaleTask, getMeta, startTask } from './background-task.js';
 
 /**
  * Mirrors the Dropbox Sign signature requests into ds_request in the
@@ -20,59 +28,29 @@ const MAX_PAGE_ATTEMPTS = 3;
 /** Safety net against a window that never reaches its cutoff. */
 const MAX_PAGES = 50;
 
-export const META_LAST_REFRESH = 'ds_last_refresh_at';
+export const REFRESH_KEYS: TaskKeys = {
-export const META_STATE = 'ds_refresh_state';
+  state: 'ds_refresh_state',
+  lastAt: 'ds_last_refresh_at',
+  lastResult: 'ds_last_refresh_result',
+};
+export const META_LAST_REFRESH = REFRESH_KEYS.lastAt;
+
+/**
+ * How far back a follow-up walk reaches beyond the previous refresh. Generous
+ * on purpose: a request created just before the last run, or a clock skew
+ * between us and Dropbox, must not fall through the gap.
+ */
+export const OVERLAP_MS = 48 * 60 * 60 * 1000;
 
 export type RefreshState = 'idle' | 'running' | `error:${string}`;
 
 const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
 
-export async function getMeta(key: string): Promise<string | null> {
-  const row = await queryOne<{ value: string | null }>(
-    'SELECT value FROM app_meta WHERE key = $1',
-    [key],
-  );
-  return row?.value ?? null;
-}
-
-export async function setMeta(key: string, value: string): Promise<void> {
-  await query(
-    `INSERT INTO app_meta (key, value) VALUES ($1, $2)
-     ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`,
-    [key, value],
-  );
-}
-
 /**
- * Claims the running slot atomically — two clicks on Refresh must not start
+ * Mirrors a batch of requests. Exported because the sync route re-checks old
- * two walks. The conditional upsert is the lock; it returns nothing when
+ * pending rows one by one, which an incremental walk no longer reaches.
- * somebody else already holds it.
  */
-async function claimRefreshSlot(): Promise<boolean> {
+export async function upsertRequests(requests: SignatureRequest[]): Promise<void> {
-  const claimed = await queryOne<{ key: string }>(
-    `INSERT INTO app_meta (key, value) VALUES ($1, 'running')
-     ON CONFLICT (key) DO UPDATE SET value = 'running', updated_at = now()
-       WHERE app_meta.value <> 'running'
-     RETURNING key`,
-    [META_STATE],
-  );
-  return Boolean(claimed);
-}
-
-/**
- * The task is in-process, so a state of "running" at startup can only be the
- * remains of a killed process.
- */
-export async function clearStaleRefreshState(log: FastifyBaseLogger): Promise<void> {
-  const stale = await queryOne<{ key: string }>(
-    `UPDATE app_meta SET value = 'idle', updated_at = now()
-      WHERE key = $1 AND value = 'running' RETURNING key`,
-    [META_STATE],
-  );
-  if (stale) log.warn('[nda-refresh] found a stale "running" state at startup, reset to idle');
-}
-
-async function upsertPage(requests: SignatureRequest[]): Promise<void> {
   if (requests.length === 0) return;
   // One transaction per page: a page either lands whole or not at all.
   await withTransaction(async (tx) => {
@@ -116,12 +94,13 @@ export interface RefreshResult {
 /** Fetches one page, backing off and retrying while Dropbox throttles us. */
 async function fetchPageWithBackoff(
   page: number,
+  query: string,
   log: FastifyBaseLogger,
   loggedBody: { done: boolean },
 ) {
   for (let attempt = 1; ; attempt += 1) {
     try {
-      return await listPage(page, 100);
+      return await listPage(page, 100, query);
     } catch (err) {
       if (!isRateLimited(err) || attempt >= MAX_PAGE_ATTEMPTS) throw err;
       // Log the body once per run: we still do not know what Dropbox means by
@@ -143,56 +122,95 @@ async function fetchPageWithBackoff(
 }
 
 /**
- * Walks the list newest-first and upserts every NDA request created on or
+ * How far back this walk has to reach.
- * after `since`, stopping at the first page whose oldest entry is older than
+ *
- * that. Throws only for the caller to record — the route has long returned.
+ * A full window is only needed the first time. Afterwards everything older
+ * than the previous refresh (minus the overlap) is already mirrored, so the
+ * walk stops after a page or two instead of re-reading a thousand requests
+ * every run. An explicitly requested `since` always wins — that is how the
+ * caller asks for a deliberate re-read of a wider period.
  */
-async function walk(since: Date, log: FastifyBaseLogger): Promise<RefreshResult> {
+export async function refreshCutoff(since: Date, explicitSince: boolean): Promise<Date> {
-  const cutoff = Math.floor(since.getTime() / 1000);
+  if (explicitSince) return since;
+  const last = await getMeta(META_LAST_REFRESH);
+  if (!last) return since; // first run: the whole window
+  const incremental = new Date(new Date(last).getTime() - OVERLAP_MS);
+  if (Number.isNaN(incremental.getTime())) return since;
+  return incremental > since ? incremental : since;
+}
+
+/**
+ * Walks the NDA requests created since `cutoffAt` and mirrors them.
+ *
+ * The filtering happens server-side via the search query, so the pages this
+ * walks are already ours: `seen` and `stored` should come out roughly equal.
+ * The two client-side filters below are safety nets — if the query were ever
+ * ignored or changed, they keep the mirror correct and say so in the log.
+ */
+async function walk(cutoffAt: Date, log: FastifyBaseLogger): Promise<RefreshResult> {
+  const cutoff = Math.floor(cutoffAt.getTime() / 1000);
+  const query = ndaSearchQuery(cutoffAt);
   const loggedBody = { done: false };
   let stored = 0;
   let seen = 0;
+  let foreign = 0;
   let page = 1;
 
   for (; page <= MAX_PAGES; page += 1) {
     if (page > 1) await sleep(PAGE_PAUSE_MS);
-    const result = await fetchPageWithBackoff(page, log, loggedBody);
+    const result = await fetchPageWithBackoff(page, query, log, loggedBody);
     if (result.requests.length === 0) break;
     seen += result.requests.length;
 
+    // Safety net 1: a title the search should have excluded.
+    const wrongTitle = result.requests.length - result.ndaRequests.length;
+    if (wrongTitle > 0) {
+      foreign += wrongTitle;
+      if (foreign === wrongTitle) {
+        const example = result.requests.find(
+          (request) => !request.title?.startsWith(NDA_TITLE_PREFIX),
+        );
+        log.warn(
+          `[nda-refresh] the search returned ${wrongTitle} request(s) with a foreign title on ` +
+            `page ${page} — filtering them out here; e.g. ${JSON.stringify(example?.title ?? '')}`,
+        );
+      }
+    }
+
+    // Safety net 2: the date clause is day-granular, so a row from the cutoff
+    // day itself can come back slightly too old.
     const inWindow = result.ndaRequests.filter((request) => request.created_at >= cutoff);
-    await upsertPage(inWindow);
+    await upsertRequests(inWindow);
     stored += inWindow.length;
 
-    // Newest first, so the last entry is the oldest one on this page.
-    const oldest = result.requests[result.requests.length - 1]?.created_at ?? 0;
-    if (oldest < cutoff) break;
     if (result.page >= result.numPages) break;
   }
 
-  log.info(`[nda-refresh] walked ${page} page(s), saw ${seen}, stored ${stored} NDA request(s)`);
+  const pages = Math.min(page, MAX_PAGES);
-  return { pages: page, stored, seen };
+  log.info(
+    `[nda-refresh] walked ${pages} page(s) with query since ${cutoffAt.toISOString().slice(0, 10)}, ` +
+      `saw ${seen}, stored ${stored} NDA request(s)` +
+      (foreign > 0 ? `, discarded ${foreign} foreign title(s)` : ''),
+  );
+  return { pages, stored, seen };
 }
 
 /**
  * Starts the walk detached and returns immediately. `false` means somebody
  * else is already refreshing.
  */
-export async function startRefresh(log: FastifyBaseLogger, since: Date): Promise<boolean> {
+export async function startRefresh(
-  if (!(await claimRefreshSlot())) return false;
+  log: FastifyBaseLogger,
-
+  since: Date,
-  void (async () => {
+  explicitSince = false,
-    try {
+): Promise<boolean> {
-      await walk(since, log);
+  return startTask(REFRESH_KEYS, 'nda-refresh', log, async () => {
-      await setMeta(META_LAST_REFRESH, new Date().toISOString());
+    // Resolved inside the task: the cutoff depends on the previous refresh,
-      await setMeta(META_STATE, 'idle');
+    // and the slot claim is what serialises two runs against each other.
-    } catch (err) {
+    const cutoff = await refreshCutoff(since, explicitSince);
-      const message = (err as Error).message;
+    return walk(cutoff, log);
-      log.error(`[nda-refresh] refresh failed: ${message}`);
+  });
-      // Kept in the state so the UI can show why, rather than a silent idle.
-      await setMeta(META_STATE, `error:${message}`.slice(0, 400)).catch(() => {});
-    }
-  })();
-
-  return true;
 }
+
+export const clearStaleRefreshState = (log: FastifyBaseLogger) =>
+  clearStaleTask(REFRESH_KEYS, 'nda-refresh', log);

web/src/App.tsx

@@ -1,5 +1,6 @@
 import { useCallback, useEffect, useState } from 'react';
 import { ApiError, api, type Staff } from './api.js';
+import { StaffName } from './staff-color.js';
 import Login from './views/Login.js';
 import Today from './views/Today.js';
 import NdaInbox from './views/NdaInbox.js';
@@ -92,7 +93,7 @@ export default function App() {
           </nav>
         </div>
         <div className="flex items-center gap-3 text-sm">
-          <span className="text-gray-600">{staff.name}</span>
+          <StaffName id={staff.id} name={staff.name} />
           <button
             className="rounded border border-gray-300 px-2 py-1 text-xs hover:bg-gray-100"
             onClick={signOut}

web/src/api.ts

@@ -283,11 +283,23 @@ export interface InboxRow {
 
 export type RefreshState = 'idle' | 'running' | string;
 
+export interface RefreshResult {
+  pages: number;
+  seen: number;
+  stored: number;
+}
+
 export interface Inbox {
   requests: InboxRow[];
   /** ISO timestamp of the last completed background refresh, null if never. */
   last_refresh_at: string | null;
   refresh_state: RefreshState;
+  /** What the last completed walk did, null before the first one. */
+  last_refresh_result: RefreshResult | null;
+  /** The sync runs in the background too, with the same state pattern. */
+  last_sync_at: string | null;
+  sync_state: RefreshState;
+  last_sync_result: SyncResult | null;
 }
 
 export interface ImportResult {
@@ -314,7 +326,13 @@ export interface SyncResult {
   signed: number;
   declined: number;
   failed: number;
+  /** Old pending mirror rows re-fetched one by one, and how many had moved on. */
+  mirror_candidates: number;
+  mirror_rechecked: number;
+  mirror_changed: number;
+  mirror_failed: number;
   warnings: string[];
+  warnings_omitted: number;
 }
 
 export class ApiError extends Error {
@@ -422,8 +440,12 @@ export const api = {
 
   /** Reads the mirrored requests out of the database — never calls Dropbox. */
   ndaInbox: (since: string) => request<Inbox>(`/api/nda-inbox?${qs({ since })}`),
-  /** Starts the background walk; resolves as soon as it is queued. */
+  /**
-  refreshInbox: (since: string) =>
+   * Starts the background walk; resolves as soon as it is queued. Passing
+   * `since` makes the server re-read that whole window — leave it out for the
+   * incremental catch-up, which is what a plain Refresh should do.
+   */
+  refreshInbox: (since?: string) =>
     post<{ state: RefreshState }>(`/api/nda-inbox/refresh?${qs({ since })}`),
   importInbox: (signatureRequestId: string, buyerId?: string, businessId?: string) =>
     post<ImportResult>('/api/nda-inbox/import', {
@@ -431,7 +453,7 @@ export const api = {
       buyer_id: buyerId,
       business_id: businessId,
     }),
-  syncInbox: () => post<SyncResult>('/api/nda-inbox/sync'),
+  syncInbox: () => post<{ state: RefreshState }>('/api/nda-inbox/sync'),
   backfillFields: () => post<BackfillResult>('/api/nda-inbox/backfill-fields'),
 };
 

web/src/staff-color.tsx

@@ -0,0 +1,71 @@
+/**
+ * A stable colour per staff member, so "who did this" is readable at a glance
+ * without reading the name. Derived from the id, which means it survives
+ * reloads, sessions and renames, and needs nothing stored anywhere.
+ */
+
+/**
+ * Ten hues that stay apart from each other and from the greys the app uses.
+ * Written out in full because Tailwind only ships classes it can see in the
+ * source — a generated `text-${hue}-700` would be purged.
+ */
+export interface StaffColor {
+  text: string;
+  dot: string;
+}
+
+const PALETTE: StaffColor[] = [
+  { text: 'text-blue-700', dot: 'bg-blue-500' },
+  { text: 'text-emerald-700', dot: 'bg-emerald-500' },
+  { text: 'text-violet-700', dot: 'bg-violet-500' },
+  { text: 'text-amber-700', dot: 'bg-amber-500' },
+  { text: 'text-rose-700', dot: 'bg-rose-500' },
+  { text: 'text-cyan-700', dot: 'bg-cyan-500' },
+  { text: 'text-lime-700', dot: 'bg-lime-600' },
+  { text: 'text-fuchsia-700', dot: 'bg-fuchsia-500' },
+  { text: 'text-orange-700', dot: 'bg-orange-500' },
+  { text: 'text-teal-700', dot: 'bg-teal-600' },
+];
+
+/** FNV-1a — short, stable, and well spread over ten buckets. */
+function hash(value: string): number {
+  let h = 0x811c9dc5;
+  for (let i = 0; i < value.length; i += 1) {
+    h ^= value.charCodeAt(i);
+    h = Math.imul(h, 0x01000193);
+  }
+  return Math.abs(h);
+}
+
+/** Grey stands in for "nobody" — a deleted staff member or an unassigned row. */
+const UNKNOWN: StaffColor = { text: 'text-gray-500', dot: 'bg-gray-400' };
+
+export function staffColor(id: string | null | undefined): StaffColor {
+  if (!id) return UNKNOWN;
+  return PALETTE[hash(id) % PALETTE.length] ?? UNKNOWN;
+}
+
+/**
+ * A staff member's name in their colour. Deliberately understated — a dot and
+ * tinted text, never a filled pill, so a list of names does not turn into a
+ * traffic light.
+ */
+export function StaffName({
+  id,
+  name,
+  dot = true,
+  className = '',
+}: {
+  id: string | null | undefined;
+  name: string | null | undefined;
+  dot?: boolean;
+  className?: string;
+}) {
+  const color = staffColor(id);
+  return (
+    <span className={`inline-flex items-center gap-1 ${color.text} ${className}`}>
+      {dot && <span className={`h-1.5 w-1.5 shrink-0 rounded-full ${color.dot}`} />}
+      {name ?? 'unknown'}
+    </span>
+  );
+}

web/src/views/NdaInbox.tsx

@@ -1,5 +1,5 @@
 import { useCallback, useEffect, useRef, useState } from 'react';
-import { api, type Inbox, type InboxRow, type InboxStatus, type SyncResult } from '../api.js';
+import { api, type Inbox, type InboxRow, type InboxStatus } from '../api.js';
 import { formatDayTimeParts, formatRelative } from '../components.js';
 
 const INPUT = 'rounded border border-gray-300 px-2 py-1.5 text-sm';
@@ -37,22 +37,34 @@ export default function NdaInbox({
   onChanged: () => void;
 }) {
   const [since, setSince] = useState(() => isoDaysAgo(90));
+  // Sending ?since= tells the server to re-read the whole window, which is a
+  // minute of paging. A plain Refresh must not do that — only a date the user
+  // actually moved since the last run asks for the wide reload, once.
+  const [sinceChanged, setSinceChanged] = useState(false);
   const [inbox, setInbox] = useState<Inbox | null>(null);
   const [error, setError] = useState<string | null>(null);
   const [loading, setLoading] = useState(false);
-  const [syncing, setSyncing] = useState(false);
-  const [syncResult, setSyncResult] = useState<SyncResult | null>(null);
   // Business chosen per row; '' means "import the NDA without a deal".
   const [picked, setPicked] = useState<Record<string, string>>({});
-  const [busy, setBusy] = useState<string | null>(null);
+  // Which row is mid-import, if any.
+  const [importing, setImporting] = useState<string | null>(null);
   const [notice, setNotice] = useState<string | null>(null);
   const poll = useRef<number | null>(null);
+  // Set when a task started *here* finishes, so its result is reported once
+  // rather than on every mount.
+  const [refreshNotice, setRefreshNotice] = useState<string | null>(null);
+  const [syncNotice, setSyncNotice] = useState<string | null>(null);
+  const watchingRefresh = useRef(false);
+  const watchingSync = useRef(false);
 
   const rows: InboxRow[] | null = inbox?.requests ?? null;
   const refreshing = inbox?.refresh_state === 'running';
-  const refreshError = inbox?.refresh_state?.startsWith('error:')
+  const syncing = inbox?.sync_state === 'running';
-    ? inbox.refresh_state.slice('error:'.length)
+  // While either task runs the table keeps polling; both report when done.
-    : null;
+  const busy = refreshing || syncing;
+  const taskError = [inbox?.refresh_state, inbox?.sync_state]
+    .find((state) => state?.startsWith('error:'))
+    ?.slice('error:'.length);
 
   /** Reads the mirror out of the database — this never talks to Dropbox. */
   const load = useCallback(async () => {
@@ -61,16 +73,9 @@ export default function NdaInbox({
       const res = await api.ndaInbox(since);
       setInbox(res);
       setError(null);
-      // Preselect the best suggestion, but never overwrite a manual choice.
+      // Deliberately no preselection: a suggestion is a guess, and an import
-      setPicked((current) => {
+      // must never attach a deal the user did not pick. Anything already
-        const next = { ...current };
+      // chosen by hand is kept across reloads.
-        for (const row of res.requests) {
-          if (next[row.signature_request_id] === undefined) {
-            next[row.signature_request_id] = row.business_suggestions[0]?.id ?? '';
-          }
-        }
-        return next;
-      });
       return res;
     } catch (err) {
       setError((err as Error).message);
@@ -87,7 +92,31 @@ export default function NdaInbox({
   // Whenever a walk is in flight — started here or elsewhere — watch it until
   // it finishes, then show whatever it brought in.
   useEffect(() => {
-    if (!refreshing) {
+    if (!refreshing && watchingRefresh.current && inbox?.last_refresh_result) {
+      watchingRefresh.current = false;
+      const { pages, seen, stored } = inbox.last_refresh_result;
+      setRefreshNotice(
+        `Refreshed: ${stored} request${stored === 1 ? '' : 's'} stored ` +
+          `from ${seen} seen over ${pages} page${pages === 1 ? '' : 's'}.`,
+      );
+    }
+    if (!syncing && watchingSync.current && inbox?.last_sync_result) {
+      watchingSync.current = false;
+      const r = inbox.last_sync_result;
+      setSyncNotice(
+        `Synced: ${r.checked} NDA(s) checked, ${r.signed} newly signed, ` +
+          `${r.declined} declined` +
+          (r.failed > 0 ? `, ${r.failed} failed` : '') +
+          ` · ${r.mirror_rechecked}/${r.mirror_candidates} older request(s) re-read, ` +
+          `${r.mirror_changed} had moved on` +
+          (r.mirror_failed > 0 ? `, ${r.mirror_failed} unreadable` : '') +
+          (r.warnings.length > 0
+            ? ` · ${r.warnings.join('; ')}${r.warnings_omitted > 0 ? ` (+${r.warnings_omitted} more)` : ''}`
+            : ''),
+      );
+    }
+
+    if (!busy) {
       if (poll.current !== null) {
         window.clearInterval(poll.current);
         poll.current = null;
@@ -102,12 +131,16 @@ export default function NdaInbox({
         poll.current = null;
       }
     };
-  }, [refreshing, load]);
+  }, [busy, refreshing, syncing, load, inbox?.last_refresh_result, inbox?.last_sync_result]);
 
   async function refresh() {
     setError(null);
+    setRefreshNotice(null);
     try {
-      await api.refreshInbox(since);
+      // Only a moved date asks for the full window; then back to incremental.
+      await api.refreshInbox(sinceChanged ? since : undefined);
+      setSinceChanged(false);
+      watchingRefresh.current = true;
       // Flip to "running" at once so the poller starts without a round trip.
       setInbox((current) => (current ? { ...current, refresh_state: 'running' } : current));
     } catch (err) {
@@ -116,22 +149,20 @@ export default function NdaInbox({
   }
 
   async function sync() {
-    setSyncing(true);
-    setSyncResult(null);
     setError(null);
+    setSyncNotice(null);
     try {
-      setSyncResult(await api.syncInbox());
+      await api.syncInbox();
-      await load();
+      watchingSync.current = true;
-      onChanged();
+      // Flip to "running" at once so the poller starts without a round trip.
+      setInbox((current) => (current ? { ...current, sync_state: 'running' } : current));
     } catch (err) {
       setError((err as Error).message);
-    } finally {
-      setSyncing(false);
     }
   }
 
   async function importRow(row: InboxRow) {
-    setBusy(row.signature_request_id);
+    setImporting(row.signature_request_id);
     setNotice(null);
     setError(null);
     try {
@@ -151,7 +182,7 @@ export default function NdaInbox({
     } catch (err) {
       setError((err as Error).message);
     } finally {
-      setBusy(null);
+      setImporting(null);
     }
   }
 
@@ -164,7 +195,11 @@ export default function NdaInbox({
             <input
               type="date"
               value={since}
-              onChange={(e) => setSince(e.target.value)}
+              onChange={(e) => {
+                setSince(e.target.value);
+                setSinceChanged(true);
+              }}
+              title="Changing the date makes the next Refresh re-read that whole window"
               className={INPUT}
             />
           </label>
@@ -177,10 +212,14 @@ export default function NdaInbox({
           <button
             onClick={refresh}
             disabled={refreshing}
-            title="Fetches the signature requests from Dropbox Sign in the background"
+            title={
+              sinceChanged
+                ? 'Reload the whole window from the changed date'
+                : 'Refresh (new since last check)'
+            }
             className="rounded border border-gray-300 bg-white px-3 py-1.5 text-sm hover:bg-gray-100 disabled:opacity-50"
           >
-            {refreshing ? 'Refreshing…' : 'Refresh'}
+            {refreshing ? 'Refreshing…' : sinceChanged ? 'Reload window' : 'Refresh'}
           </button>
           <button
             onClick={sync}
@@ -192,24 +231,16 @@ export default function NdaInbox({
         </div>
       </div>
 
-      {refreshing && (
+      {busy && (
         <p className="mb-3 text-sm text-gray-600">
-          Fetching from Dropbox Sign in the background — the table below stays usable.
+          {refreshing ? 'Fetching from' : 'Re-checking with'} Dropbox Sign in the background — the
+          table below stays usable.
         </p>
       )}
-      {refreshError && (
+      {taskError && <p className="mb-3 text-sm text-red-600">Last run failed: {taskError}</p>}
-        <p className="mb-3 text-sm text-red-600">Last refresh failed: {refreshError}</p>
-      )}
 
-      {syncResult && (
+      {refreshNotice && <p className="mb-3 text-sm text-gray-600">{refreshNotice}</p>}
-        <p className="mb-3 text-sm text-gray-600">
+      {syncNotice && <p className="mb-3 text-sm text-gray-600">{syncNotice}</p>}
-          Checked {syncResult.checked} · newly signed {syncResult.signed} · declined{' '}
-          {syncResult.declined} · failed {syncResult.failed}
-          {syncResult.warnings.length > 0 && (
-            <span className="text-amber-700"> · {syncResult.warnings.join('; ')}</span>
-          )}
-        </p>
-      )}
       {notice && <p className="mb-3 text-sm text-gray-600">{notice}</p>}
       {error && <p className="mb-3 text-sm text-red-600">{error}</p>}
 
@@ -289,10 +320,10 @@ export default function NdaInbox({
                 ) : (
                   <button
                     onClick={() => importRow(row)}
-                    disabled={busy === row.signature_request_id}
+                    disabled={importing === row.signature_request_id}
                     className="rounded bg-gray-900 px-2 py-1 text-xs text-white disabled:opacity-50"
                   >
-                    {busy === row.signature_request_id ? 'Importing…' : 'Import'}
+                    {importing === row.signature_request_id ? 'Importing…' : 'Import'}
                   </button>
                 )}
               </td>

web/src/views/Today.tsx

@@ -9,17 +9,30 @@ import {
 } from '../api.js';
 import { DEAL_LABELS, Dialog, Panel, StatusBadge, formatDay } from '../components.js';
 import { TodoRow } from '../workflow.js';
+import { StaffName } from '../staff-color.js';
 
 const INPUT = 'w-full rounded border border-gray-300 px-2 py-1 text-sm';
 
-/** Groups anything with a person attached, for the team tab. */
+interface Group<T> {
-function groupBy<T>(rows: T[], name: (row: T) => string): [string, T[]][] {
+  id: string | null;
-  const groups = new Map<string, T[]>();
+  name: string;
+  rows: T[];
+}
+
+/**
+ * Groups anything with a person attached, for the team tab. Keyed by staff id
+ * rather than name, so the group heading can carry that person's colour.
+ */
+function groupByStaff<T>(rows: T[], who: (row: T) => { id: string | null; name: string }): Group<T>[] {
+  const groups = new Map<string, Group<T>>();
   for (const row of rows) {
-    const key = name(row);
+    const { id, name } = who(row);
-    groups.set(key, [...(groups.get(key) ?? []), row]);
+    const key = id ?? name;
+    const group = groups.get(key) ?? { id, name, rows: [] };
+    group.rows.push(row);
+    groups.set(key, group);
   }
-  return [...groups.entries()].sort((a, b) => a[0].localeCompare(b[0]));
+  return [...groups.values()].sort((a, b) => a.name.localeCompare(b.name));
 }
 
 export default function Today({
@@ -110,10 +123,16 @@ export default function Today({
       {data && data.todos.length > 0 && (
         <Panel title="Todos">
           {team ? (
-            groupBy(data.todos, (todo) => todo.assigned_to.name).map(([name, todos]) => (
+            groupByStaff(data.todos, (todo) => todo.assigned_to).map((group) => (
-              <div key={name} className="mb-2 last:mb-0">
+              <div key={group.id ?? group.name} className="mb-2 last:mb-0">
-                <p className="text-xs font-medium uppercase tracking-wide text-gray-500">{name}</p>
+                <p className="text-xs font-medium uppercase tracking-wide">
-                <TodoList todos={todos} onChanged={refresh} open={{ onOpenBuyer, onOpenBusiness }} />
+                  <StaffName id={group.id} name={group.name} />
+                </p>
+                <TodoList
+                  todos={group.rows}
+                  onChanged={refresh}
+                  open={{ onOpenBuyer, onOpenBusiness }}
+                />
               </div>
             ))
           ) : (
@@ -125,11 +144,16 @@ export default function Today({
       {data && data.follow_ups.length > 0 && (
         <Panel title="Follow-ups due">
           {team ? (
-            groupBy(data.follow_ups, (row) => row.created_by_name ?? 'unassigned').map(
+            groupByStaff(data.follow_ups, (row) => ({
-              ([name, rows]) => (
+              id: row.created_by_id,
-                <div key={name} className="mb-2 last:mb-0">
+              name: row.created_by_name ?? 'unassigned',
-                  <p className="text-xs font-medium uppercase tracking-wide text-gray-500">{name}</p>
+            })).map(
-                  {rows.map((row) => (
+              (group) => (
+                <div key={group.id ?? group.name} className="mb-2 last:mb-0">
+                  <p className="text-xs font-medium uppercase tracking-wide">
+                    <StaffName id={group.id} name={group.name} />
+                  </p>
+                  {group.rows.map((row) => (
                     <FollowUpRow
                       key={row.id}
                       row={row}
@@ -278,7 +302,11 @@ function PendingNdaRow({
         </button>
         , sent <span className="text-gray-500">{formatDay(row.sent_at)}</span>
       </span>
-      {team && <span className="text-xs text-gray-500">{row.created_by_name ?? '—'}</span>}
+      {team && (
+        <span className="text-xs">
+          <StaffName id={row.created_by_id} name={row.created_by_name ?? '—'} />
+        </span>
+      )}
     </div>
   );
 }

web/src/workflow.tsx

@@ -9,6 +9,7 @@ import {
   type TodoKind,
 } from './api.js';
 import { Dialog, formatDay, formatStamp } from './components.js';
+import { StaffName } from './staff-color.js';
 
 const INPUT = 'rounded border border-gray-300 px-2 py-1 text-sm';
 
@@ -80,8 +81,9 @@ function NoteRow({
                 {note.context.label}
               </span>
             )}
-            <span>
+            <span className="inline-flex items-center gap-1">
-              {note.author?.name ?? 'unknown'} · {formatStamp(note.created_at)}
+              <StaffName id={note.author?.id} name={note.author?.name} />·{' '}
+              {formatStamp(note.created_at)}
             </span>
             <button
               onClick={() => run(() => api.updateNote(note.id, { highlight: !note.highlight }))}
@@ -281,7 +283,7 @@ export function TodoRow({
           )}
         </div>
         <div className="flex flex-wrap items-center gap-2 text-xs text-gray-400">
-          <span>{todo.assigned_to.name}</span>
+          <StaffName id={todo.assigned_to.id} name={todo.assigned_to.name} />
           {todo.due_at && (
             <span className={todo.overdue && !done ? 'font-medium text-red-600' : ''}>
               due {formatDay(todo.due_at)}