Files
bizmatch-app/README.md
2026-07-27 17:48:40 -05:00

497 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# BizMatch Phase 2 — Broker Workflow App
Module 1: foundation (Docker Compose, PostgreSQL, schema, migrations, login).
Module 2: business scan (NAS -> DB) and the first UI.
Module 3: recursive file listing, PDF streaming from the NAS and the ported
pdf.js viewer.
Module 4: the buyer side — buyers, contacts, NDA rounds, deals, the guided
"New inquiry" flow with duplicate detection and the deal status transitions.
Module 5: notes, todos and the "Today" view with the follow-up workflow.
Module 6a: Dropbox Sign as the source of incoming NDAs — inbox, one-click
import and PDF filing on the NAS.
The UI and all domain constants are English.
## Setup on 192.168.100.99 (Ubuntu 24.04)
Prerequisite: Docker + Compose plugin (`sudo apt install docker.io docker-compose-v2`).
```bash
# Unpack the project, then:
cd bizmatch-app
cp .env.example .env # optionally adjust DB_PASSWORD
docker compose up -d --build
```
The app applies all migrations on start and then listens on
`http://192.168.100.99:8090`.
> **Upgrading from Module 1:** `001_init.sql` was rewritten in place (German
> constants -> English). Applying it needs a fresh database:
> `docker compose down -v && docker compose up -d --build`. The DB held no
> production data yet, so there is nothing to migrate.
### Schema notes
`001_init.sql` holds the full base schema (Buyer / Contact / NDA / Deal /
Business / Note / Todo / Document / ExtractionJob / Staff) and is never edited
again. `002_buyer_fields.sql` adds the fields the buyer side actually collects
and is purely additive, so it applies to an existing database:
| Table | Added |
| --------- | ------------------------------------------------------------------------------------ |
| `contact` | `cell` |
| `buyer` | `address`, `state`, `background_experience`, `how_heard`, `interested_in_updates` |
| `nda` | `total_purchase_price`, `down_payment`, `intro_date` |
`003_interested_in_updates_nullable.sql` then drops the `NOT NULL` and the
default from `buyer.interested_in_updates`: the value comes off scanned intake
sheets where the field is frequently blank, so `NULL` means "not answered" and
has to stay distinct from `false` ("explicitly no"). The buyer detail shows it
as a Yes / No / not answered control, and `PATCH /api/buyers/:id` accepts all
three. The two price fields stay `text` on purpose: the paper forms contain entries like "1.2M + inventory" that no
numeric type survives. The migration also adds the three index expressions the
duplicate check needs (`lower(btrim(name))` and the digits-only phone/cell).
First smoke test:
```bash
curl http://localhost:8090/api/health
# -> {"ok":true}
# Create the three staff members (adjust the names):
curl -X POST localhost:8090/api/staff -H 'content-type: application/json' -d '{"name":"Chris"}'
curl -X POST localhost:8090/api/staff -H 'content-type: application/json' -d '{"name":"..."}'
```
## Dev mode (without the app container)
```bash
docker compose up -d db # database only
npm install
set -a; source .env; set +a
npm run dev # tsx watch, migrations run on start
# second terminal — frontend with hot reload, /api is proxied to :8090
cd web && npm install && npm run dev
```
## Test data
The acceptance scripts run against the **real** development database, so rows
they create are visible in the UI like any other. Two rules keep that from
turning into litter:
1. **Tag it.** Every contact an acceptance run creates uses an e-mail under
`@stubtest.invalid` (a reserved TLD that can never be a real address), and
every stubbed Dropbox Sign request uses an id starting with `req-`. Those
two markers are what makes test rows identifiable later, when nobody
remembers which "Priya Raman" came from where.
2. **Remove it.** Every script ends with a cleanup step wired to `trap … EXIT`,
so it runs even when the script fails halfway:
```bash
cleanup_stub_data() {
docker compose exec -T db psql -U bizmatch bizmatch -q \
-c "DELETE FROM buyer WHERE id IN (
SELECT buyer_id FROM contact
WHERE lower(btrim(email)) LIKE '%@stubtest.invalid');" \
-c "DELETE FROM nda WHERE dropbox_sign_id LIKE 'req-%';" \
-c "DELETE FROM ds_request WHERE signature_request_id LIKE 'req-%';"
}
trap cleanup_stub_data EXIT
```
Deleting the buyer cascades to its contacts, NDA rounds, deals, notes and
todos, so the one statement is enough.
**Do not truncate `ds_request`.** It was a throwaway cache only while no API
key was configured; now it mirrors the real signature requests, and a stub run
adds its rows *alongside* them. Delete the `req-*` ones and leave
`ds_last_refresh_at` alone — a truncate would throw away the real mirror and
silently misreport when it was last refreshed.
Anything that writes files (NDA filing) must point `NDA_ROOT` at a scratch
directory for the run; never let a test write into the real NAS tree.
## NAS mount
Mount it on the host via NFS, e.g. in `/etc/fstab`:
```
<truenas-ip>:/mnt/<pool>/bizmatch /mnt/bizmatch-nas nfs ro,soft,timeo=100 0 0
```
The compose file already passes `NAS_ROOT` (default `/mnt/bizmatch-nas`) into the
app container.
**Module 6a needs write access**: filing signed NDAs writes below `NDA_ROOT`
(default `<NAS_ROOT>/AA Buyers NDA's/Buyers NDA's A-Z`), so the mount has to be
`rw` rather than `ro`. Everything else the app does with the NAS is read-only;
if the mount stays read-only, imports still succeed and only report the failed
filing as a warning.
### Business directories
Directly below `NAS_ROOT` there are three status directories; every immediate
subdirectory of those is one business. The directory names are configurable
(they contain spaces and are treated as opaque strings):
| Env var | Default | Business status |
| ------------------ | -------------- | --------------- |
| `NAS_DIR_ACTIVE` | `AAA = ACTIVE` | ACTIVE |
| `NAS_DIR_SOLD` | `AAA = SOLD` | SOLD |
| `NAS_DIR_INACTIVE` | `AAA = INACTIVE` | INACTIVE |
The scan is idempotent: existing rows are matched by name and only updated when
`nas_path` or `status` changed. Businesses that exist in the DB but no longer on
disk are kept and only reported as a warning in the log. A missing configured
directory aborts the scan with an error naming the path.
## Moving to the AI machine (later)
1. `docker compose down` on .99
2. Take a dump: `docker compose exec db pg_dump -U bizmatch bizmatch > backup.sql`
3. Copy the project folder to the AI machine, run `docker compose up -d --build`
4. Restore the dump: `docker compose exec -T db psql -U bizmatch bizmatch < backup.sql`
## API (as of module 6a)
| Method | Path | Purpose | Session |
| ------ | --------------------------- | ------------------------------------------------ | ------- |
| GET | /api/health | liveness incl. DB check | no |
| GET | /api/staff | staff list | no |
| POST | /api/staff | create staff member `{name}` | no |
| POST | /api/login | login via `{staff_id}` → session cookie | no |
| GET | /api/me | signed-in staff member | yes |
| POST | /api/logout | sign out | yes |
| POST | /api/businesses/scan | scan the NAS → `{scanned, inserted, updated, missing}` | yes |
| GET | /api/businesses | list `?status=&search=` + counts per status | yes |
| GET | /api/businesses/:id | single business incl. `nas_path` | yes |
| GET | /api/businesses/:id/files | recursive listing, max depth 3 (PDFs first) | yes |
| GET | /api/businesses/:id/file | stream one file, `?path=<relative>` | yes |
| GET | /api/businesses/:id/deals | buyer activity on one business, newest first | yes |
| GET | /api/buyers | list `?search=&status=` + counts per buyer status | yes |
| GET | /api/buyers/duplicates | candidates for `?email=&name=&phone=` | yes |
| POST | /api/inquiries | guided new-inquiry flow (one transaction) | yes |
| GET | /api/buyers/:id | buyer incl. `contacts[]` and `ndas[].deals[]` | yes |
| PATCH | /api/buyers/:id | identity fields + status (+ `end_open_deals`) | yes |
| POST | /api/buyers/:id/contacts | add a contact | yes |
| PATCH | /api/contacts/:id | edit a contact (incl. `is_primary`) | yes |
| PATCH | /api/ndas/:id | edit one NDA round | yes |
| POST | /api/ndas/:id/deals | add a business to an existing round | yes |
| POST | /api/deals/:id/status | `{status, comment?}` — transition + note | yes |
| GET | /api/deals/:id/notes | notes of one deal, newest first | yes |
| POST | /api/deals/:id/follow-up-sent | `{comment?, rearm}` — note + re-arm or clear | yes |
| POST | /api/notes | create a note on exactly one reference object | yes |
| GET | /api/notes | notes of one object, `?…_id=` (+`include_related`) | yes |
| PATCH | /api/notes/:id | edit `text` / `highlight` | yes |
| DELETE | /api/notes/:id | delete a note | yes |
| POST | /api/todos | create a todo | yes |
| GET | /api/todos | `?assigned_to=&status=` + one ref id as scope | yes |
| PATCH | /api/todos/:id | text, due_at, assigned_to, kind, document_id | yes |
| POST | /api/todos/:id/done | close it (`done_by`/`done_at` = session, now) | yes |
| POST | /api/todos/:id/reopen | reopen it and clear both | yes |
| POST | /api/documents | pin a business file so a REVIEW todo can link it | yes |
| GET | /api/today | `?staff_id=` — the day's work + nav counts | yes |
| GET | /api/nda-inbox | mirrored signature requests, `?since=<iso date>` | yes |
| POST | /api/nda-inbox/refresh | start the background walk (202 / 409 if running) | yes |
| POST | /api/nda-inbox/import | import one request into buyer/contact/nda(/deal) | yes |
| POST | /api/nda-inbox/sync | refresh the status of every pending imported NDA | yes |
| GET | /api/ndas/:id/file | stream the filed NDA PDF (Range + ETag) | yes |
Everything except health, staff (GET+POST) and login requires the session
cookie; without it the API answers `401`.
### Buyer side (module 4)
Domain rules, all enforced in the API:
* **buyer** is the buying party, **contact** are its 1..n people, **nda** is one
inquiry round (a returning buyer signs a *new* NDA), **deal** is
buyer↔business inside one round. There is deliberately no uniqueness on
`(buyer_id, business_id)` — a returning buyer gets a new round with new deals
and the history stays visible.
* Deal flow `NEW → INFO_SENT → DUE_DILIGENCE → LOI → CLOSING`, `ENDED` from
anywhere. `POST /api/deals/:id/status` rejects only a no-op (`409`);
everything else is allowed on purpose, because corrections have to be
possible. Entering `INFO_SENT` sets `follow_up_at = today + 14`, entering
`ENDED` clears it.
* When an NDA becomes `SIGNED` it gets a `signed_at` (default today) and its
buyer is set back to `ACTIVE`.
* Deactivating a buyer with `{"status":"DEACTIVATED","end_open_deals":true}`
ends all their non-`ENDED` deals; the response always carries
`open_deal_count` so the UI can warn first.
`GET /api/buyers/duplicates` matches exactly, never fuzzily: normalised e-mail
(`lower(btrim(…))`), case-insensitive contact name, and phone **or** cell
compared digits-only, so `(361) 555-0101` and `3615550101` are the same number.
Numbers with fewer than 7 digits are ignored. A candidate reports every reason
it matched in `matched_on`.
`POST /api/inquiries` is the guided flow and runs in one transaction. Without
`buyer_id` it creates buyer + primary contact; with `buyer_id` it reuses the
buyer and only adds the contact when no existing contact of that buyer has the
same normalised e-mail or the same name. It then creates the NDA round and one
deal, and returns
`{buyer_id, nda_id, deal_id, created:{buyer, contact}}`. The optional
`backfill` block (`deal_status`, `nda_status`, `signed_at`, `nda_nas_path`)
files a paper record in its real state — a backfilled `INFO_SENT` still arms
the 14-day follow-up, later statuses do not.
### File listing and streaming
`/files` walks the business directory recursively (max depth 3), skipping
dotfiles, dot-directories and symlinks, and returns
`{ path, size, mtime }` with `path` relative to the business directory and
always posix-separated. PDFs come first, then everything else, each group
alphabetical.
`/file?path=…` streams one of those files straight from disk
(`createReadStream`, never buffered):
* the path is resolved against `nas_path` and then `realpath`-validated to be
inside `realpath(business dir)`. Absolute paths, `..`, leading dots, empty
paths and symlinks pointing out of the tree get `400`; a missing file `404`.
* single-range HTTP `Range` requests answer `206` with `Content-Range`,
unsatisfiable ones `416`.
* `ETag` is derived from mtime + size, `If-None-Match` answers `304`.
* `.pdf` is served as `application/pdf` (inline), anything else as
`application/octet-stream` with `Content-Disposition: attachment`.
### Notes, todos and Today (module 5)
A **note** hangs off exactly one object (buyer, deal, business or NDA round), a
**todo** off at most one — the DB enforces both, and the API mirrors it so a
wrong body gets a `400` instead of a constraint violation. Every note and todo
row carries a `context` object `{type, id, label, buyer_id, business_id}`: the
label is the business name, the buyer's company/contact name or
`Round <date>`, and the two ids let the UI link straight to the right page.
`GET /api/notes?buyer_id=…&include_related=true` additionally folds in the
notes of that buyer's rounds and deals, which is what the buyer page shows.
`GET /api/today` is the daily workqueue, narrowed to one person with
`?staff_id=` and covering the whole team without it:
* **todos** — OPEN, `due_at <= today`, each flagged `overdue` when
`due_at < today`
* **follow_ups** — deals with `follow_up_at <= today` that are not `ENDED`;
"mine" means `created_by`
* **pending_ndas** — rounds still `SENT` and not declined after
`NDA_REMINDER_DAYS` (env, default 14); "mine" means `created_by`
* **counts** — the numbers behind the nav badge
Follow-ups and pending NDAs are **virtual**: they are derived from the deal and
nda rows on every request and never materialise as todo rows, so there is
nothing to keep in sync or clean up. Answering one is
`POST /api/deals/:id/follow-up-sent`, which always writes a note and then
either re-arms the reminder for another 14 days (`rearm: true`) or clears
`follow_up_at``409` on a deal that is already ENDED. Giving up on the deal
instead is the normal `POST /api/deals/:id/status` with `ENDED`.
`POST /api/documents` is a stopgap for the REVIEW todo's file picker: it pins
one business file (validated through the same resolver the file streaming uses)
as a `document` row so `todo.document_id` can point at it. Real document
management follows in module 6.
### NDA inbox — Dropbox Sign (module 6a)
Buyers sign their NDA in Dropbox Sign, so that account is the source of new
rounds. `DROPBOX_SIGN_API_KEY` (Basic auth, key as the username, empty
password) enables the inbox routes; without it they answer `503` with a clear
message and the rest of the app is unaffected. The key lives in `.env` only —
it is never committed and never logged.
Only requests whose title starts with **`Buyer Forms - NDA`** are considered.
**The inbox is DB-backed.** Proxying the list endpoint on every view mount did
not survive contact with the real account: 700+ requests in a 90-day window
means 78 paged calls, ~74s of latency, throttling (Dropbox answers `409` as
well as `429`), and the whole list thrown away on a tab switch. So the requests
are mirrored into `ds_request` by a background task, and
* `GET /api/nda-inbox` reads **only** that table — one indexed query, instant,
unaffected by tab switches. It returns the same per-request shape as before
(status, signer, `imported`, `known_buyer` by exact normalised e-mail, up to
five `business_suggestions` by word overlap with the title remainder) plus
`last_refresh_at` and `refresh_state`.
* `POST /api/nda-inbox/refresh?since=` starts the walk and returns `202`
immediately, or `409` when one is already running — the slot is claimed with
a conditional upsert on `app_meta`, so two clicks cannot start two walks. A
`running` state left behind by a killed process is reset at startup.
The task walks the pages newest-first with a 500 ms pause between calls and
upserts every NDA request into `ds_request`; re-walking is how a row that was
pending last time is picked up as signed or declined. It stops at the first
page whose oldest entry predates the window. On `429`/`409` it honours
`Retry-After` but waits at least 10s, retries a page up to three times, and
logs the response body once per run at warn level — we still do not know what
Dropbox means by the `409` it sometimes sends. `app_meta` holds
`ds_last_refresh_at` and `ds_refresh_state` (`idle` | `running` |
`error:<msg>`).
Import and sync are unchanged and still address one `signature_request_id` at a
time. Nothing re-fetches a request per id unnecessarily: sync's candidate set
is `SENT AND NOT declined`, so signed and declined rounds are never fetched
again.
`POST /api/nda-inbox/import` is one transaction: buyer + contact (the same
reuse rules as the guided inquiry — see `ensureBuyerAndContact`), the NDA round
with `dropbox_sign_id` (UNIQUE, so a second import is a `409`), and optionally
a deal. A signed request becomes `SIGNED` and runs the usual signed rule.
`custom_fields` are ignored — field extraction is a later module.
**Declined rounds** stay `SENT` with `declined = true`: they were never signed,
but they must not look merely outstanding either. They are therefore excluded
from the sync candidate set (a decline is final on the Dropbox side) *and* from
the `pending_ndas` section of the Today view (nobody is waiting for that
signature). So that the event does not vanish silently, flipping to declined —
whether on import or during a sync — writes a highlighted note on the buyer:
`NDA declined via Dropbox Sign, <date of the round>`.
Signed PDFs are filed under `NDA_ROOT`:
```
NDA_ROOT/<letter>/active/<Last>, <First> <YYYY-MM-DD> NDA.pdf buyer is active
NDA_ROOT/<letter>/<Last>, <First> <YYYY-MM-DD> NDA.pdf buyer is not
```
`<letter>` is the first letter of the last name, uppercased; anything outside
AZ is filed under `_`. Collisions get ` (2)`, ` (3)` … before the extension.
Setting a buyer to `DEACTIVATED` moves their filed NDAs out of `active/` and
`ACTIVE` moves them back; each move is wrapped individually, and a failure only
adds `{nda_id, error}` to the `warnings` array of the PATCH response — the
status change itself always stands. The same holds for the import: a PDF that
cannot be downloaded or written returns a `warning` instead of rolling back an
import that already succeeded.
`POST /api/nda-inbox/sync` re-checks every NDA that still has status `SENT`, a
`dropbox_sign_id` and `declined = false`; it applies the signed rule and files
the PDF for the ones that came in, flags the ones that were declined, and
answers with `{checked, signed, declined, failed, warnings}`.
`DROPBOX_SIGN_BASE_URL` exists so the whole flow can be exercised against a
local stub; leave it unset in production.
## Frontend
`web/` is a Vite + React + TypeScript app with Tailwind v4 (no router, no state
library). The header carries the nav entries **Today**, **NDA Inbox**,
**Businesses** and **Buyers**; routing is a hand-rolled `{view, id}` state in
`App.tsx`. Today is the landing view and its nav entry shows a red badge with the signed-in user's
overdue + due todos. There is no polling, so the badge is refreshed on mount,
on every view change and after any action that can move an item off the list.
Views:
* login ("Who is working?")
* today — "My day" / "Team" tabs (Team groups todos by assignee and follow-ups
by their creator) over three sections: todos (overdue in red, checkbox to
complete, context chip opens the buyer or business), follow-ups due (with
"Follow-up sent…" → comment + "wait another 14 days" / "stop waiting", and
"End deal…" → the status dialog preset to ENDED) and NDA signatures pending.
Empty state: "Nothing due. Enjoy your coffee."
* nda inbox — renders straight from the mirror on mount, never from Dropbox.
The header shows "Last refreshed: &lt;relative time&gt;"; Refresh starts the
background walk and polls every 3s until it reports idle, then reloads the
table, while the table stays usable throughout. Plus "Sync signatures", and
one row per request with a status chip, a known-buyer chip linking to the
buyer (or "new"), a business select preselected to the best suggestion
("— no deal —" to import the round on its own) and an Import button.
Imported rows show a checkmark that opens the buyer instead.
* business list (tabs with counts, search, "Scan NAS now")
* business detail — a master-detail split filling the viewport: file table
left, PDF viewer right, plus collapsed "Buyer activity", "Notes" and "Todos"
panels above it
* buyer list (status chips with counts, search over company/contact/e-mail,
"New inquiry")
* new inquiry — contact + business picker; while typing a known name, e-mail or
phone a warning panel lists the duplicate candidates with "Use this buyer"
(locks the buyer, shown as a chip with an undo) or "Create new buyer anyway".
The collapsible "Backfill existing deal (paper records)" section files
historic deals in their real state.
* buyer detail — status header with Deactivate/Reactivate (warns about the open
deals it would end), inline-editable identity panel, contacts with a primary
star, a notes panel covering the buyer *and* their rounds and deals, a todos
panel, and the NDA rounds newest first: editable round fields, the deals of
the round with an action menu (next step, "End deal", plus a "Correct to…"
section) that opens a comment dialog, and a collapsed "Notes & todos" section
per deal. A round with a filed PDF gets a "View PDF" button opening the
pdf.js viewer on `/api/ndas/:id/file`; a declined round carries a red
"Declined" badge.
Notes are written in a composer at the top of every notes panel; the red flag
button marks a note as important, and flagged notes get a red left border and a
light red background wherever they appear.
In dev, Vite proxies `/api` to `http://localhost:8090`. In production the
Fastify app serves `web/dist` via `@fastify/static` with an SPA fallback to
`index.html` for all non-`/api` routes; the Dockerfile builds the frontend in
its own stage and copies `web/dist` into the runtime image.
### PDF viewer
The viewer is the proven one from the phase-1 Deno desktop app (see
`viewer-phase1/`), ported nearly byte-identical. It lives in
`web/public/viewer/` as plain, unbundled ES modules — Vite serves `public/`
as-is, so the same files work in dev and prod. The React app embeds it in an
`<iframe>`:
```
/viewer/index.html?file=<urlencoded /api/businesses/:id/file?path=...>
```
The page refuses any `file` value that is not a root-relative `/api/` path, and
the iframe is same-origin, so the normal session cookie authenticates it.
`web/public/pdfjs/` holds the pdf.js runtime, copied out of
`node_modules/pdfjs-dist` (pinned to exactly 6.1.200) by
`web/scripts/copy-pdfjs.mjs`, which runs on `predev` and `prebuild` — also
inside the Docker web stage. The directory is generated and git-ignored:
```
web/public/pdfjs/legacy/ pdf.min.mjs + pdf.worker.min.mjs
web/public/pdfjs/wasm/ CCITT-G4/JBIG2, JPEG2000 and ICC decoders
web/public/pdfjs/standard_fonts/ standardFontDataUrl
web/public/pdfjs/iccs/ iccUrl
```
The `wasm/` directory is what makes scanned B/W pages render at all; without it
pdf.js fails the decoders silently and shows blank white canvases.
## Structure
```
migrations/ numbered SQL migrations
001_init.sql full schema
002_buyer_fields.sql buyer-side fields from the NDA form + intake sheet
003_…_nullable.sql interested_in_updates becomes tri-state
004_reset_….sql one-time reset of that column to NULL
005_nda_dropbox_….sql dropbox_sign_id, signer_name, declined on nda
006_ds_request_….sql ds_request mirror + the app_meta key/value table
src/
config.ts env configuration
db.ts pg pool, query helpers, withTransaction
session.ts the staff-id cookie
http.ts input coercion, PATCH/reference helpers, file responses
migrate.ts migration runner (transactional, advisory lock)
business-scan.ts NAS scan, recursive listing, safe file path resolution
dropbox-sign.ts thin Dropbox Sign REST client (list, get, download)
nda-files.ts naming, filing and active/inactive moves of NDA PDFs
nda-refresh.ts the background walk that mirrors requests into ds_request
server.ts Fastify app (health, staff, login, businesses, file, static)
buyer-routes.ts buyers, contacts, NDA rounds, deals, the inquiry flow
workflow-routes.ts notes, todos, documents, the Today view, follow-ups
nda-inbox-routes.ts the NDA inbox, import, signature sync, NDA PDF streaming
web/
scripts/copy-pdfjs.mjs pdfjs-dist -> public/pdfjs/ (predev + prebuild)
public/viewer/ standalone, unbundled pdf.js viewer page
public/pdfjs/ generated, git-ignored pdf.js runtime
src/api.ts typed API client
src/App.tsx session gate + nav (with the Today badge) + view switch
src/components.tsx shared bits (badges, inline fields, business picker, dialog)
src/workflow.tsx notes panel, todo list and the add-todo dialog
src/views/ Login, Today, NdaInbox, Businesses, BusinessDetail,
Buyers, BuyerDetail, NewInquiry
viewer-phase1/ reference copy of the phase-1 desktop viewer
```