The documented shadow-database-free fallback for generating migration SQL (--from-migrations ... --to-schema-datamodel ...) was never actually tested and fails for the same P3014 reason `migrate dev` does, since --from-migrations also requires --shadow-database-url internally. Verified against the local dev DB that --from-schema-datasource (live introspection, no shadow DB) diffed against --to-schema-datamodel (static file read) works in both the no-op case and a real ADD COLUMN case, and documented that instead. Also corrected the inaccurate claim that this matched Task 3's approach (Task 3 used --from-empty) and moved the suggested SQL output path out of the repo root into the OS temp directory. Also brings markFailed's errorMessage/errorLog params in line with markDone's existing ?? null guard, and adds orderBy to findExpiredJobs to match findPendingJobs, for consistency. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
91 lines
8.4 KiB
Markdown
91 lines
8.4 KiB
Markdown
## Approach
|
|
- Read existing files before writing. Don't re-read unless changed.
|
|
- Thorough in reasoning, concise in output.
|
|
- Skip files over 100KB unless required.
|
|
- No sycophantic openers or closing fluff.
|
|
- No emojis or em-dashes.
|
|
- Do not guess APIs, versions, flags, commit SHAs, or package names. Verify by reading code or docs before asserting.
|
|
|
|
<!-- code-review-graph MCP tools -->
|
|
## MCP Tools: code-review-graph
|
|
|
|
**IMPORTANT: This project has a knowledge graph. ALWAYS use the
|
|
code-review-graph MCP tools BEFORE using Grep/Glob/Read to explore
|
|
the codebase.** The graph is faster, cheaper (fewer tokens), and gives
|
|
you structural context (callers, dependents, test coverage) that file
|
|
scanning cannot.
|
|
|
|
### When to use graph tools FIRST
|
|
|
|
- **Exploring code**: `semantic_search_nodes` or `query_graph` instead of Grep
|
|
- **Understanding impact**: `get_impact_radius` instead of manually tracing imports
|
|
- **Code review**: `detect_changes` + `get_review_context` instead of reading entire files
|
|
- **Finding relationships**: `query_graph` with callers_of/callees_of/imports_of/tests_for
|
|
- **Architecture questions**: `get_architecture_overview` + `list_communities`
|
|
|
|
Fall back to Grep/Glob/Read **only** when the graph doesn't cover what you need.
|
|
|
|
### Key Tools
|
|
|
|
| Tool | Use when |
|
|
| ------ | ---------- |
|
|
| `detect_changes` | Reviewing code changes — gives risk-scored analysis |
|
|
| `get_review_context` | Need source snippets for review — token-efficient |
|
|
| `get_impact_radius` | Understanding blast radius of a change |
|
|
| `get_affected_flows` | Finding which execution paths are impacted |
|
|
| `query_graph` | Tracing callers, callees, imports, tests, dependencies |
|
|
| `semantic_search_nodes` | Finding functions/classes by name or keyword |
|
|
| `get_architecture_overview` | Understanding high-level codebase structure |
|
|
| `refactor_tool` | Planning renames, finding dead code |
|
|
|
|
### Workflow
|
|
|
|
1. The graph auto-updates on file changes (via hooks).
|
|
2. Use `detect_changes` for code review.
|
|
3. Use `get_affected_flows` to understand impact.
|
|
4. Use `query_graph` pattern="tests_for" to check coverage.
|
|
|
|
## Local environment / running tests
|
|
|
|
- `.env` holds **production** credentials (o2switch host, real DB name/password). Never load it for local runs or tests.
|
|
- `.env.local` holds the local dev DB credentials (`DB_HOST=127.0.0.1`, `DB_USER=convert_user`, `DB_NAME=file_converter`, `DB_PASSWORD=change_me`). This is what local testing should use.
|
|
- `src/config.js` uses `dotenv/config`, which only loads `.env` and does not override variables already present in `process.env`. There's no vitest/dotenv wiring that picks up `.env.local` automatically.
|
|
- To run tests locally without touching prod config, pass the `.env.local` values as inline env vars so they take precedence before `dotenv/config` runs, e.g.:
|
|
```
|
|
DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run
|
|
```
|
|
- A local MariaDB is expected to already be running on `127.0.0.1:3306` with the `file_converter` DB and `convert_user` credentials seeded.
|
|
- Known pre-existing failures unrelated to any fix: `test/cleanup.test.js` ("deletes an expired pending job...") and `test/jobs/jobRepository.test.js` ("finds expired jobs and allows deleting them") — both fail on `main` independent of other changes (looks like a clock/timezone mismatch around `expiresAt` comparisons, not yet root-caused). Don't assume a change caused these; verify against `main` first if they show up again.
|
|
|
|
## Database schema & migrations
|
|
|
|
- `prisma/schema.prisma` is the source of truth for the `conversion_jobs` schema; `prisma/migrations/` is its version history. `db/schema.sql` no longer exists.
|
|
- Prisma CLI commands (`prisma migrate dev`, `prisma migrate deploy`, `prisma generate`) need `DATABASE_URL` in their own process environment, separate from the app. Compute it from the same `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` values used for local tests, via `node scripts/printDatabaseUrl.js` — never write `DATABASE_URL` into `.env` or `.env.local`.
|
|
- **`prisma migrate dev` does NOT work against the local dev DB** — verified by actually running it (`DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate dev --name test_shadow_db_probe`). It needs to create/drop a temporary shadow database to detect schema drift, and the local `convert_user` does not have `CREATE DATABASE`/`DROP DATABASE` privileges, so it fails immediately with:
|
|
```
|
|
Error: P3014
|
|
Prisma Migrate could not create the shadow database. Please make sure the database user has permission to create databases.
|
|
Original error: Error code: P1010
|
|
User was denied access on the database `prisma_migrate_shadow_db_...`
|
|
```
|
|
No migration folder is created when it fails this way (it errors before writing anything), so nothing needs cleaning up afterward. Do not use `migrate dev` in this project until `convert_user`'s privileges change.
|
|
- To create a new migration locally after editing `prisma/schema.prisma`, use the shadow-database-free fallback instead. Note this is **not** the same pattern Task 3 used for the `0_init` baseline — Task 3 used `--from-empty` (diffing against a blank schema), which is a different mode that also happens not to need a shadow database, but is not applicable here since the local DB is not empty. The verified fallback for an already-baselined DB is `--from-schema-datasource` (introspects the live DB's actual current structure — a real DB connection, but not a shadow database) diffed against `--to-schema-datamodel` (reads the target state straight from a schema file, no DB connection at all):
|
|
```
|
|
DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage \
|
|
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate diff \
|
|
--from-schema-datasource prisma/schema.prisma --to-schema-datamodel prisma/schema.prisma --script > /tmp/migration.sql
|
|
```
|
|
This was verified twice against the local dev DB: with `prisma/schema.prisma` unchanged it printed `-- This is an empty migration.` (no shadow database requested, no error); with a throwaway copy of the schema (an extra `notes String? @db.Text` field on `ConversionJob`, passed as `--to-schema-datamodel <path-to-throwaway-copy>`) it printed a correct `ALTER TABLE conversion_jobs ADD COLUMN notes TEXT NULL;` — again with no shadow database involved. The earlier documented fallback here (`--from-migrations prisma/migrations`) was wrong: `--from-migrations` internally requires `--shadow-database-url` too, so it fails for the same P3014 reason `migrate dev` does; it was never actually tested before being written down.
|
|
|
|
Write the generated SQL to a path under the OS temp directory (e.g. `/tmp/migration.sql`), never to a file in the repo (an untracked `migration.sql` in the repo root is easy to accidentally commit). Review it, then manually create the next `prisma/migrations/<timestamp>_<description>/migration.sql` folder with that content, and mark it applied without executing it (since you'll apply it for real via `migrate deploy` or by hand):
|
|
```
|
|
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate resolve --applied <timestamp>_<description>
|
|
```
|
|
Delete the temp SQL file once it's been copied into the migration folder.
|
|
- To apply pending migrations in production: with the real `DB_*` values loaded from `.env`, run `DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate deploy` before starting the app.
|
|
- **ONE-TIME step, required before the very first `migrate deploy` against any environment whose `conversion_jobs` table predates Prisma** (this includes o2switch production, which still has the table created by hand from the now-deleted `db/schema.sql`, but no `_prisma_migrations` tracking table): do **not** run `migrate deploy` first. Instead, baseline that environment exactly like Task 3 did locally:
|
|
```
|
|
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate resolve --applied 0_init
|
|
```
|
|
This tells Prisma that `0_init` is already applied (the table already exists) without trying to `CREATE TABLE` it again. Run this once, ever, per environment — after that, `migrate deploy` is the correct command for all subsequent deployments to that environment. Running `migrate deploy` first (without this baseline step) against such an environment will fail with a MySQL "table already exists" error and leave Prisma's migration history in a failed state requiring manual recovery.
|