# Prisma ORM and DB Migrations Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Replace the hand-rolled `mariadb` pool and raw-SQL repository with Prisma Client for data access, and introduce Prisma Migrate (with a no-data-loss baseline) as the schema migration system for `conversion_jobs`. **Architecture:** `prisma/schema.prisma` becomes the single source of truth for the `conversion_jobs` table (replacing `db/schema.sql`), with `prisma/migrations/` as its version history. `src/db.js` swaps its exported `mariadb` pool for a singleton `PrismaClient`. `src/jobs/jobRepository.js` is rewritten onto Prisma's model API, keeping the same exported function names and `(client, ...)` argument shape so `src/app.js`, `src/worker.js`, and `src/cleanup.js` only need a `pool` → `prisma` rename, not a logic change. **Tech Stack:** Prisma ORM (`prisma`, `@prisma/client`) against the existing MariaDB instance via Prisma's `mysql` provider. Node.js ESM, Vitest, Supertest — all unchanged. ## Global Constraints - `prisma/schema.prisma` must reproduce `db/schema.sql` column-for-column, so the initial migration is a no-op against databases that already have the table (spec: "Schema"). Three documented, accepted exceptions where Prisma's MySQL provider has no schema-level equivalent (verified empirically against the real DB's `SHOW CREATE TABLE conversion_jobs`, and confirmed acceptable by the user on 2026-07-31): (1) table collation — the live MariaDB server defaults to `utf8mb4_uca1400_ai_ci` (a MariaDB-specific collation), but Prisma always emits `utf8mb4_unicode_ci` in generated DDL with no schema attribute to override it; (2) `updatedAt`'s `ON UPDATE CURRENT_TIMESTAMP` — Prisma's `@updatedAt` is implemented client-side only for the `mysql` provider and never emits a DB-level `ON UPDATE` clause; (3) storage engine — Prisma's generated DDL never emits an explicit `ENGINE=...` clause (no schema-level attribute controls it), relying on the server default, which is InnoDB on effectively every real MySQL/MariaDB install including this project's (confirmed during Task 3 execution on 2026-07-31, same root cause as (1): no schema-level attribute exists). None of the three have practical impact here: the baseline (Task 3) marks the initial migration applied without ever executing its SQL against the real database, and every write to `conversion_jobs` after this migration goes through Prisma Client, which sets `updatedAt` itself; a fresh environment created via `prisma migrate deploy` would still get InnoDB from the server default. All other columns (types, defaults, index/constraint names) must match exactly — see Task 1's schema, which uses `@db.DateTime(0)` and explicit `map:` names for this reason. - `DATABASE_URL` is never written into `.env` or `.env.local` — it is always computed on demand from the existing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` variables (spec: "Running Prisma CLI locally"). `.env` holds production credentials and must never be loaded for local work (CLAUDE.md). - The connection string includes `?connection_limit=10`, preserving the current pool's `connectionLimit: 10` (spec: "Running Prisma CLI locally"). - `getJobByUuid`, `findPendingJobs`, and `findExpiredJobs` must never expose `errorLog` — only `getJobErrorLog` does (spec: "Data access layer"). - `src/jobs/jobRepository.js` exported function names and their `(client, ...)` first-argument shape do not change (spec: "Data access layer"). - The local test convention of passing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` inline (documented in CLAUDE.md) must keep working unchanged after this migration. - After full implementation, the only acceptable test failures are the two already documented in CLAUDE.md (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related) — no new regressions (spec: "Behavior change to watch for"). --- ### Task 1: Add Prisma dependencies and the schema definition **Files:** - Modify: `package.json` - Create: `prisma/schema.prisma` **Interfaces:** - Produces: `prisma/schema.prisma` defining model `ConversionJob` (Prisma Client generates typed methods from this in Task 4). - [ ] **Step 1: Install the Prisma packages** Run: `npm install @prisma/client` Expected: `package.json` dependencies gains an `"@prisma/client"` entry; `package-lock.json` updated. Run: `npm install --save-dev prisma` Expected: `package.json` devDependencies gains a `"prisma"` entry; `package-lock.json` updated. - [ ] **Step 2: Create the Prisma schema** Create `prisma/schema.prisma`: ```prisma datasource db { provider = "mysql" url = env("DATABASE_URL") } generator client { provider = "prisma-client-js" } enum JobStatus { pending processing done failed } model ConversionJob { id Int @id @default(autoincrement()) @db.UnsignedInt uuid String @unique(map: "uniq_uuid") @db.Char(36) status JobStatus @default(pending) family String @db.VarChar(32) sourceFormat String @map("source_format") @db.VarChar(16) targetFormat String @map("target_format") @db.VarChar(16) originalFilename String @map("original_filename") @db.VarChar(255) inputPath String @map("input_path") @db.VarChar(255) outputPath String? @map("output_path") @db.VarChar(255) inputMimeType String @map("input_mime_type") @db.VarChar(128) outputMimeType String? @map("output_mime_type") @db.VarChar(128) inputSizeBytes Int @map("input_size_bytes") @db.UnsignedInt outputSizeBytes Int? @map("output_size_bytes") @db.UnsignedInt quality Int? @db.UnsignedSmallInt conversionDurationSeconds Decimal? @map("conversion_duration_seconds") @db.Decimal(10, 3) errorMessage String? @map("error_message") @db.VarChar(255) errorLog String? @map("error_log") @db.Text createdAt DateTime @default(now()) @map("created_at") @db.DateTime(0) updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.DateTime(0) expiresAt DateTime @map("expires_at") @db.DateTime(0) cleanedAt DateTime? @map("cleaned_at") @db.DateTime(0) @@index([status], map: "idx_status") @@index([expiresAt], map: "idx_expires_at") @@map("conversion_jobs") } ``` `@db.DateTime(0)` matches the real table's `datetime` columns (no fractional seconds) — without it Prisma defaults to `DATETIME(3)`/`CURRENT_TIMESTAMP(3)`, which would both mismatch the existing column type and risk an "Invalid default value" error on a fresh `prisma migrate deploy` (MySQL/MariaDB require the `CURRENT_TIMESTAMP(n)` default's precision to match the column's own precision). The `map: "..."` arguments make Prisma reuse the existing constraint/index names (`uniq_uuid`, `idx_status`, `idx_expires_at`) instead of auto-generating new ones. Verified empirically via `prisma migrate diff --from-empty --to-schema-datamodel` in a scratch directory on 2026-07-31. - [ ] **Step 3: Add the postinstall script** Edit `package.json`: old: ```json "scripts": { "start": "node src/server.js", "worker": "node src/worker.js", "cleanup": "node src/cleanup.js", "test": "vitest run", "build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend" }, ``` new: ```json "scripts": { "start": "node src/server.js", "worker": "node src/worker.js", "cleanup": "node src/cleanup.js", "test": "vitest run", "build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend", "postinstall": "prisma generate" }, ``` - [ ] **Step 4: Validate the schema** Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma validate` Expected: `The schema at prisma/schema.prisma is valid 🚀` (the connection string here is a placeholder only used to satisfy `env("DATABASE_URL")` resolution — `validate` does not connect to a database). - [ ] **Step 5: Generate the client** Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma generate` Expected: output confirms the client was generated, e.g. `Generated Prisma Client ... to ./node_modules/@prisma/client`. - [ ] **Step 6: Commit** ```bash git add package.json package-lock.json prisma/schema.prisma git commit -m "Add Prisma schema and client generation" ``` --- ### Task 2: Add `buildDatabaseUrl` and the `printDatabaseUrl` CLI helper **Files:** - Modify: `src/db.js` - Modify: `test/db.test.js` - Create: `scripts/printDatabaseUrl.js` **Interfaces:** - Consumes: `config.db = { host, user, password, database }` shape from `loadConfig()` (`src/config.js`, unchanged). - Produces: `buildDatabaseUrl(config): string` — used by Task 4's `getPrismaClient` and by `scripts/printDatabaseUrl.js`. This task is purely additive: `getPool`/`closePool` and the existing `mariadb` pool are untouched, so the whole app and test suite keep working exactly as today after this task. - [ ] **Step 1: Write the failing tests** Edit `test/db.test.js`, adding a new `describe` block above the existing `describe('getPool', ...)`: ```js import { describe, it, expect, afterAll } from 'vitest'; import { getPool, closePool, buildDatabaseUrl } from '../src/db.js'; import { loadConfig } from '../src/config.js'; describe('buildDatabaseUrl', () => { it('builds a mysql connection string from db config', () => { const url = buildDatabaseUrl({ db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' }, }); expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10'); }); it('percent-encodes special characters in user and password', () => { const url = buildDatabaseUrl({ db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' }, }); expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10'); }); }); describe('getPool', () => { afterAll(async () => { await closePool(); }); it('returns a working pool that can run a query', async () => { const config = loadConfig(); const pool = getPool(config); const rows = await pool.query('SELECT 1 AS value'); expect(Number(rows[0].value)).toBe(1); }); it('returns the same pool instance on repeated calls', () => { const config = loadConfig(); const poolA = getPool(config); const poolB = getPool(config); expect(poolA).toBe(poolB); }); }); ``` - [ ] **Step 2: Run the test file to verify the new tests fail** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js` Expected: FAIL — `buildDatabaseUrl is not a function` (or import error), the two new tests fail; the two pre-existing `getPool` tests still pass. - [ ] **Step 3: Implement `buildDatabaseUrl`** Edit `src/db.js`: old: ```js import mariadb from 'mariadb'; let pool; export function getPool(config) { ``` new: ```js import mariadb from 'mariadb'; let pool; export function buildDatabaseUrl(config) { const { host, user, password, database } = config.db; return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`; } export function getPool(config) { ``` - [ ] **Step 4: Run the test file to verify it passes** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js` Expected: PASS — all 4 tests green. - [ ] **Step 5: Create the CLI helper** Create `scripts/printDatabaseUrl.js`: ```js import { loadConfig } from '../src/config.js'; import { buildDatabaseUrl } from '../src/db.js'; console.log(buildDatabaseUrl(loadConfig())); ``` - [ ] **Step 6: Verify the helper manually** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js` Expected stdout: `mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10` - [ ] **Step 7: Commit** ```bash git add src/db.js test/db.test.js scripts/printDatabaseUrl.js git commit -m "Add buildDatabaseUrl and a CLI helper to print it" ``` --- ### Task 3: Baseline Prisma Migrate against the existing database **Files:** - Create: `prisma/migrations/migration_lock.toml` - Create: `prisma/migrations/0_init/migration.sql` - Modify: `CLAUDE.md` **Interfaces:** - Consumes: `scripts/printDatabaseUrl.js` (Task 2). - Produces: a migration history that Prisma considers already applied on the local dev DB, with `conversion_jobs` and its rows untouched. This task only touches Prisma's own bookkeeping (`prisma/migrations/`, and the `_prisma_migrations` tracking table it creates in the target database) — it must not run any SQL against `conversion_jobs` itself. - [ ] **Step 1: Record the current row count as a safety check** Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"` Note the number returned — it must be identical after Step 5. - [ ] **Step 2: Confirm the diff flag name for this installed Prisma version** Run: `npx prisma migrate diff --help` Expected: the help text lists a flag for rendering a schema file's data model as the diff target without connecting to a live database — documented in Prisma's migration-diffing workflow as `--to-schema-datamodel `. Confirm this exact flag name in the output before proceeding; if this installed version names it differently, substitute the flag `--help` reports in Step 4. - [ ] **Step 3: Create the migration folder and lock file** Run: `mkdir -p prisma/migrations/0_init` Create `prisma/migrations/migration_lock.toml`: ```toml # Please do not edit this file manually # It should be added in your version-control system (e.g., Git) provider = "mysql" ``` - [ ] **Step 4: Generate the migration SQL from the schema file (no database connection needed for this)** Run: ```bash npx prisma migrate diff \ --from-empty \ --to-schema-datamodel prisma/schema.prisma \ --script > prisma/migrations/0_init/migration.sql ``` Expected: `prisma/migrations/0_init/migration.sql` is created containing a single `CREATE TABLE conversion_jobs (...)` statement (plus its indexes) — no `DROP` or `ALTER` statements. - [ ] **Step 5: Read the generated SQL and confirm it matches `db/schema.sql`** Read `prisma/migrations/0_init/migration.sql` and compare column names, types, and constraints against `db/schema.sql`. They must describe the same table. If Prisma rendered a type differently than expected (e.g. a default value clause), stop and reconcile `prisma/schema.prisma` in Task 1 before continuing — do not hand-edit the generated SQL, regenerate it after fixing the schema. - [ ] **Step 6: Mark the migration as applied (bookkeeping only, does not run the SQL)** Run: ```bash DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \ npx prisma migrate resolve --applied 0_init ``` Expected: `Migration 0_init marked as applied.` - [ ] **Step 7: Verify migration status and that data is untouched** Run: ```bash DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \ npx prisma migrate status ``` Expected: `Database schema is up to date!` Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"` Expected: the same row count recorded in Step 1. - [ ] **Step 8: Document the migration workflow in CLAUDE.md** Edit `CLAUDE.md`, adding a new section after the existing "Local environment / running tests" section: ```markdown ## Database schema & migrations - `prisma/schema.prisma` is the source of truth for the `conversion_jobs` schema; `prisma/migrations/` is its version history. `db/schema.sql` no longer exists. - Prisma CLI commands (`prisma migrate dev`, `prisma migrate deploy`, `prisma generate`) need `DATABASE_URL` in their own process environment, separate from the app. Compute it from the same `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` values used for local tests, via `node scripts/printDatabaseUrl.js` — never write `DATABASE_URL` into `.env` or `.env.local`. - To create a new migration locally after editing `prisma/schema.prisma`: ``` DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage \ DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate dev --name ``` - To apply pending migrations in production: with the real `DB_*` values loaded from `.env`, run `DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate deploy` before starting the app. ``` - [ ] **Step 9: Commit** ```bash git add prisma/migrations CLAUDE.md git commit -m "Baseline Prisma Migrate against the existing conversion_jobs table" ``` --- ### Task 4: Rewrite the data access layer onto Prisma Client **Files:** - Modify: `src/db.js` - Modify: `src/jobs/jobRepository.js` - Modify: `test/db.test.js` - Modify: `test/jobs/jobRepository.test.js` **Interfaces:** - Produces: `getPrismaClient(config): PrismaClient`, `closePrismaClient(): Promise` (replacing `getPool`/`closePool`); `createJob`, `getJobByUuid`, `getJobErrorLog`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` — all still `(prisma, ...)` first-argument, same names, now backed by Prisma. After this task, `test/db.test.js` and `test/jobs/jobRepository.test.js` pass in isolation. Every other file that still imports `getPool`/`closePool` from `src/db.js` (`src/app.js`, `src/server.js`, `src/worker.js`, `src/cleanup.js`, and 5 remaining test files) will fail to import until Task 5 lands — this is expected and addressed next. - [ ] **Step 1: Rewrite `src/db.js`** Replace the full file content of `src/db.js`: ```js import { PrismaClient } from '@prisma/client'; let prisma; export function buildDatabaseUrl(config) { const { host, user, password, database } = config.db; return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`; } export function getPrismaClient(config) { if (!prisma) { process.env.DATABASE_URL = buildDatabaseUrl(config); prisma = new PrismaClient(); } return prisma; } export async function closePrismaClient() { if (prisma) { await prisma.$disconnect(); prisma = undefined; } } ``` - [ ] **Step 2: Rewrite `src/jobs/jobRepository.js`** Replace the full file content of `src/jobs/jobRepository.js`: ```js const jobSelect = { id: true, uuid: true, status: true, family: true, sourceFormat: true, targetFormat: true, originalFilename: true, inputPath: true, outputPath: true, inputMimeType: true, outputMimeType: true, inputSizeBytes: true, outputSizeBytes: true, quality: true, conversionDurationSeconds: true, errorMessage: true, createdAt: true, updatedAt: true, expiresAt: true, cleanedAt: true, }; export async function createJob(prisma, job) { await prisma.conversionJob.create({ data: { uuid: job.uuid, family: job.family, sourceFormat: job.sourceFormat, targetFormat: job.targetFormat, originalFilename: job.originalFilename, inputPath: job.inputPath, inputMimeType: job.inputMimeType, inputSizeBytes: job.inputSizeBytes, expiresAt: job.expiresAt, quality: job.quality ?? null, }, }); } export async function getJobByUuid(prisma, uuid) { return prisma.conversionJob.findUnique({ where: { uuid }, select: jobSelect }); } export async function getJobErrorLog(prisma, id) { const job = await prisma.conversionJob.findUnique({ where: { id }, select: { errorLog: true } }); return job?.errorLog ?? null; } export async function markProcessing(prisma, id) { await prisma.conversionJob.update({ where: { id }, data: { status: 'processing' } }); } export async function markDone(prisma, id, { outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }) { await prisma.conversionJob.update({ where: { id }, data: { status: 'done', outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }, }); } export async function markFailed(prisma, id, { errorMessage, errorLog }) { await prisma.conversionJob.update({ where: { id }, data: { status: 'failed', errorMessage, errorLog }, }); } export async function findPendingJobs(prisma, limit) { return prisma.conversionJob.findMany({ where: { status: 'pending' }, orderBy: { createdAt: 'asc' }, take: limit, select: jobSelect, }); } export async function findExpiredJobs(prisma) { return prisma.conversionJob.findMany({ where: { expiresAt: { lt: new Date() }, cleanedAt: null }, select: jobSelect, }); } export async function markCleaned(prisma, id) { await prisma.conversionJob.update({ where: { id }, data: { cleanedAt: new Date() } }); } ``` - [ ] **Step 3: Rewrite `test/db.test.js`** Replace the full file content of `test/db.test.js`: ```js import { describe, it, expect, afterAll } from 'vitest'; import { getPrismaClient, closePrismaClient, buildDatabaseUrl } from '../src/db.js'; import { loadConfig } from '../src/config.js'; describe('buildDatabaseUrl', () => { it('builds a mysql connection string from db config', () => { const url = buildDatabaseUrl({ db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' }, }); expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10'); }); it('percent-encodes special characters in user and password', () => { const url = buildDatabaseUrl({ db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' }, }); expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10'); }); }); describe('getPrismaClient', () => { afterAll(async () => { await closePrismaClient(); }); it('returns a working client that can run a query', async () => { const config = loadConfig(); const prisma = getPrismaClient(config); const rows = await prisma.$queryRaw`SELECT 1 AS value`; expect(Number(rows[0].value)).toBe(1); }); it('returns the same client instance on repeated calls', () => { const config = loadConfig(); const clientA = getPrismaClient(config); const clientB = getPrismaClient(config); expect(clientA).toBe(clientB); }); }); ``` - [ ] **Step 4: Rewrite `test/jobs/jobRepository.test.js`** Replace the full file content of `test/jobs/jobRepository.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import { getPrismaClient, closePrismaClient } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { createJob, getJobByUuid, getJobErrorLog, markProcessing, markDone, markFailed, findPendingJobs, findExpiredJobs, markCleaned, } from '../../src/jobs/jobRepository.js'; let prisma; beforeAll(() => { prisma = getPrismaClient(loadConfig()); }); afterAll(async () => { await closePrismaClient(); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); function baseJob(overrides = {}) { return { uuid: overrides.uuid ?? '11111111-1111-4111-8111-111111111111', family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${overrides.uuid ?? '11111111-1111-4111-8111-111111111111'}.png`, inputMimeType: 'image/png', inputSizeBytes: 1024, expiresAt: new Date(Date.now() + 3600 * 1000), ...overrides, }; } describe('jobRepository', () => { it('creates and retrieves a pending job', async () => { await createJob(prisma, baseJob()); const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); expect(job.id).toEqual(expect.any(Number)); expect(job.uuid).toBe('11111111-1111-4111-8111-111111111111'); expect(job.status).toBe('pending'); expect(job.family).toBe('image'); expect(job.sourceFormat).toBe('png'); expect(job.targetFormat).toBe('webp'); expect(job.originalFilename).toBe('photo.png'); expect(job.inputMimeType).toBe('image/png'); expect(job.inputSizeBytes).toBe(1024); expect(job.outputPath).toBeNull(); expect(job.outputSizeBytes).toBeNull(); expect(job.conversionDurationSeconds).toBeNull(); expect(job.errorMessage).toBeNull(); expect(job.cleanedAt).toBeNull(); expect(job.quality).toBeNull(); }); it('stores and retrieves a numeric quality value', async () => { await createJob(prisma, baseJob({ uuid: '66666666-6666-4666-8666-666666666666', quality: 82 })); const job = await getJobByUuid(prisma, '66666666-6666-4666-8666-666666666666'); expect(job.quality).toBe(82); }); it('returns null for an unknown uuid', async () => { const job = await getJobByUuid(prisma, '22222222-2222-4222-8222-222222222222'); expect(job).toBeNull(); }); it('transitions a job through processing to done', async () => { await createJob(prisma, baseJob()); await markProcessing(prisma, (await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111')).id); const processing = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); expect(processing.status).toBe('processing'); await markDone(prisma, processing.id, { outputPath: '11111111-1111-4111-8111-111111111111.webp', outputMimeType: 'image/webp', outputSizeBytes: 2048, conversionDurationSeconds: 1.5, }); const done = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); expect(done.status).toBe('done'); expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp'); expect(done.outputMimeType).toBe('image/webp'); expect(done.outputSizeBytes).toBe(2048); expect(Number(done.conversionDurationSeconds)).toBe(1.5); }); it('marks a job failed with a short message and a separate detailed log', async () => { await createJob(prisma, baseJob()); const created = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); await markFailed(prisma, created.id, { errorMessage: 'Conversion failed, please try again', errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)', }); const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); expect(job.status).toBe('failed'); expect(job.errorMessage).toBe('Conversion failed, please try again'); expect(job.errorLog).toBeUndefined(); const errorLog = await getJobErrorLog(prisma, created.id); expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)'); }); it('finds pending jobs oldest first, up to a limit', async () => { await createJob(prisma, baseJob({ uuid: '33333333-3333-4333-8333-333333333333' })); await createJob(prisma, baseJob({ uuid: '44444444-4444-4444-8444-444444444444' })); await createJob(prisma, baseJob({ uuid: '55555555-5555-4555-8555-555555555555' })); const jobs = await findPendingJobs(prisma, 2); expect(jobs).toHaveLength(2); expect(jobs[0].uuid).toBe('33333333-3333-4333-8333-333333333333'); expect(jobs[1].uuid).toBe('44444444-4444-4444-8444-444444444444'); }); it('finds expired jobs and allows marking them cleaned without deleting the row', async () => { await createJob(prisma, baseJob({ expiresAt: new Date(Date.now() - 1000) })); const expired = await findExpiredJobs(prisma); expect(expired).toHaveLength(1); expect(expired[0].uuid).toBe('11111111-1111-4111-8111-111111111111'); await markCleaned(prisma, expired[0].id); const stillPresent = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); expect(stillPresent).not.toBeNull(); expect(stillPresent.cleanedAt).not.toBeNull(); const afterCleaning = await findExpiredJobs(prisma); expect(afterCleaning).toHaveLength(0); }); }); ``` - [ ] **Step 5: Run the two updated test files** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js test/jobs/jobRepository.test.js` Expected: PASS — all tests green. (Running the full suite at this point will show import errors in other files; that's expected until Task 5.) - [ ] **Step 6: Commit** ```bash git add src/db.js src/jobs/jobRepository.js test/db.test.js test/jobs/jobRepository.test.js git commit -m "Rewrite the data access layer onto Prisma Client" ``` --- ### Task 5: Wire up the remaining call sites and tests **Files:** - Modify: `src/app.js` - Modify: `src/server.js` - Modify: `src/worker.js` - Modify: `src/cleanup.js` - Modify: `test/api/jobs.test.js` - Modify: `test/api/jobStatus.test.js` - Modify: `test/api/endToEnd.test.js` - Modify: `test/worker.test.js` - Modify: `test/cleanup.test.js` **Interfaces:** - Consumes: `getPrismaClient`, `closePrismaClient` (Task 4, `src/db.js`); `createJob`, `getJobByUuid`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` (Task 4, `src/jobs/jobRepository.js`). - Produces: `createApp(config, prisma)` (renamed second parameter, same behavior). After this task the full test suite should pass except the two pre-existing documented failures. - [ ] **Step 1: Update `src/app.js`** Edit `src/app.js`: old: ```js export function createApp(config, pool) { ``` new: ```js export function createApp(config, prisma) { ``` old: ```js const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); await createJob(pool, { ``` new: ```js const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); await createJob(prisma, { ``` old: ```js app.get('/api/jobs/:id', async (req, res) => { const job = await getJobByUuid(pool, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } ``` new: ```js app.get('/api/jobs/:id', async (req, res) => { const job = await getJobByUuid(prisma, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } ``` old: ```js app.get('/api/jobs/:id/download', async (req, res) => { const job = await getJobByUuid(pool, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } if (job.status !== 'done') { ``` new: ```js app.get('/api/jobs/:id/download', async (req, res) => { const job = await getJobByUuid(prisma, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } if (job.status !== 'done') { ``` - [ ] **Step 2: Update `src/server.js`** Replace the full file content of `src/server.js`: ```js import { loadConfig } from './config.js'; import { getPrismaClient } from './db.js'; import { ensureStorageDirs } from './storage.js'; import { createApp } from './app.js'; async function main() { const config = loadConfig(); await ensureStorageDirs(config); const prisma = getPrismaClient(config); const app = createApp(config, prisma); app.listen(config.port, () => { console.log(`File converter API listening on port ${config.port}`); }); } main().catch((error) => { console.error('Failed to start server:', error); process.exit(1); }); ``` - [ ] **Step 3: Update `src/worker.js`** Replace the full file content of `src/worker.js`: ```js import { pathToFileURL } from 'node:url'; import fs from 'node:fs/promises'; import { loadConfig } from './config.js'; import { getPrismaClient } from './db.js'; import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js'; import { outputMimeType } from './mime.js'; import { resolve as resolveConverter } from './converters/registry.js'; import { registerImageConverters } from './converters/image.js'; import { registerImageToPdfConverter } from './converters/imageToPdf.js'; import { registerDocumentConverters } from './converters/document.js'; import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js'; const JOB_TIMEOUT_MS = 60000; function withTimeout(promise, ms) { let timeoutId; const timeout = new Promise((_, reject) => { timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms); }); return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId)); } async function processJob(prisma, config, job) { await markProcessing(prisma, job.id); const inputFilePath = uploadPath(config, job.uuid, job.sourceFormat); const outputFilePath = outputPath(config, job.uuid, job.targetFormat); try { const entry = resolveConverter(job.sourceFormat, job.targetFormat); if (!entry) { throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`); } const startedAt = Date.now(); await withTimeout(entry.convert(inputFilePath, outputFilePath, { quality: job.quality }), JOB_TIMEOUT_MS); const conversionDurationSeconds = (Date.now() - startedAt) / 1000; const { size: outputSizeBytes } = await fs.stat(outputFilePath); await markDone(prisma, job.id, { outputPath: `${job.uuid}.${job.targetFormat}`, outputMimeType: outputMimeType(job.targetFormat), outputSizeBytes, conversionDurationSeconds, }); } catch (error) { await deleteIfExists(outputFilePath); await markFailed(prisma, job.id, { errorMessage: 'Conversion failed, please try again.', errorLog: error.stack ?? String(error), }); } } export async function processPendingJobs(prisma, config) { const jobs = await findPendingJobs(prisma, config.workerConcurrency); await Promise.all(jobs.map((job) => processJob(prisma, config, job))); return jobs.length; } export function startWorker(prisma, config) { const interval = setInterval(() => { processPendingJobs(prisma, config).catch((error) => { console.error('Error while processing pending jobs:', error); }); }, config.workerPollIntervalMs); return () => clearInterval(interval); } async function main() { const config = loadConfig(); await ensureStorageDirs(config); const prisma = getPrismaClient(config); registerImageConverters(); registerImageToPdfConverter(); registerDocumentConverters(); startWorker(prisma, config); console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`); } main(); ``` - [ ] **Step 4: Update `src/cleanup.js`** Replace the full file content of `src/cleanup.js`: ```js import { pathToFileURL } from 'node:url'; import { loadConfig } from './config.js'; import { getPrismaClient, closePrismaClient } from './db.js'; import { uploadPath, outputPath, deleteIfExists } from './storage.js'; import { findExpiredJobs, markCleaned } from './jobs/jobRepository.js'; export async function runCleanup(prisma, config) { const expiredJobs = await findExpiredJobs(prisma); for (const job of expiredJobs) { await deleteIfExists(uploadPath(config, job.uuid, job.sourceFormat)); await deleteIfExists(outputPath(config, job.uuid, job.targetFormat)); await markCleaned(prisma, job.id); } return expiredJobs.length; } async function main() { const config = loadConfig(); const prisma = getPrismaClient(config); const deletedCount = await runCleanup(prisma, config); console.log(`Cleanup: removed ${deletedCount} expired job(s).`); await closePrismaClient(); } if (import.meta.url === pathToFileURL(process.argv[1]).href) { main(); } ``` - [ ] **Step 5: Update `test/api/jobs.test.js`** Replace the full file content of `test/api/jobs.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { createApp } from '../../src/app.js'; import { getPrismaClient, closePrismaClient } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { getJobByUuid } from '../../src/jobs/jobRepository.js'; import { ensureStorageDirs } from '../../src/storage.js'; let app; let prisma; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) }; await ensureStorageDirs(config); prisma = getPrismaClient(config); app = createApp(config, prisma); }); afterAll(async () => { await closePrismaClient(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); describe('GET /api/formats', () => { it('lists valid target formats for a known source format', async () => { const response = await request(app).get('/api/formats').query({ source: 'png' }); expect(response.status).toBe(200); expect(response.body.targets).toContain('webp'); expect(response.body.targets).toContain('pdf'); }); it('returns an empty list for an unknown source format', async () => { const response = await request(app).get('/api/formats').query({ source: 'made-up' }); expect(response.status).toBe(200); expect(response.body.targets).toEqual([]); }); }); describe('POST /api/jobs', () => { it('creates a pending job for a valid image upload', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs).toHaveLength(1); expect(response.body.jobs[0].status).toBe('pending'); expect(response.body.jobs[0].file).toBe('photo.png'); const job = await getJobByUuid(prisma, response.body.jobs[0].id); expect(job.status).toBe('pending'); expect(job.sourceFormat).toBe('png'); expect(job.targetFormat).toBe('webp'); expect(job.originalFilename).toBe('photo.png'); expect(job.inputMimeType).toBe('image/png'); expect(job.inputSizeBytes).toBeGreaterThan(0); expect(job.quality).toBeNull(); }); it('creates a pending job with a quality level for a supported format', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['jpg'])) .field('qualities', JSON.stringify([45])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].status).toBe('pending'); const job = await getJobByUuid(prisma, response.body.jobs[0].id); expect(job.quality).toBe(45); }); it('rejects an out-of-range quality without failing the whole batch', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['jpg'])) .field('qualities', JSON.stringify([500])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/Invalid quality/); }); it('rejects a quality value for a format that does not support one', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['gif'])) .field('qualities', JSON.stringify([50])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/Invalid quality/); }); it('returns 400 when qualities length does not match the number of files', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['jpg'])) .field('qualities', JSON.stringify([10, 20])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(400); }); it('rejects a file whose content does not match its extension, without failing the whole batch', async () => { const fakePath = path.join(config.storageDir, 'fake.png'); await fs.writeFile(fakePath, 'not actually a png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fakePath, 'fake.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/does not match/); await fs.unlink(fakePath); }); it('rejects an unsupported source/target pair', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['made-up-format'])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/); }); it('returns 400 when targetFormats length does not match the number of files', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify([])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(400); }); it('returns 400 when no files are uploaded', async () => { const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify([])); expect(response.status).toBe(400); }); }); ``` - [ ] **Step 6: Update `test/api/jobStatus.test.js`** Replace the full file content of `test/api/jobStatus.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { createApp } from '../../src/app.js'; import { getPrismaClient, closePrismaClient } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { createJob, getJobByUuid, markDone } from '../../src/jobs/jobRepository.js'; import { ensureStorageDirs, outputPath } from '../../src/storage.js'; let app; let prisma; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) }; await ensureStorageDirs(config); prisma = getPrismaClient(config); app = createApp(config, prisma); }); afterAll(async () => { await closePrismaClient(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); function baseJob(uuid) { return { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'holiday photo.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() + 3600 * 1000), }; } describe('GET /api/jobs/:id', () => { it('returns job status without the error log field', async () => { const uuid = '66666666-6666-4666-8666-666666666666'; await createJob(prisma, baseJob(uuid)); const response = await request(app).get(`/api/jobs/${uuid}`); expect(response.status).toBe(200); expect(response.body.id).toBe(uuid); expect(response.body.status).toBe('pending'); expect(response.body.originalFilename).toBe('holiday photo.png'); expect(response.body.errorLog).toBeUndefined(); }); it('returns 404 for an unknown id', async () => { const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000'); expect(response.status).toBe(404); }); }); describe('GET /api/jobs/:id/download', () => { it('streams the converted file with correct headers once done', async () => { const uuid = '77777777-7777-4777-8777-777777777777'; await createJob(prisma, baseJob(uuid)); const filePath = outputPath(config, uuid, 'webp'); await fs.writeFile(filePath, Buffer.from('fake webp bytes')); const created = await getJobByUuid(prisma, uuid); await markDone(prisma, created.id, { outputPath: `${uuid}.webp`, outputMimeType: 'image/webp', outputSizeBytes: 16, conversionDurationSeconds: 0.2, }); const response = await request(app).get(`/api/jobs/${uuid}/download`); expect(response.status).toBe(200); expect(response.headers['content-type']).toBe('image/webp'); expect(response.headers['content-disposition']).toContain('holiday photo.webp'); expect(response.body ?? response.text).toBeDefined(); }); it('returns 409 when the job is not done yet', async () => { const uuid = '88888888-8888-4888-8888-888888888888'; await createJob(prisma, baseJob(uuid)); const response = await request(app).get(`/api/jobs/${uuid}/download`); expect(response.status).toBe(409); }); it('returns 404 for an unknown id', async () => { const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download'); expect(response.status).toBe(404); }); }); describe('unexpected server errors', () => { it('returns a generic 500 without leaking internal error details', async () => { const originalFindUnique = prisma.conversionJob.findUnique.bind(prisma.conversionJob); prisma.conversionJob.findUnique = () => Promise.reject(new Error('connection reset by peer')); const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999'); prisma.conversionJob.findUnique = originalFindUnique; expect(response.status).toBe(500); expect(response.body.error).toBe('Internal server error'); expect(JSON.stringify(response.body)).not.toContain('connection reset by peer'); }); }); ``` - [ ] **Step 7: Update `test/api/endToEnd.test.js`** Replace the full file content of `test/api/endToEnd.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { Document, Paragraph, TextRun, Packer } from 'docx'; import { createApp } from '../../src/app.js'; import { getPrismaClient, closePrismaClient } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { ensureStorageDirs } from '../../src/storage.js'; import { processPendingJobs } from '../../src/worker.js'; let app; let prisma; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) }; await ensureStorageDirs(config); prisma = getPrismaClient(config); app = createApp(config, prisma); }); afterAll(async () => { await closePrismaClient(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); async function waitForDone(id, maxAttempts = 20) { for (let attempt = 0; attempt < maxAttempts; attempt += 1) { await processPendingJobs(prisma, config); const response = await request(app).get(`/api/jobs/${id}`); if (response.body.status === 'done' || response.body.status === 'failed') { return response.body; } } throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`); } describe('end-to-end: image family', () => { it('uploads a PNG, converts it to WebP, and downloads the result', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const uploadResponse = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fixturePath, 'photo.png'); const { id } = uploadResponse.body.jobs[0]; const finalStatus = await waitForDone(id); expect(finalStatus.status).toBe('done'); const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); expect(downloadResponse.status).toBe(200); expect(downloadResponse.headers['content-type']).toBe('image/webp'); expect(downloadResponse.headers['content-disposition']).toContain('photo.webp'); }, 20000); }); describe('end-to-end: document family', () => { it('uploads a DOCX, converts it to PDF, and downloads the result', async () => { const doc = new Document({ sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }], }); const buffer = await Packer.toBuffer(doc); const docxPath = path.join(config.storageDir, 'e2e-fixture.docx'); await fs.writeFile(docxPath, buffer); const uploadResponse = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['pdf'])) .attach('files', docxPath, 'report.docx'); const { id } = uploadResponse.body.jobs[0]; const finalStatus = await waitForDone(id); expect(finalStatus.status).toBe('done'); const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); expect(downloadResponse.status).toBe(200); expect(downloadResponse.headers['content-type']).toBe('application/pdf'); expect(downloadResponse.headers['content-disposition']).toContain('report.pdf'); await fs.unlink(docxPath); }, 20000); }); ``` - [ ] **Step 8: Update `test/worker.test.js`** Replace the full file content of `test/worker.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import sharp from 'sharp'; import { getPrismaClient, closePrismaClient } from '../src/db.js'; import { loadConfig } from '../src/config.js'; import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; import { createJob, getJobByUuid, getJobErrorLog } from '../src/jobs/jobRepository.js'; import { registerImageConverters } from '../src/converters/image.js'; import { processPendingJobs } from '../src/worker.js'; let prisma; let config; beforeAll(async () => { registerImageConverters(); config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) }; await ensureStorageDirs(config); prisma = getPrismaClient(config); }); afterAll(async () => { await closePrismaClient(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); async function createPendingImageJob(uuid, sourceFormat, targetFormat) { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const inputFilePath = uploadPath(config, uuid, sourceFormat); await fs.copyFile(fixturePath, inputFilePath); const { size: inputSizeBytes } = await fs.stat(inputFilePath); await createJob(prisma, { uuid, family: 'image', sourceFormat, targetFormat, originalFilename: `photo.${sourceFormat}`, inputPath: `${uuid}.${sourceFormat}`, inputMimeType: 'image/png', inputSizeBytes, expiresAt: new Date(Date.now() + 3600 * 1000), }); } describe('processPendingJobs', () => { it('converts a pending image job to done', async () => { const uuid = '99999999-9999-4999-8999-999999999999'; await createPendingImageJob(uuid, 'png', 'webp'); const processedCount = await processPendingJobs(prisma, config); expect(processedCount).toBe(1); const job = await getJobByUuid(prisma, uuid); expect(job.status).toBe('done'); expect(job.outputPath).toBe(`${uuid}.webp`); expect(job.outputMimeType).toBe('image/webp'); expect(job.outputSizeBytes).toBeGreaterThan(0); expect(Number(job.conversionDurationSeconds)).toBeGreaterThanOrEqual(0); const stat = await fs.stat(outputPath(config, uuid, 'webp')); expect(stat.size).toBeGreaterThan(0); }); it('passes the job quality through to the converter, shrinking output for a low quality value', async () => { const lowUuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; const defaultUuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; const noisyBuffer = await sharp({ create: { width: 256, height: 256, channels: 3, noise: { type: 'gaussian', mean: 128, sigma: 40 }, }, }) .png() .toBuffer(); for (const uuid of [lowUuid, defaultUuid]) { await fs.writeFile(uploadPath(config, uuid, 'png'), noisyBuffer); } await createJob(prisma, { uuid: lowUuid, family: 'image', sourceFormat: 'png', targetFormat: 'jpg', originalFilename: 'noisy.png', inputPath: `${lowUuid}.png`, inputMimeType: 'image/png', inputSizeBytes: noisyBuffer.length, expiresAt: new Date(Date.now() + 3600 * 1000), quality: 5, }); await createJob(prisma, { uuid: defaultUuid, family: 'image', sourceFormat: 'png', targetFormat: 'jpg', originalFilename: 'noisy.png', inputPath: `${defaultUuid}.png`, inputMimeType: 'image/png', inputSizeBytes: noisyBuffer.length, expiresAt: new Date(Date.now() + 3600 * 1000), }); await processPendingJobs(prisma, { ...config, workerConcurrency: 2 }); const lowJob = await getJobByUuid(prisma, lowUuid); const defaultJob = await getJobByUuid(prisma, defaultUuid); expect(lowJob.status).toBe('done'); expect(defaultJob.status).toBe('done'); expect(lowJob.outputSizeBytes).toBeLessThan(defaultJob.outputSizeBytes); }); it('marks a job failed with a safe message and a detailed log when the converter throws', async () => { const uuid = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'; await createJob(prisma, { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'missing.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() + 3600 * 1000), }); // Note: input file is intentionally never written, so sharp will throw ENOENT. await processPendingJobs(prisma, config); const job = await getJobByUuid(prisma, uuid); expect(job.status).toBe('failed'); expect(job.errorMessage).toBe('Conversion failed, please try again.'); const errorLog = await getJobErrorLog(prisma, job.id); expect(errorLog).toMatch(/input file is missing/i); }); it('only picks up as many jobs as workerConcurrency allows', async () => { await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp'); await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp'); await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp'); const limitedConfig = { ...config, workerConcurrency: 2 }; const processedCount = await processPendingJobs(prisma, limitedConfig); expect(processedCount).toBe(2); }); }); ``` - [ ] **Step 9: Update `test/cleanup.test.js`** Replace the full file content of `test/cleanup.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { getPrismaClient, closePrismaClient } from '../src/db.js'; import { loadConfig } from '../src/config.js'; import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; import { createJob, markDone, getJobByUuid } from '../src/jobs/jobRepository.js'; import { runCleanup } from '../src/cleanup.js'; let prisma; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) }; await ensureStorageDirs(config); prisma = getPrismaClient(config); }); afterAll(async () => { await closePrismaClient(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await prisma.conversionJob.deleteMany(); }); describe('runCleanup', () => { it('marks an expired done job cleaned and removes its input and output files, keeping the row', async () => { const uuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes'); await fs.writeFile(outputPath(config, uuid, 'webp'), 'output bytes'); await createJob(prisma, { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() - 1000), }); const created = await getJobByUuid(prisma, uuid); await markDone(prisma, created.id, { outputPath: `${uuid}.webp`, outputMimeType: 'image/webp', outputSizeBytes: 12, conversionDurationSeconds: 0.5, }); const cleanedCount = await runCleanup(prisma, config); expect(cleanedCount).toBe(1); const job = await getJobByUuid(prisma, uuid); expect(job).not.toBeNull(); expect(job.cleanedAt).not.toBeNull(); await expect(fs.stat(uploadPath(config, uuid, 'png'))).rejects.toThrow(); await expect(fs.stat(outputPath(config, uuid, 'webp'))).rejects.toThrow(); }); it('marks an expired pending job (with no output file) cleaned without throwing', async () => { const uuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes'); await createJob(prisma, { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() - 1000), }); const cleanedCount = await runCleanup(prisma, config); expect(cleanedCount).toBe(1); const job = await getJobByUuid(prisma, uuid); expect(job).not.toBeNull(); expect(job.cleanedAt).not.toBeNull(); }); it('leaves non-expired jobs untouched', async () => { const uuid = '12121212-1212-4212-8212-121212121212'; await createJob(prisma, { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() + 3600 * 1000), }); const cleanedCount = await runCleanup(prisma, config); expect(cleanedCount).toBe(0); const job = await getJobByUuid(prisma, uuid); expect(job).not.toBeNull(); expect(job.cleanedAt).toBeNull(); }); it('does not re-process an already cleaned expired job', async () => { const uuid = '13131313-1313-4313-8313-131313131313'; await createJob(prisma, { uuid, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${uuid}.png`, inputMimeType: 'image/png', inputSizeBytes: 11, expiresAt: new Date(Date.now() - 1000), }); const firstRun = await runCleanup(prisma, config); const secondRun = await runCleanup(prisma, config); expect(firstRun).toBe(1); expect(secondRun).toBe(0); }); }); ``` - [ ] **Step 10: Run the full test suite** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` Expected: all tests pass except the two pre-existing documented failures (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related — verify against `main` if unsure whether a failure is new). - [ ] **Step 11: Commit** ```bash git add src/app.js src/server.js src/worker.js src/cleanup.js test/api/jobs.test.js test/api/jobStatus.test.js test/api/endToEnd.test.js test/worker.test.js test/cleanup.test.js git commit -m "Wire the app, worker, and cleanup entry points onto Prisma Client" ``` --- ### Task 6: Remove the mariadb dependency and the old schema file **Files:** - Modify: `package.json` - Delete: `db/schema.sql` **Interfaces:** none (cleanup only — no code depends on `mariadb` or `db/schema.sql` after Task 5). - [ ] **Step 1: Delete the old schema file** Run: `git rm db/schema.sql` - [ ] **Step 2: Remove the `mariadb` dependency** Edit `package.json`, removing the `"mariadb"` line from `dependencies` (keep the rest of the dependency list and its alphabetical ordering intact). - [ ] **Step 3: Update the lockfile** Run: `npm install` Expected: `package-lock.json` updates to remove `mariadb` and its transitive dependencies; no other dependency versions change. - [ ] **Step 4: Run the full test suite** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` Expected: same result as Task 5 Step 10 — all pass except the two pre-existing documented failures. - [ ] **Step 5: Commit** ```bash git add package.json package-lock.json db/schema.sql git commit -m "Remove the mariadb dependency and the superseded schema.sql" ``` --- ### Task 7: Final verification **Files:** none (verification only). - [ ] **Step 1: Run the full test suite one more time** Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` Expected: identical results to Task 6 Step 4. - [ ] **Step 2: Confirm the frontend build is unaffected** Run: `npm run build` Expected: succeeds, unrelated to the DB layer change. - [ ] **Step 3: Confirm `prisma generate` runs cleanly on a fresh install** Run: `rm -rf node_modules && npm install` Expected: `postinstall` runs `prisma generate` automatically as part of `npm install`, with no errors. No commit needed for this task if all checks pass — it is verification only.