diff --git a/docs/superpowers/plans/2026-07-31-prisma-orm-migrations.md b/docs/superpowers/plans/2026-07-31-prisma-orm-migrations.md new file mode 100644 index 0000000..3aaba1e --- /dev/null +++ b/docs/superpowers/plans/2026-07-31-prisma-orm-migrations.md @@ -0,0 +1,1711 @@ +# Prisma ORM and DB Migrations Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the hand-rolled `mariadb` pool and raw-SQL repository with Prisma Client for data access, and introduce Prisma Migrate (with a no-data-loss baseline) as the schema migration system for `conversion_jobs`. + +**Architecture:** `prisma/schema.prisma` becomes the single source of truth for the `conversion_jobs` table (replacing `db/schema.sql`), with `prisma/migrations/` as its version history. `src/db.js` swaps its exported `mariadb` pool for a singleton `PrismaClient`. `src/jobs/jobRepository.js` is rewritten onto Prisma's model API, keeping the same exported function names and `(client, ...)` argument shape so `src/app.js`, `src/worker.js`, and `src/cleanup.js` only need a `pool` → `prisma` rename, not a logic change. + +**Tech Stack:** Prisma ORM (`prisma`, `@prisma/client`) against the existing MariaDB instance via Prisma's `mysql` provider. Node.js ESM, Vitest, Supertest — all unchanged. + +## Global Constraints + +- `prisma/schema.prisma` must reproduce `db/schema.sql` column-for-column, so the initial migration is a no-op against databases that already have the table (spec: "Schema"). +- `DATABASE_URL` is never written into `.env` or `.env.local` — it is always computed on demand from the existing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` variables (spec: "Running Prisma CLI locally"). `.env` holds production credentials and must never be loaded for local work (CLAUDE.md). +- The connection string includes `?connection_limit=10`, preserving the current pool's `connectionLimit: 10` (spec: "Running Prisma CLI locally"). +- `getJobByUuid`, `findPendingJobs`, and `findExpiredJobs` must never expose `errorLog` — only `getJobErrorLog` does (spec: "Data access layer"). +- `src/jobs/jobRepository.js` exported function names and their `(client, ...)` first-argument shape do not change (spec: "Data access layer"). +- The local test convention of passing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` inline (documented in CLAUDE.md) must keep working unchanged after this migration. +- After full implementation, the only acceptable test failures are the two already documented in CLAUDE.md (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related) — no new regressions (spec: "Behavior change to watch for"). + +--- + +### Task 1: Add Prisma dependencies and the schema definition + +**Files:** +- Modify: `package.json` +- Create: `prisma/schema.prisma` + +**Interfaces:** +- Produces: `prisma/schema.prisma` defining model `ConversionJob` (Prisma Client generates typed methods from this in Task 4). + +- [ ] **Step 1: Install the Prisma packages** + +Run: `npm install @prisma/client` +Expected: `package.json` dependencies gains an `"@prisma/client"` entry; `package-lock.json` updated. + +Run: `npm install --save-dev prisma` +Expected: `package.json` devDependencies gains a `"prisma"` entry; `package-lock.json` updated. + +- [ ] **Step 2: Create the Prisma schema** + +Create `prisma/schema.prisma`: + +```prisma +datasource db { + provider = "mysql" + url = env("DATABASE_URL") +} + +generator client { + provider = "prisma-client-js" +} + +enum JobStatus { + pending + processing + done + failed +} + +model ConversionJob { + id Int @id @default(autoincrement()) @db.UnsignedInt + uuid String @unique @db.Char(36) + status JobStatus @default(pending) + family String @db.VarChar(32) + sourceFormat String @map("source_format") @db.VarChar(16) + targetFormat String @map("target_format") @db.VarChar(16) + originalFilename String @map("original_filename") @db.VarChar(255) + inputPath String @map("input_path") @db.VarChar(255) + outputPath String? @map("output_path") @db.VarChar(255) + inputMimeType String @map("input_mime_type") @db.VarChar(128) + outputMimeType String? @map("output_mime_type") @db.VarChar(128) + inputSizeBytes Int @map("input_size_bytes") @db.UnsignedInt + outputSizeBytes Int? @map("output_size_bytes") @db.UnsignedInt + quality Int? @db.UnsignedSmallInt + conversionDurationSeconds Decimal? @map("conversion_duration_seconds") @db.Decimal(10, 3) + errorMessage String? @map("error_message") @db.VarChar(255) + errorLog String? @map("error_log") @db.Text + createdAt DateTime @default(now()) @map("created_at") + updatedAt DateTime @default(now()) @updatedAt @map("updated_at") + expiresAt DateTime @map("expires_at") + cleanedAt DateTime? @map("cleaned_at") + + @@index([status]) + @@index([expiresAt]) + @@map("conversion_jobs") +} +``` + +- [ ] **Step 3: Add the postinstall script** + +Edit `package.json`: + +old: +```json + "scripts": { + "start": "node src/server.js", + "worker": "node src/worker.js", + "cleanup": "node src/cleanup.js", + "test": "vitest run", + "build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend" + }, +``` + +new: +```json + "scripts": { + "start": "node src/server.js", + "worker": "node src/worker.js", + "cleanup": "node src/cleanup.js", + "test": "vitest run", + "build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend", + "postinstall": "prisma generate" + }, +``` + +- [ ] **Step 4: Validate the schema** + +Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma validate` +Expected: `The schema at prisma/schema.prisma is valid 🚀` (the connection string here is a placeholder only used to satisfy `env("DATABASE_URL")` resolution — `validate` does not connect to a database). + +- [ ] **Step 5: Generate the client** + +Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma generate` +Expected: output confirms the client was generated, e.g. `Generated Prisma Client ... to ./node_modules/@prisma/client`. + +- [ ] **Step 6: Commit** + +```bash +git add package.json package-lock.json prisma/schema.prisma +git commit -m "Add Prisma schema and client generation" +``` + +--- + +### Task 2: Add `buildDatabaseUrl` and the `printDatabaseUrl` CLI helper + +**Files:** +- Modify: `src/db.js` +- Modify: `test/db.test.js` +- Create: `scripts/printDatabaseUrl.js` + +**Interfaces:** +- Consumes: `config.db = { host, user, password, database }` shape from `loadConfig()` (`src/config.js`, unchanged). +- Produces: `buildDatabaseUrl(config): string` — used by Task 4's `getPrismaClient` and by `scripts/printDatabaseUrl.js`. + +This task is purely additive: `getPool`/`closePool` and the existing `mariadb` pool are untouched, so the whole app and test suite keep working exactly as today after this task. + +- [ ] **Step 1: Write the failing tests** + +Edit `test/db.test.js`, adding a new `describe` block above the existing `describe('getPool', ...)`: + +```js +import { describe, it, expect, afterAll } from 'vitest'; +import { getPool, closePool, buildDatabaseUrl } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; + +describe('buildDatabaseUrl', () => { + it('builds a mysql connection string from db config', () => { + const url = buildDatabaseUrl({ + db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' }, + }); + + expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10'); + }); + + it('percent-encodes special characters in user and password', () => { + const url = buildDatabaseUrl({ + db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' }, + }); + + expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10'); + }); +}); + +describe('getPool', () => { + afterAll(async () => { + await closePool(); + }); + + it('returns a working pool that can run a query', async () => { + const config = loadConfig(); + const pool = getPool(config); + + const rows = await pool.query('SELECT 1 AS value'); + + expect(Number(rows[0].value)).toBe(1); + }); + + it('returns the same pool instance on repeated calls', () => { + const config = loadConfig(); + const poolA = getPool(config); + const poolB = getPool(config); + + expect(poolA).toBe(poolB); + }); +}); +``` + +- [ ] **Step 2: Run the test file to verify the new tests fail** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js` +Expected: FAIL — `buildDatabaseUrl is not a function` (or import error), the two new tests fail; the two pre-existing `getPool` tests still pass. + +- [ ] **Step 3: Implement `buildDatabaseUrl`** + +Edit `src/db.js`: + +old: +```js +import mariadb from 'mariadb'; + +let pool; + +export function getPool(config) { +``` + +new: +```js +import mariadb from 'mariadb'; + +let pool; + +export function buildDatabaseUrl(config) { + const { host, user, password, database } = config.db; + return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`; +} + +export function getPool(config) { +``` + +- [ ] **Step 4: Run the test file to verify it passes** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js` +Expected: PASS — all 4 tests green. + +- [ ] **Step 5: Create the CLI helper** + +Create `scripts/printDatabaseUrl.js`: + +```js +import { loadConfig } from '../src/config.js'; +import { buildDatabaseUrl } from '../src/db.js'; + +console.log(buildDatabaseUrl(loadConfig())); +``` + +- [ ] **Step 6: Verify the helper manually** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js` +Expected stdout: `mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10` + +- [ ] **Step 7: Commit** + +```bash +git add src/db.js test/db.test.js scripts/printDatabaseUrl.js +git commit -m "Add buildDatabaseUrl and a CLI helper to print it" +``` + +--- + +### Task 3: Baseline Prisma Migrate against the existing database + +**Files:** +- Create: `prisma/migrations/migration_lock.toml` +- Create: `prisma/migrations/0_init/migration.sql` +- Modify: `CLAUDE.md` + +**Interfaces:** +- Consumes: `scripts/printDatabaseUrl.js` (Task 2). +- Produces: a migration history that Prisma considers already applied on the local dev DB, with `conversion_jobs` and its rows untouched. + +This task only touches Prisma's own bookkeeping (`prisma/migrations/`, and the `_prisma_migrations` tracking table it creates in the target database) — it must not run any SQL against `conversion_jobs` itself. + +- [ ] **Step 1: Record the current row count as a safety check** + +Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"` +Note the number returned — it must be identical after Step 5. + +- [ ] **Step 2: Confirm the diff flag name for this installed Prisma version** + +Run: `npx prisma migrate diff --help` +Expected: the help text lists a flag for rendering a schema file's data model as the diff target without connecting to a live database — documented in Prisma's migration-diffing workflow as `--to-schema-datamodel `. Confirm this exact flag name in the output before proceeding; if this installed version names it differently, substitute the flag `--help` reports in Step 4. + +- [ ] **Step 3: Create the migration folder and lock file** + +Run: `mkdir -p prisma/migrations/0_init` + +Create `prisma/migrations/migration_lock.toml`: + +```toml +# Please do not edit this file manually +# It should be added in your version-control system (e.g., Git) +provider = "mysql" +``` + +- [ ] **Step 4: Generate the migration SQL from the schema file (no database connection needed for this)** + +Run: +```bash +npx prisma migrate diff \ + --from-empty \ + --to-schema-datamodel prisma/schema.prisma \ + --script > prisma/migrations/0_init/migration.sql +``` +Expected: `prisma/migrations/0_init/migration.sql` is created containing a single `CREATE TABLE conversion_jobs (...)` statement (plus its indexes) — no `DROP` or `ALTER` statements. + +- [ ] **Step 5: Read the generated SQL and confirm it matches `db/schema.sql`** + +Read `prisma/migrations/0_init/migration.sql` and compare column names, types, and constraints against `db/schema.sql`. They must describe the same table. If Prisma rendered a type differently than expected (e.g. a default value clause), stop and reconcile `prisma/schema.prisma` in Task 1 before continuing — do not hand-edit the generated SQL, regenerate it after fixing the schema. + +- [ ] **Step 6: Mark the migration as applied (bookkeeping only, does not run the SQL)** + +Run: +```bash +DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \ + npx prisma migrate resolve --applied 0_init +``` +Expected: `Migration 0_init marked as applied.` + +- [ ] **Step 7: Verify migration status and that data is untouched** + +Run: +```bash +DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \ + npx prisma migrate status +``` +Expected: `Database schema is up to date!` + +Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"` +Expected: the same row count recorded in Step 1. + +- [ ] **Step 8: Document the migration workflow in CLAUDE.md** + +Edit `CLAUDE.md`, adding a new section after the existing "Local environment / running tests" section: + +```markdown +## Database schema & migrations + +- `prisma/schema.prisma` is the source of truth for the `conversion_jobs` schema; `prisma/migrations/` is its version history. `db/schema.sql` no longer exists. +- Prisma CLI commands (`prisma migrate dev`, `prisma migrate deploy`, `prisma generate`) need `DATABASE_URL` in their own process environment, separate from the app. Compute it from the same `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` values used for local tests, via `node scripts/printDatabaseUrl.js` — never write `DATABASE_URL` into `.env` or `.env.local`. +- To create a new migration locally after editing `prisma/schema.prisma`: + ``` + DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage \ + DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate dev --name + ``` +- To apply pending migrations in production: with the real `DB_*` values loaded from `.env`, run `DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate deploy` before starting the app. +``` + +- [ ] **Step 9: Commit** + +```bash +git add prisma/migrations CLAUDE.md +git commit -m "Baseline Prisma Migrate against the existing conversion_jobs table" +``` + +--- + +### Task 4: Rewrite the data access layer onto Prisma Client + +**Files:** +- Modify: `src/db.js` +- Modify: `src/jobs/jobRepository.js` +- Modify: `test/db.test.js` +- Modify: `test/jobs/jobRepository.test.js` + +**Interfaces:** +- Produces: `getPrismaClient(config): PrismaClient`, `closePrismaClient(): Promise` (replacing `getPool`/`closePool`); `createJob`, `getJobByUuid`, `getJobErrorLog`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` — all still `(prisma, ...)` first-argument, same names, now backed by Prisma. + +After this task, `test/db.test.js` and `test/jobs/jobRepository.test.js` pass in isolation. Every other file that still imports `getPool`/`closePool` from `src/db.js` (`src/app.js`, `src/server.js`, `src/worker.js`, `src/cleanup.js`, and 5 remaining test files) will fail to import until Task 5 lands — this is expected and addressed next. + +- [ ] **Step 1: Rewrite `src/db.js`** + +Replace the full file content of `src/db.js`: + +```js +import { PrismaClient } from '@prisma/client'; + +let prisma; + +export function buildDatabaseUrl(config) { + const { host, user, password, database } = config.db; + return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`; +} + +export function getPrismaClient(config) { + if (!prisma) { + process.env.DATABASE_URL = buildDatabaseUrl(config); + prisma = new PrismaClient(); + } + return prisma; +} + +export async function closePrismaClient() { + if (prisma) { + await prisma.$disconnect(); + prisma = undefined; + } +} +``` + +- [ ] **Step 2: Rewrite `src/jobs/jobRepository.js`** + +Replace the full file content of `src/jobs/jobRepository.js`: + +```js +const jobSelect = { + id: true, + uuid: true, + status: true, + family: true, + sourceFormat: true, + targetFormat: true, + originalFilename: true, + inputPath: true, + outputPath: true, + inputMimeType: true, + outputMimeType: true, + inputSizeBytes: true, + outputSizeBytes: true, + quality: true, + conversionDurationSeconds: true, + errorMessage: true, + createdAt: true, + updatedAt: true, + expiresAt: true, + cleanedAt: true, +}; + +export async function createJob(prisma, job) { + await prisma.conversionJob.create({ + data: { + uuid: job.uuid, + family: job.family, + sourceFormat: job.sourceFormat, + targetFormat: job.targetFormat, + originalFilename: job.originalFilename, + inputPath: job.inputPath, + inputMimeType: job.inputMimeType, + inputSizeBytes: job.inputSizeBytes, + expiresAt: job.expiresAt, + quality: job.quality ?? null, + }, + }); +} + +export async function getJobByUuid(prisma, uuid) { + return prisma.conversionJob.findUnique({ where: { uuid }, select: jobSelect }); +} + +export async function getJobErrorLog(prisma, id) { + const job = await prisma.conversionJob.findUnique({ where: { id }, select: { errorLog: true } }); + return job?.errorLog ?? null; +} + +export async function markProcessing(prisma, id) { + await prisma.conversionJob.update({ where: { id }, data: { status: 'processing' } }); +} + +export async function markDone(prisma, id, { outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }) { + await prisma.conversionJob.update({ + where: { id }, + data: { status: 'done', outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }, + }); +} + +export async function markFailed(prisma, id, { errorMessage, errorLog }) { + await prisma.conversionJob.update({ + where: { id }, + data: { status: 'failed', errorMessage, errorLog }, + }); +} + +export async function findPendingJobs(prisma, limit) { + return prisma.conversionJob.findMany({ + where: { status: 'pending' }, + orderBy: { createdAt: 'asc' }, + take: limit, + select: jobSelect, + }); +} + +export async function findExpiredJobs(prisma) { + return prisma.conversionJob.findMany({ + where: { expiresAt: { lt: new Date() }, cleanedAt: null }, + select: jobSelect, + }); +} + +export async function markCleaned(prisma, id) { + await prisma.conversionJob.update({ where: { id }, data: { cleanedAt: new Date() } }); +} +``` + +- [ ] **Step 3: Rewrite `test/db.test.js`** + +Replace the full file content of `test/db.test.js`: + +```js +import { describe, it, expect, afterAll } from 'vitest'; +import { getPrismaClient, closePrismaClient, buildDatabaseUrl } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; + +describe('buildDatabaseUrl', () => { + it('builds a mysql connection string from db config', () => { + const url = buildDatabaseUrl({ + db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' }, + }); + + expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10'); + }); + + it('percent-encodes special characters in user and password', () => { + const url = buildDatabaseUrl({ + db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' }, + }); + + expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10'); + }); +}); + +describe('getPrismaClient', () => { + afterAll(async () => { + await closePrismaClient(); + }); + + it('returns a working client that can run a query', async () => { + const config = loadConfig(); + const prisma = getPrismaClient(config); + + const rows = await prisma.$queryRaw`SELECT 1 AS value`; + + expect(Number(rows[0].value)).toBe(1); + }); + + it('returns the same client instance on repeated calls', () => { + const config = loadConfig(); + const clientA = getPrismaClient(config); + const clientB = getPrismaClient(config); + + expect(clientA).toBe(clientB); + }); +}); +``` + +- [ ] **Step 4: Rewrite `test/jobs/jobRepository.test.js`** + +Replace the full file content of `test/jobs/jobRepository.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { getPrismaClient, closePrismaClient } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { + createJob, + getJobByUuid, + getJobErrorLog, + markProcessing, + markDone, + markFailed, + findPendingJobs, + findExpiredJobs, + markCleaned, +} from '../../src/jobs/jobRepository.js'; + +let prisma; + +beforeAll(() => { + prisma = getPrismaClient(loadConfig()); +}); + +afterAll(async () => { + await closePrismaClient(); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +function baseJob(overrides = {}) { + return { + uuid: overrides.uuid ?? '11111111-1111-4111-8111-111111111111', + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${overrides.uuid ?? '11111111-1111-4111-8111-111111111111'}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 1024, + expiresAt: new Date(Date.now() + 3600 * 1000), + ...overrides, + }; +} + +describe('jobRepository', () => { + it('creates and retrieves a pending job', async () => { + await createJob(prisma, baseJob()); + + const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + + expect(job.id).toEqual(expect.any(Number)); + expect(job.uuid).toBe('11111111-1111-4111-8111-111111111111'); + expect(job.status).toBe('pending'); + expect(job.family).toBe('image'); + expect(job.sourceFormat).toBe('png'); + expect(job.targetFormat).toBe('webp'); + expect(job.originalFilename).toBe('photo.png'); + expect(job.inputMimeType).toBe('image/png'); + expect(job.inputSizeBytes).toBe(1024); + expect(job.outputPath).toBeNull(); + expect(job.outputSizeBytes).toBeNull(); + expect(job.conversionDurationSeconds).toBeNull(); + expect(job.errorMessage).toBeNull(); + expect(job.cleanedAt).toBeNull(); + expect(job.quality).toBeNull(); + }); + + it('stores and retrieves a numeric quality value', async () => { + await createJob(prisma, baseJob({ uuid: '66666666-6666-4666-8666-666666666666', quality: 82 })); + + const job = await getJobByUuid(prisma, '66666666-6666-4666-8666-666666666666'); + expect(job.quality).toBe(82); + }); + + it('returns null for an unknown uuid', async () => { + const job = await getJobByUuid(prisma, '22222222-2222-4222-8222-222222222222'); + expect(job).toBeNull(); + }); + + it('transitions a job through processing to done', async () => { + await createJob(prisma, baseJob()); + + await markProcessing(prisma, (await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111')).id); + const processing = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + expect(processing.status).toBe('processing'); + + await markDone(prisma, processing.id, { + outputPath: '11111111-1111-4111-8111-111111111111.webp', + outputMimeType: 'image/webp', + outputSizeBytes: 2048, + conversionDurationSeconds: 1.5, + }); + const done = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + expect(done.status).toBe('done'); + expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp'); + expect(done.outputMimeType).toBe('image/webp'); + expect(done.outputSizeBytes).toBe(2048); + expect(Number(done.conversionDurationSeconds)).toBe(1.5); + }); + + it('marks a job failed with a short message and a separate detailed log', async () => { + await createJob(prisma, baseJob()); + const created = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + + await markFailed(prisma, created.id, { + errorMessage: 'Conversion failed, please try again', + errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)', + }); + + const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + expect(job.status).toBe('failed'); + expect(job.errorMessage).toBe('Conversion failed, please try again'); + expect(job.errorLog).toBeUndefined(); + + const errorLog = await getJobErrorLog(prisma, created.id); + expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)'); + }); + + it('finds pending jobs oldest first, up to a limit', async () => { + await createJob(prisma, baseJob({ uuid: '33333333-3333-4333-8333-333333333333' })); + await createJob(prisma, baseJob({ uuid: '44444444-4444-4444-8444-444444444444' })); + await createJob(prisma, baseJob({ uuid: '55555555-5555-4555-8555-555555555555' })); + + const jobs = await findPendingJobs(prisma, 2); + + expect(jobs).toHaveLength(2); + expect(jobs[0].uuid).toBe('33333333-3333-4333-8333-333333333333'); + expect(jobs[1].uuid).toBe('44444444-4444-4444-8444-444444444444'); + }); + + it('finds expired jobs and allows marking them cleaned without deleting the row', async () => { + await createJob(prisma, baseJob({ expiresAt: new Date(Date.now() - 1000) })); + + const expired = await findExpiredJobs(prisma); + expect(expired).toHaveLength(1); + expect(expired[0].uuid).toBe('11111111-1111-4111-8111-111111111111'); + + await markCleaned(prisma, expired[0].id); + + const stillPresent = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111'); + expect(stillPresent).not.toBeNull(); + expect(stillPresent.cleanedAt).not.toBeNull(); + + const afterCleaning = await findExpiredJobs(prisma); + expect(afterCleaning).toHaveLength(0); + }); +}); +``` + +- [ ] **Step 5: Run the two updated test files** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js test/jobs/jobRepository.test.js` +Expected: PASS — all tests green. (Running the full suite at this point will show import errors in other files; that's expected until Task 5.) + +- [ ] **Step 6: Commit** + +```bash +git add src/db.js src/jobs/jobRepository.js test/db.test.js test/jobs/jobRepository.test.js +git commit -m "Rewrite the data access layer onto Prisma Client" +``` + +--- + +### Task 5: Wire up the remaining call sites and tests + +**Files:** +- Modify: `src/app.js` +- Modify: `src/server.js` +- Modify: `src/worker.js` +- Modify: `src/cleanup.js` +- Modify: `test/api/jobs.test.js` +- Modify: `test/api/jobStatus.test.js` +- Modify: `test/api/endToEnd.test.js` +- Modify: `test/worker.test.js` +- Modify: `test/cleanup.test.js` + +**Interfaces:** +- Consumes: `getPrismaClient`, `closePrismaClient` (Task 4, `src/db.js`); `createJob`, `getJobByUuid`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` (Task 4, `src/jobs/jobRepository.js`). +- Produces: `createApp(config, prisma)` (renamed second parameter, same behavior). + +After this task the full test suite should pass except the two pre-existing documented failures. + +- [ ] **Step 1: Update `src/app.js`** + +Edit `src/app.js`: + +old: +```js +export function createApp(config, pool) { +``` +new: +```js +export function createApp(config, prisma) { +``` + +old: +```js + const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); + await createJob(pool, { +``` +new: +```js + const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); + await createJob(prisma, { +``` + +old: +```js + app.get('/api/jobs/:id', async (req, res) => { + const job = await getJobByUuid(pool, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } +``` +new: +```js + app.get('/api/jobs/:id', async (req, res) => { + const job = await getJobByUuid(prisma, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } +``` + +old: +```js + app.get('/api/jobs/:id/download', async (req, res) => { + const job = await getJobByUuid(pool, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } + if (job.status !== 'done') { +``` +new: +```js + app.get('/api/jobs/:id/download', async (req, res) => { + const job = await getJobByUuid(prisma, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } + if (job.status !== 'done') { +``` + +- [ ] **Step 2: Update `src/server.js`** + +Replace the full file content of `src/server.js`: + +```js +import { loadConfig } from './config.js'; +import { getPrismaClient } from './db.js'; +import { ensureStorageDirs } from './storage.js'; +import { createApp } from './app.js'; + +async function main() { + const config = loadConfig(); + await ensureStorageDirs(config); + const prisma = getPrismaClient(config); + const app = createApp(config, prisma); + + app.listen(config.port, () => { + console.log(`File converter API listening on port ${config.port}`); + }); +} + +main().catch((error) => { + console.error('Failed to start server:', error); + process.exit(1); +}); +``` + +- [ ] **Step 3: Update `src/worker.js`** + +Replace the full file content of `src/worker.js`: + +```js +import { pathToFileURL } from 'node:url'; +import fs from 'node:fs/promises'; +import { loadConfig } from './config.js'; +import { getPrismaClient } from './db.js'; +import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js'; +import { outputMimeType } from './mime.js'; +import { resolve as resolveConverter } from './converters/registry.js'; +import { registerImageConverters } from './converters/image.js'; +import { registerImageToPdfConverter } from './converters/imageToPdf.js'; +import { registerDocumentConverters } from './converters/document.js'; +import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js'; + +const JOB_TIMEOUT_MS = 60000; + +function withTimeout(promise, ms) { + let timeoutId; + const timeout = new Promise((_, reject) => { + timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms); + }); + return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId)); +} + +async function processJob(prisma, config, job) { + await markProcessing(prisma, job.id); + + const inputFilePath = uploadPath(config, job.uuid, job.sourceFormat); + const outputFilePath = outputPath(config, job.uuid, job.targetFormat); + + try { + const entry = resolveConverter(job.sourceFormat, job.targetFormat); + if (!entry) { + throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`); + } + + const startedAt = Date.now(); + await withTimeout(entry.convert(inputFilePath, outputFilePath, { quality: job.quality }), JOB_TIMEOUT_MS); + const conversionDurationSeconds = (Date.now() - startedAt) / 1000; + const { size: outputSizeBytes } = await fs.stat(outputFilePath); + + await markDone(prisma, job.id, { + outputPath: `${job.uuid}.${job.targetFormat}`, + outputMimeType: outputMimeType(job.targetFormat), + outputSizeBytes, + conversionDurationSeconds, + }); + } catch (error) { + await deleteIfExists(outputFilePath); + await markFailed(prisma, job.id, { + errorMessage: 'Conversion failed, please try again.', + errorLog: error.stack ?? String(error), + }); + } +} + +export async function processPendingJobs(prisma, config) { + const jobs = await findPendingJobs(prisma, config.workerConcurrency); + await Promise.all(jobs.map((job) => processJob(prisma, config, job))); + return jobs.length; +} + +export function startWorker(prisma, config) { + const interval = setInterval(() => { + processPendingJobs(prisma, config).catch((error) => { + console.error('Error while processing pending jobs:', error); + }); + }, config.workerPollIntervalMs); + return () => clearInterval(interval); +} + +async function main() { + const config = loadConfig(); + await ensureStorageDirs(config); + const prisma = getPrismaClient(config); + + registerImageConverters(); + registerImageToPdfConverter(); + registerDocumentConverters(); + + startWorker(prisma, config); + console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`); +} + +main(); +``` + +- [ ] **Step 4: Update `src/cleanup.js`** + +Replace the full file content of `src/cleanup.js`: + +```js +import { pathToFileURL } from 'node:url'; +import { loadConfig } from './config.js'; +import { getPrismaClient, closePrismaClient } from './db.js'; +import { uploadPath, outputPath, deleteIfExists } from './storage.js'; +import { findExpiredJobs, markCleaned } from './jobs/jobRepository.js'; + +export async function runCleanup(prisma, config) { + const expiredJobs = await findExpiredJobs(prisma); + + for (const job of expiredJobs) { + await deleteIfExists(uploadPath(config, job.uuid, job.sourceFormat)); + await deleteIfExists(outputPath(config, job.uuid, job.targetFormat)); + await markCleaned(prisma, job.id); + } + + return expiredJobs.length; +} + +async function main() { + const config = loadConfig(); + const prisma = getPrismaClient(config); + const deletedCount = await runCleanup(prisma, config); + console.log(`Cleanup: removed ${deletedCount} expired job(s).`); + await closePrismaClient(); +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + main(); +} +``` + +- [ ] **Step 5: Update `test/api/jobs.test.js`** + +Replace the full file content of `test/api/jobs.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { createApp } from '../../src/app.js'; +import { getPrismaClient, closePrismaClient } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { getJobByUuid } from '../../src/jobs/jobRepository.js'; +import { ensureStorageDirs } from '../../src/storage.js'; + +let app; +let prisma; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) }; + await ensureStorageDirs(config); + prisma = getPrismaClient(config); + app = createApp(config, prisma); +}); + +afterAll(async () => { + await closePrismaClient(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +describe('GET /api/formats', () => { + it('lists valid target formats for a known source format', async () => { + const response = await request(app).get('/api/formats').query({ source: 'png' }); + + expect(response.status).toBe(200); + expect(response.body.targets).toContain('webp'); + expect(response.body.targets).toContain('pdf'); + }); + + it('returns an empty list for an unknown source format', async () => { + const response = await request(app).get('/api/formats').query({ source: 'made-up' }); + + expect(response.status).toBe(200); + expect(response.body.targets).toEqual([]); + }); +}); + +describe('POST /api/jobs', () => { + it('creates a pending job for a valid image upload', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs).toHaveLength(1); + expect(response.body.jobs[0].status).toBe('pending'); + expect(response.body.jobs[0].file).toBe('photo.png'); + + const job = await getJobByUuid(prisma, response.body.jobs[0].id); + expect(job.status).toBe('pending'); + expect(job.sourceFormat).toBe('png'); + expect(job.targetFormat).toBe('webp'); + expect(job.originalFilename).toBe('photo.png'); + expect(job.inputMimeType).toBe('image/png'); + expect(job.inputSizeBytes).toBeGreaterThan(0); + expect(job.quality).toBeNull(); + }); + + it('creates a pending job with a quality level for a supported format', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['jpg'])) + .field('qualities', JSON.stringify([45])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].status).toBe('pending'); + + const job = await getJobByUuid(prisma, response.body.jobs[0].id); + expect(job.quality).toBe(45); + }); + + it('rejects an out-of-range quality without failing the whole batch', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['jpg'])) + .field('qualities', JSON.stringify([500])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/Invalid quality/); + }); + + it('rejects a quality value for a format that does not support one', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['gif'])) + .field('qualities', JSON.stringify([50])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/Invalid quality/); + }); + + it('returns 400 when qualities length does not match the number of files', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['jpg'])) + .field('qualities', JSON.stringify([10, 20])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(400); + }); + + it('rejects a file whose content does not match its extension, without failing the whole batch', async () => { + const fakePath = path.join(config.storageDir, 'fake.png'); + await fs.writeFile(fakePath, 'not actually a png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fakePath, 'fake.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/does not match/); + + await fs.unlink(fakePath); + }); + + it('rejects an unsupported source/target pair', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['made-up-format'])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/); + }); + + it('returns 400 when targetFormats length does not match the number of files', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify([])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(400); + }); + + it('returns 400 when no files are uploaded', async () => { + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify([])); + + expect(response.status).toBe(400); + }); +}); +``` + +- [ ] **Step 6: Update `test/api/jobStatus.test.js`** + +Replace the full file content of `test/api/jobStatus.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { createApp } from '../../src/app.js'; +import { getPrismaClient, closePrismaClient } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { createJob, getJobByUuid, markDone } from '../../src/jobs/jobRepository.js'; +import { ensureStorageDirs, outputPath } from '../../src/storage.js'; + +let app; +let prisma; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) }; + await ensureStorageDirs(config); + prisma = getPrismaClient(config); + app = createApp(config, prisma); +}); + +afterAll(async () => { + await closePrismaClient(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +function baseJob(uuid) { + return { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'holiday photo.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() + 3600 * 1000), + }; +} + +describe('GET /api/jobs/:id', () => { + it('returns job status without the error log field', async () => { + const uuid = '66666666-6666-4666-8666-666666666666'; + await createJob(prisma, baseJob(uuid)); + + const response = await request(app).get(`/api/jobs/${uuid}`); + + expect(response.status).toBe(200); + expect(response.body.id).toBe(uuid); + expect(response.body.status).toBe('pending'); + expect(response.body.originalFilename).toBe('holiday photo.png'); + expect(response.body.errorLog).toBeUndefined(); + }); + + it('returns 404 for an unknown id', async () => { + const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000'); + expect(response.status).toBe(404); + }); +}); + +describe('GET /api/jobs/:id/download', () => { + it('streams the converted file with correct headers once done', async () => { + const uuid = '77777777-7777-4777-8777-777777777777'; + await createJob(prisma, baseJob(uuid)); + const filePath = outputPath(config, uuid, 'webp'); + await fs.writeFile(filePath, Buffer.from('fake webp bytes')); + const created = await getJobByUuid(prisma, uuid); + await markDone(prisma, created.id, { + outputPath: `${uuid}.webp`, + outputMimeType: 'image/webp', + outputSizeBytes: 16, + conversionDurationSeconds: 0.2, + }); + + const response = await request(app).get(`/api/jobs/${uuid}/download`); + + expect(response.status).toBe(200); + expect(response.headers['content-type']).toBe('image/webp'); + expect(response.headers['content-disposition']).toContain('holiday photo.webp'); + expect(response.body ?? response.text).toBeDefined(); + }); + + it('returns 409 when the job is not done yet', async () => { + const uuid = '88888888-8888-4888-8888-888888888888'; + await createJob(prisma, baseJob(uuid)); + + const response = await request(app).get(`/api/jobs/${uuid}/download`); + + expect(response.status).toBe(409); + }); + + it('returns 404 for an unknown id', async () => { + const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download'); + expect(response.status).toBe(404); + }); +}); + +describe('unexpected server errors', () => { + it('returns a generic 500 without leaking internal error details', async () => { + const originalFindUnique = prisma.conversionJob.findUnique.bind(prisma.conversionJob); + prisma.conversionJob.findUnique = () => Promise.reject(new Error('connection reset by peer')); + + const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999'); + + prisma.conversionJob.findUnique = originalFindUnique; + + expect(response.status).toBe(500); + expect(response.body.error).toBe('Internal server error'); + expect(JSON.stringify(response.body)).not.toContain('connection reset by peer'); + }); +}); +``` + +- [ ] **Step 7: Update `test/api/endToEnd.test.js`** + +Replace the full file content of `test/api/endToEnd.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { Document, Paragraph, TextRun, Packer } from 'docx'; +import { createApp } from '../../src/app.js'; +import { getPrismaClient, closePrismaClient } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { ensureStorageDirs } from '../../src/storage.js'; +import { processPendingJobs } from '../../src/worker.js'; + +let app; +let prisma; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) }; + await ensureStorageDirs(config); + prisma = getPrismaClient(config); + app = createApp(config, prisma); +}); + +afterAll(async () => { + await closePrismaClient(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +async function waitForDone(id, maxAttempts = 20) { + for (let attempt = 0; attempt < maxAttempts; attempt += 1) { + await processPendingJobs(prisma, config); + const response = await request(app).get(`/api/jobs/${id}`); + if (response.body.status === 'done' || response.body.status === 'failed') { + return response.body; + } + } + throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`); +} + +describe('end-to-end: image family', () => { + it('uploads a PNG, converts it to WebP, and downloads the result', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const uploadResponse = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fixturePath, 'photo.png'); + + const { id } = uploadResponse.body.jobs[0]; + const finalStatus = await waitForDone(id); + expect(finalStatus.status).toBe('done'); + + const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); + expect(downloadResponse.status).toBe(200); + expect(downloadResponse.headers['content-type']).toBe('image/webp'); + expect(downloadResponse.headers['content-disposition']).toContain('photo.webp'); + }, 20000); +}); + +describe('end-to-end: document family', () => { + it('uploads a DOCX, converts it to PDF, and downloads the result', async () => { + const doc = new Document({ + sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }], + }); + const buffer = await Packer.toBuffer(doc); + const docxPath = path.join(config.storageDir, 'e2e-fixture.docx'); + await fs.writeFile(docxPath, buffer); + + const uploadResponse = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['pdf'])) + .attach('files', docxPath, 'report.docx'); + + const { id } = uploadResponse.body.jobs[0]; + const finalStatus = await waitForDone(id); + expect(finalStatus.status).toBe('done'); + + const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); + expect(downloadResponse.status).toBe(200); + expect(downloadResponse.headers['content-type']).toBe('application/pdf'); + expect(downloadResponse.headers['content-disposition']).toContain('report.pdf'); + + await fs.unlink(docxPath); + }, 20000); +}); +``` + +- [ ] **Step 8: Update `test/worker.test.js`** + +Replace the full file content of `test/worker.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import sharp from 'sharp'; +import { getPrismaClient, closePrismaClient } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; +import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; +import { createJob, getJobByUuid, getJobErrorLog } from '../src/jobs/jobRepository.js'; +import { registerImageConverters } from '../src/converters/image.js'; +import { processPendingJobs } from '../src/worker.js'; + +let prisma; +let config; + +beforeAll(async () => { + registerImageConverters(); + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) }; + await ensureStorageDirs(config); + prisma = getPrismaClient(config); +}); + +afterAll(async () => { + await closePrismaClient(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +async function createPendingImageJob(uuid, sourceFormat, targetFormat) { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + const inputFilePath = uploadPath(config, uuid, sourceFormat); + await fs.copyFile(fixturePath, inputFilePath); + const { size: inputSizeBytes } = await fs.stat(inputFilePath); + + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat, + targetFormat, + originalFilename: `photo.${sourceFormat}`, + inputPath: `${uuid}.${sourceFormat}`, + inputMimeType: 'image/png', + inputSizeBytes, + expiresAt: new Date(Date.now() + 3600 * 1000), + }); +} + +describe('processPendingJobs', () => { + it('converts a pending image job to done', async () => { + const uuid = '99999999-9999-4999-8999-999999999999'; + await createPendingImageJob(uuid, 'png', 'webp'); + + const processedCount = await processPendingJobs(prisma, config); + + expect(processedCount).toBe(1); + const job = await getJobByUuid(prisma, uuid); + expect(job.status).toBe('done'); + expect(job.outputPath).toBe(`${uuid}.webp`); + expect(job.outputMimeType).toBe('image/webp'); + expect(job.outputSizeBytes).toBeGreaterThan(0); + expect(Number(job.conversionDurationSeconds)).toBeGreaterThanOrEqual(0); + + const stat = await fs.stat(outputPath(config, uuid, 'webp')); + expect(stat.size).toBeGreaterThan(0); + }); + + it('passes the job quality through to the converter, shrinking output for a low quality value', async () => { + const lowUuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; + const defaultUuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; + + const noisyBuffer = await sharp({ + create: { + width: 256, + height: 256, + channels: 3, + noise: { type: 'gaussian', mean: 128, sigma: 40 }, + }, + }) + .png() + .toBuffer(); + + for (const uuid of [lowUuid, defaultUuid]) { + await fs.writeFile(uploadPath(config, uuid, 'png'), noisyBuffer); + } + + await createJob(prisma, { + uuid: lowUuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'jpg', + originalFilename: 'noisy.png', + inputPath: `${lowUuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: noisyBuffer.length, + expiresAt: new Date(Date.now() + 3600 * 1000), + quality: 5, + }); + await createJob(prisma, { + uuid: defaultUuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'jpg', + originalFilename: 'noisy.png', + inputPath: `${defaultUuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: noisyBuffer.length, + expiresAt: new Date(Date.now() + 3600 * 1000), + }); + + await processPendingJobs(prisma, { ...config, workerConcurrency: 2 }); + + const lowJob = await getJobByUuid(prisma, lowUuid); + const defaultJob = await getJobByUuid(prisma, defaultUuid); + expect(lowJob.status).toBe('done'); + expect(defaultJob.status).toBe('done'); + expect(lowJob.outputSizeBytes).toBeLessThan(defaultJob.outputSizeBytes); + }); + + it('marks a job failed with a safe message and a detailed log when the converter throws', async () => { + const uuid = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'; + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'missing.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() + 3600 * 1000), + }); + // Note: input file is intentionally never written, so sharp will throw ENOENT. + + await processPendingJobs(prisma, config); + + const job = await getJobByUuid(prisma, uuid); + expect(job.status).toBe('failed'); + expect(job.errorMessage).toBe('Conversion failed, please try again.'); + + const errorLog = await getJobErrorLog(prisma, job.id); + expect(errorLog).toMatch(/input file is missing/i); + }); + + it('only picks up as many jobs as workerConcurrency allows', async () => { + await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp'); + await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp'); + await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp'); + + const limitedConfig = { ...config, workerConcurrency: 2 }; + const processedCount = await processPendingJobs(prisma, limitedConfig); + + expect(processedCount).toBe(2); + }); +}); +``` + +- [ ] **Step 9: Update `test/cleanup.test.js`** + +Replace the full file content of `test/cleanup.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { getPrismaClient, closePrismaClient } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; +import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; +import { createJob, markDone, getJobByUuid } from '../src/jobs/jobRepository.js'; +import { runCleanup } from '../src/cleanup.js'; + +let prisma; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) }; + await ensureStorageDirs(config); + prisma = getPrismaClient(config); +}); + +afterAll(async () => { + await closePrismaClient(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await prisma.conversionJob.deleteMany(); +}); + +describe('runCleanup', () => { + it('marks an expired done job cleaned and removes its input and output files, keeping the row', async () => { + const uuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; + await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes'); + await fs.writeFile(outputPath(config, uuid, 'webp'), 'output bytes'); + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() - 1000), + }); + const created = await getJobByUuid(prisma, uuid); + await markDone(prisma, created.id, { + outputPath: `${uuid}.webp`, + outputMimeType: 'image/webp', + outputSizeBytes: 12, + conversionDurationSeconds: 0.5, + }); + + const cleanedCount = await runCleanup(prisma, config); + + expect(cleanedCount).toBe(1); + const job = await getJobByUuid(prisma, uuid); + expect(job).not.toBeNull(); + expect(job.cleanedAt).not.toBeNull(); + await expect(fs.stat(uploadPath(config, uuid, 'png'))).rejects.toThrow(); + await expect(fs.stat(outputPath(config, uuid, 'webp'))).rejects.toThrow(); + }); + + it('marks an expired pending job (with no output file) cleaned without throwing', async () => { + const uuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; + await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes'); + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() - 1000), + }); + + const cleanedCount = await runCleanup(prisma, config); + + expect(cleanedCount).toBe(1); + const job = await getJobByUuid(prisma, uuid); + expect(job).not.toBeNull(); + expect(job.cleanedAt).not.toBeNull(); + }); + + it('leaves non-expired jobs untouched', async () => { + const uuid = '12121212-1212-4212-8212-121212121212'; + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() + 3600 * 1000), + }); + + const cleanedCount = await runCleanup(prisma, config); + + expect(cleanedCount).toBe(0); + const job = await getJobByUuid(prisma, uuid); + expect(job).not.toBeNull(); + expect(job.cleanedAt).toBeNull(); + }); + + it('does not re-process an already cleaned expired job', async () => { + const uuid = '13131313-1313-4313-8313-131313131313'; + await createJob(prisma, { + uuid, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${uuid}.png`, + inputMimeType: 'image/png', + inputSizeBytes: 11, + expiresAt: new Date(Date.now() - 1000), + }); + + const firstRun = await runCleanup(prisma, config); + const secondRun = await runCleanup(prisma, config); + + expect(firstRun).toBe(1); + expect(secondRun).toBe(0); + }); +}); +``` + +- [ ] **Step 10: Run the full test suite** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` +Expected: all tests pass except the two pre-existing documented failures (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related — verify against `main` if unsure whether a failure is new). + +- [ ] **Step 11: Commit** + +```bash +git add src/app.js src/server.js src/worker.js src/cleanup.js test/api/jobs.test.js test/api/jobStatus.test.js test/api/endToEnd.test.js test/worker.test.js test/cleanup.test.js +git commit -m "Wire the app, worker, and cleanup entry points onto Prisma Client" +``` + +--- + +### Task 6: Remove the mariadb dependency and the old schema file + +**Files:** +- Modify: `package.json` +- Delete: `db/schema.sql` + +**Interfaces:** none (cleanup only — no code depends on `mariadb` or `db/schema.sql` after Task 5). + +- [ ] **Step 1: Delete the old schema file** + +Run: `git rm db/schema.sql` + +- [ ] **Step 2: Remove the `mariadb` dependency** + +Edit `package.json`, removing the `"mariadb"` line from `dependencies` (keep the rest of the dependency list and its alphabetical ordering intact). + +- [ ] **Step 3: Update the lockfile** + +Run: `npm install` +Expected: `package-lock.json` updates to remove `mariadb` and its transitive dependencies; no other dependency versions change. + +- [ ] **Step 4: Run the full test suite** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` +Expected: same result as Task 5 Step 10 — all pass except the two pre-existing documented failures. + +- [ ] **Step 5: Commit** + +```bash +git add package.json package-lock.json db/schema.sql +git commit -m "Remove the mariadb dependency and the superseded schema.sql" +``` + +--- + +### Task 7: Final verification + +**Files:** none (verification only). + +- [ ] **Step 1: Run the full test suite one more time** + +Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run` +Expected: identical results to Task 6 Step 4. + +- [ ] **Step 2: Confirm the frontend build is unaffected** + +Run: `npm run build` +Expected: succeeds, unrelated to the DB layer change. + +- [ ] **Step 3: Confirm `prisma generate` runs cleanly on a fresh install** + +Run: `rm -rf node_modules && npm install` +Expected: `postinstall` runs `prisma generate` automatically as part of `npm install`, with no errors. + +No commit needed for this task if all checks pass — it is verification only.