Files
convert/docs/superpowers/plans/2026-07-31-prisma-orm-migrations.md
anthonyandClaude Sonnet 5 852f7bc8bc Document a third accepted Prisma DDL exception (storage engine)
Task 3 found Prisma's generated migration SQL never emits an
explicit ENGINE=... clause, same root cause as the already-accepted
collation exception (no schema-level attribute exists). No practical
impact: the baseline never executes this SQL against the real DB,
and any fresh deploy still gets InnoDB from the server default.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 01:17:15 +02:00

1714 lines
62 KiB
Markdown

# Prisma ORM and DB Migrations Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the hand-rolled `mariadb` pool and raw-SQL repository with Prisma Client for data access, and introduce Prisma Migrate (with a no-data-loss baseline) as the schema migration system for `conversion_jobs`.
**Architecture:** `prisma/schema.prisma` becomes the single source of truth for the `conversion_jobs` table (replacing `db/schema.sql`), with `prisma/migrations/` as its version history. `src/db.js` swaps its exported `mariadb` pool for a singleton `PrismaClient`. `src/jobs/jobRepository.js` is rewritten onto Prisma's model API, keeping the same exported function names and `(client, ...)` argument shape so `src/app.js`, `src/worker.js`, and `src/cleanup.js` only need a `pool``prisma` rename, not a logic change.
**Tech Stack:** Prisma ORM (`prisma`, `@prisma/client`) against the existing MariaDB instance via Prisma's `mysql` provider. Node.js ESM, Vitest, Supertest — all unchanged.
## Global Constraints
- `prisma/schema.prisma` must reproduce `db/schema.sql` column-for-column, so the initial migration is a no-op against databases that already have the table (spec: "Schema"). Three documented, accepted exceptions where Prisma's MySQL provider has no schema-level equivalent (verified empirically against the real DB's `SHOW CREATE TABLE conversion_jobs`, and confirmed acceptable by the user on 2026-07-31): (1) table collation — the live MariaDB server defaults to `utf8mb4_uca1400_ai_ci` (a MariaDB-specific collation), but Prisma always emits `utf8mb4_unicode_ci` in generated DDL with no schema attribute to override it; (2) `updatedAt`'s `ON UPDATE CURRENT_TIMESTAMP` — Prisma's `@updatedAt` is implemented client-side only for the `mysql` provider and never emits a DB-level `ON UPDATE` clause; (3) storage engine — Prisma's generated DDL never emits an explicit `ENGINE=...` clause (no schema-level attribute controls it), relying on the server default, which is InnoDB on effectively every real MySQL/MariaDB install including this project's (confirmed during Task 3 execution on 2026-07-31, same root cause as (1): no schema-level attribute exists). None of the three have practical impact here: the baseline (Task 3) marks the initial migration applied without ever executing its SQL against the real database, and every write to `conversion_jobs` after this migration goes through Prisma Client, which sets `updatedAt` itself; a fresh environment created via `prisma migrate deploy` would still get InnoDB from the server default. All other columns (types, defaults, index/constraint names) must match exactly — see Task 1's schema, which uses `@db.DateTime(0)` and explicit `map:` names for this reason.
- `DATABASE_URL` is never written into `.env` or `.env.local` — it is always computed on demand from the existing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` variables (spec: "Running Prisma CLI locally"). `.env` holds production credentials and must never be loaded for local work (CLAUDE.md).
- The connection string includes `?connection_limit=10`, preserving the current pool's `connectionLimit: 10` (spec: "Running Prisma CLI locally").
- `getJobByUuid`, `findPendingJobs`, and `findExpiredJobs` must never expose `errorLog` — only `getJobErrorLog` does (spec: "Data access layer").
- `src/jobs/jobRepository.js` exported function names and their `(client, ...)` first-argument shape do not change (spec: "Data access layer").
- The local test convention of passing `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` inline (documented in CLAUDE.md) must keep working unchanged after this migration.
- After full implementation, the only acceptable test failures are the two already documented in CLAUDE.md (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related) — no new regressions (spec: "Behavior change to watch for").
---
### Task 1: Add Prisma dependencies and the schema definition
**Files:**
- Modify: `package.json`
- Create: `prisma/schema.prisma`
**Interfaces:**
- Produces: `prisma/schema.prisma` defining model `ConversionJob` (Prisma Client generates typed methods from this in Task 4).
- [ ] **Step 1: Install the Prisma packages**
Run: `npm install @prisma/client`
Expected: `package.json` dependencies gains an `"@prisma/client"` entry; `package-lock.json` updated.
Run: `npm install --save-dev prisma`
Expected: `package.json` devDependencies gains a `"prisma"` entry; `package-lock.json` updated.
- [ ] **Step 2: Create the Prisma schema**
Create `prisma/schema.prisma`:
```prisma
datasource db {
provider = "mysql"
url = env("DATABASE_URL")
}
generator client {
provider = "prisma-client-js"
}
enum JobStatus {
pending
processing
done
failed
}
model ConversionJob {
id Int @id @default(autoincrement()) @db.UnsignedInt
uuid String @unique(map: "uniq_uuid") @db.Char(36)
status JobStatus @default(pending)
family String @db.VarChar(32)
sourceFormat String @map("source_format") @db.VarChar(16)
targetFormat String @map("target_format") @db.VarChar(16)
originalFilename String @map("original_filename") @db.VarChar(255)
inputPath String @map("input_path") @db.VarChar(255)
outputPath String? @map("output_path") @db.VarChar(255)
inputMimeType String @map("input_mime_type") @db.VarChar(128)
outputMimeType String? @map("output_mime_type") @db.VarChar(128)
inputSizeBytes Int @map("input_size_bytes") @db.UnsignedInt
outputSizeBytes Int? @map("output_size_bytes") @db.UnsignedInt
quality Int? @db.UnsignedSmallInt
conversionDurationSeconds Decimal? @map("conversion_duration_seconds") @db.Decimal(10, 3)
errorMessage String? @map("error_message") @db.VarChar(255)
errorLog String? @map("error_log") @db.Text
createdAt DateTime @default(now()) @map("created_at") @db.DateTime(0)
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.DateTime(0)
expiresAt DateTime @map("expires_at") @db.DateTime(0)
cleanedAt DateTime? @map("cleaned_at") @db.DateTime(0)
@@index([status], map: "idx_status")
@@index([expiresAt], map: "idx_expires_at")
@@map("conversion_jobs")
}
```
`@db.DateTime(0)` matches the real table's `datetime` columns (no fractional seconds) — without it Prisma defaults to `DATETIME(3)`/`CURRENT_TIMESTAMP(3)`, which would both mismatch the existing column type and risk an "Invalid default value" error on a fresh `prisma migrate deploy` (MySQL/MariaDB require the `CURRENT_TIMESTAMP(n)` default's precision to match the column's own precision). The `map: "..."` arguments make Prisma reuse the existing constraint/index names (`uniq_uuid`, `idx_status`, `idx_expires_at`) instead of auto-generating new ones. Verified empirically via `prisma migrate diff --from-empty --to-schema-datamodel` in a scratch directory on 2026-07-31.
- [ ] **Step 3: Add the postinstall script**
Edit `package.json`:
old:
```json
"scripts": {
"start": "node src/server.js",
"worker": "node src/worker.js",
"cleanup": "node src/cleanup.js",
"test": "vitest run",
"build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend"
},
```
new:
```json
"scripts": {
"start": "node src/server.js",
"worker": "node src/worker.js",
"cleanup": "node src/cleanup.js",
"test": "vitest run",
"build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend",
"postinstall": "prisma generate"
},
```
- [ ] **Step 4: Validate the schema**
Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma validate`
Expected: `The schema at prisma/schema.prisma is valid 🚀` (the connection string here is a placeholder only used to satisfy `env("DATABASE_URL")` resolution — `validate` does not connect to a database).
- [ ] **Step 5: Generate the client**
Run: `DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma generate`
Expected: output confirms the client was generated, e.g. `Generated Prisma Client ... to ./node_modules/@prisma/client`.
- [ ] **Step 6: Commit**
```bash
git add package.json package-lock.json prisma/schema.prisma
git commit -m "Add Prisma schema and client generation"
```
---
### Task 2: Add `buildDatabaseUrl` and the `printDatabaseUrl` CLI helper
**Files:**
- Modify: `src/db.js`
- Modify: `test/db.test.js`
- Create: `scripts/printDatabaseUrl.js`
**Interfaces:**
- Consumes: `config.db = { host, user, password, database }` shape from `loadConfig()` (`src/config.js`, unchanged).
- Produces: `buildDatabaseUrl(config): string` — used by Task 4's `getPrismaClient` and by `scripts/printDatabaseUrl.js`.
This task is purely additive: `getPool`/`closePool` and the existing `mariadb` pool are untouched, so the whole app and test suite keep working exactly as today after this task.
- [ ] **Step 1: Write the failing tests**
Edit `test/db.test.js`, adding a new `describe` block above the existing `describe('getPool', ...)`:
```js
import { describe, it, expect, afterAll } from 'vitest';
import { getPool, closePool, buildDatabaseUrl } from '../src/db.js';
import { loadConfig } from '../src/config.js';
describe('buildDatabaseUrl', () => {
it('builds a mysql connection string from db config', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' },
});
expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10');
});
it('percent-encodes special characters in user and password', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' },
});
expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10');
});
});
describe('getPool', () => {
afterAll(async () => {
await closePool();
});
it('returns a working pool that can run a query', async () => {
const config = loadConfig();
const pool = getPool(config);
const rows = await pool.query('SELECT 1 AS value');
expect(Number(rows[0].value)).toBe(1);
});
it('returns the same pool instance on repeated calls', () => {
const config = loadConfig();
const poolA = getPool(config);
const poolB = getPool(config);
expect(poolA).toBe(poolB);
});
});
```
- [ ] **Step 2: Run the test file to verify the new tests fail**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js`
Expected: FAIL — `buildDatabaseUrl is not a function` (or import error), the two new tests fail; the two pre-existing `getPool` tests still pass.
- [ ] **Step 3: Implement `buildDatabaseUrl`**
Edit `src/db.js`:
old:
```js
import mariadb from 'mariadb';
let pool;
export function getPool(config) {
```
new:
```js
import mariadb from 'mariadb';
let pool;
export function buildDatabaseUrl(config) {
const { host, user, password, database } = config.db;
return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`;
}
export function getPool(config) {
```
- [ ] **Step 4: Run the test file to verify it passes**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js`
Expected: PASS — all 4 tests green.
- [ ] **Step 5: Create the CLI helper**
Create `scripts/printDatabaseUrl.js`:
```js
import { loadConfig } from '../src/config.js';
import { buildDatabaseUrl } from '../src/db.js';
console.log(buildDatabaseUrl(loadConfig()));
```
- [ ] **Step 6: Verify the helper manually**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js`
Expected stdout: `mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10`
- [ ] **Step 7: Commit**
```bash
git add src/db.js test/db.test.js scripts/printDatabaseUrl.js
git commit -m "Add buildDatabaseUrl and a CLI helper to print it"
```
---
### Task 3: Baseline Prisma Migrate against the existing database
**Files:**
- Create: `prisma/migrations/migration_lock.toml`
- Create: `prisma/migrations/0_init/migration.sql`
- Modify: `CLAUDE.md`
**Interfaces:**
- Consumes: `scripts/printDatabaseUrl.js` (Task 2).
- Produces: a migration history that Prisma considers already applied on the local dev DB, with `conversion_jobs` and its rows untouched.
This task only touches Prisma's own bookkeeping (`prisma/migrations/`, and the `_prisma_migrations` tracking table it creates in the target database) — it must not run any SQL against `conversion_jobs` itself.
- [ ] **Step 1: Record the current row count as a safety check**
Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"`
Note the number returned — it must be identical after Step 5.
- [ ] **Step 2: Confirm the diff flag name for this installed Prisma version**
Run: `npx prisma migrate diff --help`
Expected: the help text lists a flag for rendering a schema file's data model as the diff target without connecting to a live database — documented in Prisma's migration-diffing workflow as `--to-schema-datamodel <path>`. Confirm this exact flag name in the output before proceeding; if this installed version names it differently, substitute the flag `--help` reports in Step 4.
- [ ] **Step 3: Create the migration folder and lock file**
Run: `mkdir -p prisma/migrations/0_init`
Create `prisma/migrations/migration_lock.toml`:
```toml
# Please do not edit this file manually
# It should be added in your version-control system (e.g., Git)
provider = "mysql"
```
- [ ] **Step 4: Generate the migration SQL from the schema file (no database connection needed for this)**
Run:
```bash
npx prisma migrate diff \
--from-empty \
--to-schema-datamodel prisma/schema.prisma \
--script > prisma/migrations/0_init/migration.sql
```
Expected: `prisma/migrations/0_init/migration.sql` is created containing a single `CREATE TABLE conversion_jobs (...)` statement (plus its indexes) — no `DROP` or `ALTER` statements.
- [ ] **Step 5: Read the generated SQL and confirm it matches `db/schema.sql`**
Read `prisma/migrations/0_init/migration.sql` and compare column names, types, and constraints against `db/schema.sql`. They must describe the same table. If Prisma rendered a type differently than expected (e.g. a default value clause), stop and reconcile `prisma/schema.prisma` in Task 1 before continuing — do not hand-edit the generated SQL, regenerate it after fixing the schema.
- [ ] **Step 6: Mark the migration as applied (bookkeeping only, does not run the SQL)**
Run:
```bash
DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \
npx prisma migrate resolve --applied 0_init
```
Expected: `Migration 0_init marked as applied.`
- [ ] **Step 7: Verify migration status and that data is untouched**
Run:
```bash
DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \
npx prisma migrate status
```
Expected: `Database schema is up to date!`
Run: `mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"`
Expected: the same row count recorded in Step 1.
- [ ] **Step 8: Document the migration workflow in CLAUDE.md**
Edit `CLAUDE.md`, adding a new section after the existing "Local environment / running tests" section:
```markdown
## Database schema & migrations
- `prisma/schema.prisma` is the source of truth for the `conversion_jobs` schema; `prisma/migrations/` is its version history. `db/schema.sql` no longer exists.
- Prisma CLI commands (`prisma migrate dev`, `prisma migrate deploy`, `prisma generate`) need `DATABASE_URL` in their own process environment, separate from the app. Compute it from the same `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` values used for local tests, via `node scripts/printDatabaseUrl.js` — never write `DATABASE_URL` into `.env` or `.env.local`.
- To create a new migration locally after editing `prisma/schema.prisma`:
```
DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage \
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate dev --name <description>
```
- To apply pending migrations in production: with the real `DB_*` values loaded from `.env`, run `DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate deploy` before starting the app.
```
- [ ] **Step 9: Commit**
```bash
git add prisma/migrations CLAUDE.md
git commit -m "Baseline Prisma Migrate against the existing conversion_jobs table"
```
---
### Task 4: Rewrite the data access layer onto Prisma Client
**Files:**
- Modify: `src/db.js`
- Modify: `src/jobs/jobRepository.js`
- Modify: `test/db.test.js`
- Modify: `test/jobs/jobRepository.test.js`
**Interfaces:**
- Produces: `getPrismaClient(config): PrismaClient`, `closePrismaClient(): Promise<void>` (replacing `getPool`/`closePool`); `createJob`, `getJobByUuid`, `getJobErrorLog`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` — all still `(prisma, ...)` first-argument, same names, now backed by Prisma.
After this task, `test/db.test.js` and `test/jobs/jobRepository.test.js` pass in isolation. Every other file that still imports `getPool`/`closePool` from `src/db.js` (`src/app.js`, `src/server.js`, `src/worker.js`, `src/cleanup.js`, and 5 remaining test files) will fail to import until Task 5 lands — this is expected and addressed next.
- [ ] **Step 1: Rewrite `src/db.js`**
Replace the full file content of `src/db.js`:
```js
import { PrismaClient } from '@prisma/client';
let prisma;
export function buildDatabaseUrl(config) {
const { host, user, password, database } = config.db;
return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`;
}
export function getPrismaClient(config) {
if (!prisma) {
process.env.DATABASE_URL = buildDatabaseUrl(config);
prisma = new PrismaClient();
}
return prisma;
}
export async function closePrismaClient() {
if (prisma) {
await prisma.$disconnect();
prisma = undefined;
}
}
```
- [ ] **Step 2: Rewrite `src/jobs/jobRepository.js`**
Replace the full file content of `src/jobs/jobRepository.js`:
```js
const jobSelect = {
id: true,
uuid: true,
status: true,
family: true,
sourceFormat: true,
targetFormat: true,
originalFilename: true,
inputPath: true,
outputPath: true,
inputMimeType: true,
outputMimeType: true,
inputSizeBytes: true,
outputSizeBytes: true,
quality: true,
conversionDurationSeconds: true,
errorMessage: true,
createdAt: true,
updatedAt: true,
expiresAt: true,
cleanedAt: true,
};
export async function createJob(prisma, job) {
await prisma.conversionJob.create({
data: {
uuid: job.uuid,
family: job.family,
sourceFormat: job.sourceFormat,
targetFormat: job.targetFormat,
originalFilename: job.originalFilename,
inputPath: job.inputPath,
inputMimeType: job.inputMimeType,
inputSizeBytes: job.inputSizeBytes,
expiresAt: job.expiresAt,
quality: job.quality ?? null,
},
});
}
export async function getJobByUuid(prisma, uuid) {
return prisma.conversionJob.findUnique({ where: { uuid }, select: jobSelect });
}
export async function getJobErrorLog(prisma, id) {
const job = await prisma.conversionJob.findUnique({ where: { id }, select: { errorLog: true } });
return job?.errorLog ?? null;
}
export async function markProcessing(prisma, id) {
await prisma.conversionJob.update({ where: { id }, data: { status: 'processing' } });
}
export async function markDone(prisma, id, { outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }) {
await prisma.conversionJob.update({
where: { id },
data: { status: 'done', outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds },
});
}
export async function markFailed(prisma, id, { errorMessage, errorLog }) {
await prisma.conversionJob.update({
where: { id },
data: { status: 'failed', errorMessage, errorLog },
});
}
export async function findPendingJobs(prisma, limit) {
return prisma.conversionJob.findMany({
where: { status: 'pending' },
orderBy: { createdAt: 'asc' },
take: limit,
select: jobSelect,
});
}
export async function findExpiredJobs(prisma) {
return prisma.conversionJob.findMany({
where: { expiresAt: { lt: new Date() }, cleanedAt: null },
select: jobSelect,
});
}
export async function markCleaned(prisma, id) {
await prisma.conversionJob.update({ where: { id }, data: { cleanedAt: new Date() } });
}
```
- [ ] **Step 3: Rewrite `test/db.test.js`**
Replace the full file content of `test/db.test.js`:
```js
import { describe, it, expect, afterAll } from 'vitest';
import { getPrismaClient, closePrismaClient, buildDatabaseUrl } from '../src/db.js';
import { loadConfig } from '../src/config.js';
describe('buildDatabaseUrl', () => {
it('builds a mysql connection string from db config', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' },
});
expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10');
});
it('percent-encodes special characters in user and password', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' },
});
expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10');
});
});
describe('getPrismaClient', () => {
afterAll(async () => {
await closePrismaClient();
});
it('returns a working client that can run a query', async () => {
const config = loadConfig();
const prisma = getPrismaClient(config);
const rows = await prisma.$queryRaw`SELECT 1 AS value`;
expect(Number(rows[0].value)).toBe(1);
});
it('returns the same client instance on repeated calls', () => {
const config = loadConfig();
const clientA = getPrismaClient(config);
const clientB = getPrismaClient(config);
expect(clientA).toBe(clientB);
});
});
```
- [ ] **Step 4: Rewrite `test/jobs/jobRepository.test.js`**
Replace the full file content of `test/jobs/jobRepository.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import {
createJob,
getJobByUuid,
getJobErrorLog,
markProcessing,
markDone,
markFailed,
findPendingJobs,
findExpiredJobs,
markCleaned,
} from '../../src/jobs/jobRepository.js';
let prisma;
beforeAll(() => {
prisma = getPrismaClient(loadConfig());
});
afterAll(async () => {
await closePrismaClient();
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
function baseJob(overrides = {}) {
return {
uuid: overrides.uuid ?? '11111111-1111-4111-8111-111111111111',
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${overrides.uuid ?? '11111111-1111-4111-8111-111111111111'}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 1024,
expiresAt: new Date(Date.now() + 3600 * 1000),
...overrides,
};
}
describe('jobRepository', () => {
it('creates and retrieves a pending job', async () => {
await createJob(prisma, baseJob());
const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(job.id).toEqual(expect.any(Number));
expect(job.uuid).toBe('11111111-1111-4111-8111-111111111111');
expect(job.status).toBe('pending');
expect(job.family).toBe('image');
expect(job.sourceFormat).toBe('png');
expect(job.targetFormat).toBe('webp');
expect(job.originalFilename).toBe('photo.png');
expect(job.inputMimeType).toBe('image/png');
expect(job.inputSizeBytes).toBe(1024);
expect(job.outputPath).toBeNull();
expect(job.outputSizeBytes).toBeNull();
expect(job.conversionDurationSeconds).toBeNull();
expect(job.errorMessage).toBeNull();
expect(job.cleanedAt).toBeNull();
expect(job.quality).toBeNull();
});
it('stores and retrieves a numeric quality value', async () => {
await createJob(prisma, baseJob({ uuid: '66666666-6666-4666-8666-666666666666', quality: 82 }));
const job = await getJobByUuid(prisma, '66666666-6666-4666-8666-666666666666');
expect(job.quality).toBe(82);
});
it('returns null for an unknown uuid', async () => {
const job = await getJobByUuid(prisma, '22222222-2222-4222-8222-222222222222');
expect(job).toBeNull();
});
it('transitions a job through processing to done', async () => {
await createJob(prisma, baseJob());
await markProcessing(prisma, (await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111')).id);
const processing = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(processing.status).toBe('processing');
await markDone(prisma, processing.id, {
outputPath: '11111111-1111-4111-8111-111111111111.webp',
outputMimeType: 'image/webp',
outputSizeBytes: 2048,
conversionDurationSeconds: 1.5,
});
const done = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(done.status).toBe('done');
expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp');
expect(done.outputMimeType).toBe('image/webp');
expect(done.outputSizeBytes).toBe(2048);
expect(Number(done.conversionDurationSeconds)).toBe(1.5);
});
it('marks a job failed with a short message and a separate detailed log', async () => {
await createJob(prisma, baseJob());
const created = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
await markFailed(prisma, created.id, {
errorMessage: 'Conversion failed, please try again',
errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)',
});
const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(job.status).toBe('failed');
expect(job.errorMessage).toBe('Conversion failed, please try again');
expect(job.errorLog).toBeUndefined();
const errorLog = await getJobErrorLog(prisma, created.id);
expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)');
});
it('finds pending jobs oldest first, up to a limit', async () => {
await createJob(prisma, baseJob({ uuid: '33333333-3333-4333-8333-333333333333' }));
await createJob(prisma, baseJob({ uuid: '44444444-4444-4444-8444-444444444444' }));
await createJob(prisma, baseJob({ uuid: '55555555-5555-4555-8555-555555555555' }));
const jobs = await findPendingJobs(prisma, 2);
expect(jobs).toHaveLength(2);
expect(jobs[0].uuid).toBe('33333333-3333-4333-8333-333333333333');
expect(jobs[1].uuid).toBe('44444444-4444-4444-8444-444444444444');
});
it('finds expired jobs and allows marking them cleaned without deleting the row', async () => {
await createJob(prisma, baseJob({ expiresAt: new Date(Date.now() - 1000) }));
const expired = await findExpiredJobs(prisma);
expect(expired).toHaveLength(1);
expect(expired[0].uuid).toBe('11111111-1111-4111-8111-111111111111');
await markCleaned(prisma, expired[0].id);
const stillPresent = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(stillPresent).not.toBeNull();
expect(stillPresent.cleanedAt).not.toBeNull();
const afterCleaning = await findExpiredJobs(prisma);
expect(afterCleaning).toHaveLength(0);
});
});
```
- [ ] **Step 5: Run the two updated test files**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js test/jobs/jobRepository.test.js`
Expected: PASS — all tests green. (Running the full suite at this point will show import errors in other files; that's expected until Task 5.)
- [ ] **Step 6: Commit**
```bash
git add src/db.js src/jobs/jobRepository.js test/db.test.js test/jobs/jobRepository.test.js
git commit -m "Rewrite the data access layer onto Prisma Client"
```
---
### Task 5: Wire up the remaining call sites and tests
**Files:**
- Modify: `src/app.js`
- Modify: `src/server.js`
- Modify: `src/worker.js`
- Modify: `src/cleanup.js`
- Modify: `test/api/jobs.test.js`
- Modify: `test/api/jobStatus.test.js`
- Modify: `test/api/endToEnd.test.js`
- Modify: `test/worker.test.js`
- Modify: `test/cleanup.test.js`
**Interfaces:**
- Consumes: `getPrismaClient`, `closePrismaClient` (Task 4, `src/db.js`); `createJob`, `getJobByUuid`, `markProcessing`, `markDone`, `markFailed`, `findPendingJobs`, `findExpiredJobs`, `markCleaned` (Task 4, `src/jobs/jobRepository.js`).
- Produces: `createApp(config, prisma)` (renamed second parameter, same behavior).
After this task the full test suite should pass except the two pre-existing documented failures.
- [ ] **Step 1: Update `src/app.js`**
Edit `src/app.js`:
old:
```js
export function createApp(config, pool) {
```
new:
```js
export function createApp(config, prisma) {
```
old:
```js
const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000);
await createJob(pool, {
```
new:
```js
const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000);
await createJob(prisma, {
```
old:
```js
app.get('/api/jobs/:id', async (req, res) => {
const job = await getJobByUuid(pool, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
```
new:
```js
app.get('/api/jobs/:id', async (req, res) => {
const job = await getJobByUuid(prisma, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
```
old:
```js
app.get('/api/jobs/:id/download', async (req, res) => {
const job = await getJobByUuid(pool, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
if (job.status !== 'done') {
```
new:
```js
app.get('/api/jobs/:id/download', async (req, res) => {
const job = await getJobByUuid(prisma, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
if (job.status !== 'done') {
```
- [ ] **Step 2: Update `src/server.js`**
Replace the full file content of `src/server.js`:
```js
import { loadConfig } from './config.js';
import { getPrismaClient } from './db.js';
import { ensureStorageDirs } from './storage.js';
import { createApp } from './app.js';
async function main() {
const config = loadConfig();
await ensureStorageDirs(config);
const prisma = getPrismaClient(config);
const app = createApp(config, prisma);
app.listen(config.port, () => {
console.log(`File converter API listening on port ${config.port}`);
});
}
main().catch((error) => {
console.error('Failed to start server:', error);
process.exit(1);
});
```
- [ ] **Step 3: Update `src/worker.js`**
Replace the full file content of `src/worker.js`:
```js
import { pathToFileURL } from 'node:url';
import fs from 'node:fs/promises';
import { loadConfig } from './config.js';
import { getPrismaClient } from './db.js';
import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js';
import { outputMimeType } from './mime.js';
import { resolve as resolveConverter } from './converters/registry.js';
import { registerImageConverters } from './converters/image.js';
import { registerImageToPdfConverter } from './converters/imageToPdf.js';
import { registerDocumentConverters } from './converters/document.js';
import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js';
const JOB_TIMEOUT_MS = 60000;
function withTimeout(promise, ms) {
let timeoutId;
const timeout = new Promise((_, reject) => {
timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms);
});
return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId));
}
async function processJob(prisma, config, job) {
await markProcessing(prisma, job.id);
const inputFilePath = uploadPath(config, job.uuid, job.sourceFormat);
const outputFilePath = outputPath(config, job.uuid, job.targetFormat);
try {
const entry = resolveConverter(job.sourceFormat, job.targetFormat);
if (!entry) {
throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`);
}
const startedAt = Date.now();
await withTimeout(entry.convert(inputFilePath, outputFilePath, { quality: job.quality }), JOB_TIMEOUT_MS);
const conversionDurationSeconds = (Date.now() - startedAt) / 1000;
const { size: outputSizeBytes } = await fs.stat(outputFilePath);
await markDone(prisma, job.id, {
outputPath: `${job.uuid}.${job.targetFormat}`,
outputMimeType: outputMimeType(job.targetFormat),
outputSizeBytes,
conversionDurationSeconds,
});
} catch (error) {
await deleteIfExists(outputFilePath);
await markFailed(prisma, job.id, {
errorMessage: 'Conversion failed, please try again.',
errorLog: error.stack ?? String(error),
});
}
}
export async function processPendingJobs(prisma, config) {
const jobs = await findPendingJobs(prisma, config.workerConcurrency);
await Promise.all(jobs.map((job) => processJob(prisma, config, job)));
return jobs.length;
}
export function startWorker(prisma, config) {
const interval = setInterval(() => {
processPendingJobs(prisma, config).catch((error) => {
console.error('Error while processing pending jobs:', error);
});
}, config.workerPollIntervalMs);
return () => clearInterval(interval);
}
async function main() {
const config = loadConfig();
await ensureStorageDirs(config);
const prisma = getPrismaClient(config);
registerImageConverters();
registerImageToPdfConverter();
registerDocumentConverters();
startWorker(prisma, config);
console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`);
}
main();
```
- [ ] **Step 4: Update `src/cleanup.js`**
Replace the full file content of `src/cleanup.js`:
```js
import { pathToFileURL } from 'node:url';
import { loadConfig } from './config.js';
import { getPrismaClient, closePrismaClient } from './db.js';
import { uploadPath, outputPath, deleteIfExists } from './storage.js';
import { findExpiredJobs, markCleaned } from './jobs/jobRepository.js';
export async function runCleanup(prisma, config) {
const expiredJobs = await findExpiredJobs(prisma);
for (const job of expiredJobs) {
await deleteIfExists(uploadPath(config, job.uuid, job.sourceFormat));
await deleteIfExists(outputPath(config, job.uuid, job.targetFormat));
await markCleaned(prisma, job.id);
}
return expiredJobs.length;
}
async function main() {
const config = loadConfig();
const prisma = getPrismaClient(config);
const deletedCount = await runCleanup(prisma, config);
console.log(`Cleanup: removed ${deletedCount} expired job(s).`);
await closePrismaClient();
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
main();
}
```
- [ ] **Step 5: Update `test/api/jobs.test.js`**
Replace the full file content of `test/api/jobs.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { getJobByUuid } from '../../src/jobs/jobRepository.js';
import { ensureStorageDirs } from '../../src/storage.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
describe('GET /api/formats', () => {
it('lists valid target formats for a known source format', async () => {
const response = await request(app).get('/api/formats').query({ source: 'png' });
expect(response.status).toBe(200);
expect(response.body.targets).toContain('webp');
expect(response.body.targets).toContain('pdf');
});
it('returns an empty list for an unknown source format', async () => {
const response = await request(app).get('/api/formats').query({ source: 'made-up' });
expect(response.status).toBe(200);
expect(response.body.targets).toEqual([]);
});
});
describe('POST /api/jobs', () => {
it('creates a pending job for a valid image upload', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs).toHaveLength(1);
expect(response.body.jobs[0].status).toBe('pending');
expect(response.body.jobs[0].file).toBe('photo.png');
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
expect(job.status).toBe('pending');
expect(job.sourceFormat).toBe('png');
expect(job.targetFormat).toBe('webp');
expect(job.originalFilename).toBe('photo.png');
expect(job.inputMimeType).toBe('image/png');
expect(job.inputSizeBytes).toBeGreaterThan(0);
expect(job.quality).toBeNull();
});
it('creates a pending job with a quality level for a supported format', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([45]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].status).toBe('pending');
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
expect(job.quality).toBe(45);
});
it('rejects an out-of-range quality without failing the whole batch', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([500]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
});
it('rejects a quality value for a format that does not support one', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['gif']))
.field('qualities', JSON.stringify([50]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
});
it('returns 400 when qualities length does not match the number of files', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([10, 20]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(400);
});
it('rejects a file whose content does not match its extension, without failing the whole batch', async () => {
const fakePath = path.join(config.storageDir, 'fake.png');
await fs.writeFile(fakePath, 'not actually a png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fakePath, 'fake.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/does not match/);
await fs.unlink(fakePath);
});
it('rejects an unsupported source/target pair', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['made-up-format']))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/);
});
it('returns 400 when targetFormats length does not match the number of files', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify([]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(400);
});
it('returns 400 when no files are uploaded', async () => {
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify([]));
expect(response.status).toBe(400);
});
});
```
- [ ] **Step 6: Update `test/api/jobStatus.test.js`**
Replace the full file content of `test/api/jobStatus.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { createJob, getJobByUuid, markDone } from '../../src/jobs/jobRepository.js';
import { ensureStorageDirs, outputPath } from '../../src/storage.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
function baseJob(uuid) {
return {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'holiday photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
};
}
describe('GET /api/jobs/:id', () => {
it('returns job status without the error log field', async () => {
const uuid = '66666666-6666-4666-8666-666666666666';
await createJob(prisma, baseJob(uuid));
const response = await request(app).get(`/api/jobs/${uuid}`);
expect(response.status).toBe(200);
expect(response.body.id).toBe(uuid);
expect(response.body.status).toBe('pending');
expect(response.body.originalFilename).toBe('holiday photo.png');
expect(response.body.errorLog).toBeUndefined();
});
it('returns 404 for an unknown id', async () => {
const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000');
expect(response.status).toBe(404);
});
});
describe('GET /api/jobs/:id/download', () => {
it('streams the converted file with correct headers once done', async () => {
const uuid = '77777777-7777-4777-8777-777777777777';
await createJob(prisma, baseJob(uuid));
const filePath = outputPath(config, uuid, 'webp');
await fs.writeFile(filePath, Buffer.from('fake webp bytes'));
const created = await getJobByUuid(prisma, uuid);
await markDone(prisma, created.id, {
outputPath: `${uuid}.webp`,
outputMimeType: 'image/webp',
outputSizeBytes: 16,
conversionDurationSeconds: 0.2,
});
const response = await request(app).get(`/api/jobs/${uuid}/download`);
expect(response.status).toBe(200);
expect(response.headers['content-type']).toBe('image/webp');
expect(response.headers['content-disposition']).toContain('holiday photo.webp');
expect(response.body ?? response.text).toBeDefined();
});
it('returns 409 when the job is not done yet', async () => {
const uuid = '88888888-8888-4888-8888-888888888888';
await createJob(prisma, baseJob(uuid));
const response = await request(app).get(`/api/jobs/${uuid}/download`);
expect(response.status).toBe(409);
});
it('returns 404 for an unknown id', async () => {
const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download');
expect(response.status).toBe(404);
});
});
describe('unexpected server errors', () => {
it('returns a generic 500 without leaking internal error details', async () => {
const originalFindUnique = prisma.conversionJob.findUnique.bind(prisma.conversionJob);
prisma.conversionJob.findUnique = () => Promise.reject(new Error('connection reset by peer'));
const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999');
prisma.conversionJob.findUnique = originalFindUnique;
expect(response.status).toBe(500);
expect(response.body.error).toBe('Internal server error');
expect(JSON.stringify(response.body)).not.toContain('connection reset by peer');
});
});
```
- [ ] **Step 7: Update `test/api/endToEnd.test.js`**
Replace the full file content of `test/api/endToEnd.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { Document, Paragraph, TextRun, Packer } from 'docx';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { ensureStorageDirs } from '../../src/storage.js';
import { processPendingJobs } from '../../src/worker.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
async function waitForDone(id, maxAttempts = 20) {
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
await processPendingJobs(prisma, config);
const response = await request(app).get(`/api/jobs/${id}`);
if (response.body.status === 'done' || response.body.status === 'failed') {
return response.body;
}
}
throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`);
}
describe('end-to-end: image family', () => {
it('uploads a PNG, converts it to WebP, and downloads the result', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const uploadResponse = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fixturePath, 'photo.png');
const { id } = uploadResponse.body.jobs[0];
const finalStatus = await waitForDone(id);
expect(finalStatus.status).toBe('done');
const downloadResponse = await request(app).get(`/api/jobs/${id}/download`);
expect(downloadResponse.status).toBe(200);
expect(downloadResponse.headers['content-type']).toBe('image/webp');
expect(downloadResponse.headers['content-disposition']).toContain('photo.webp');
}, 20000);
});
describe('end-to-end: document family', () => {
it('uploads a DOCX, converts it to PDF, and downloads the result', async () => {
const doc = new Document({
sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }],
});
const buffer = await Packer.toBuffer(doc);
const docxPath = path.join(config.storageDir, 'e2e-fixture.docx');
await fs.writeFile(docxPath, buffer);
const uploadResponse = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['pdf']))
.attach('files', docxPath, 'report.docx');
const { id } = uploadResponse.body.jobs[0];
const finalStatus = await waitForDone(id);
expect(finalStatus.status).toBe('done');
const downloadResponse = await request(app).get(`/api/jobs/${id}/download`);
expect(downloadResponse.status).toBe(200);
expect(downloadResponse.headers['content-type']).toBe('application/pdf');
expect(downloadResponse.headers['content-disposition']).toContain('report.pdf');
await fs.unlink(docxPath);
}, 20000);
});
```
- [ ] **Step 8: Update `test/worker.test.js`**
Replace the full file content of `test/worker.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import sharp from 'sharp';
import { getPrismaClient, closePrismaClient } from '../src/db.js';
import { loadConfig } from '../src/config.js';
import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js';
import { createJob, getJobByUuid, getJobErrorLog } from '../src/jobs/jobRepository.js';
import { registerImageConverters } from '../src/converters/image.js';
import { processPendingJobs } from '../src/worker.js';
let prisma;
let config;
beforeAll(async () => {
registerImageConverters();
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
async function createPendingImageJob(uuid, sourceFormat, targetFormat) {
const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png');
const inputFilePath = uploadPath(config, uuid, sourceFormat);
await fs.copyFile(fixturePath, inputFilePath);
const { size: inputSizeBytes } = await fs.stat(inputFilePath);
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat,
targetFormat,
originalFilename: `photo.${sourceFormat}`,
inputPath: `${uuid}.${sourceFormat}`,
inputMimeType: 'image/png',
inputSizeBytes,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
}
describe('processPendingJobs', () => {
it('converts a pending image job to done', async () => {
const uuid = '99999999-9999-4999-8999-999999999999';
await createPendingImageJob(uuid, 'png', 'webp');
const processedCount = await processPendingJobs(prisma, config);
expect(processedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job.status).toBe('done');
expect(job.outputPath).toBe(`${uuid}.webp`);
expect(job.outputMimeType).toBe('image/webp');
expect(job.outputSizeBytes).toBeGreaterThan(0);
expect(Number(job.conversionDurationSeconds)).toBeGreaterThanOrEqual(0);
const stat = await fs.stat(outputPath(config, uuid, 'webp'));
expect(stat.size).toBeGreaterThan(0);
});
it('passes the job quality through to the converter, shrinking output for a low quality value', async () => {
const lowUuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee';
const defaultUuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff';
const noisyBuffer = await sharp({
create: {
width: 256,
height: 256,
channels: 3,
noise: { type: 'gaussian', mean: 128, sigma: 40 },
},
})
.png()
.toBuffer();
for (const uuid of [lowUuid, defaultUuid]) {
await fs.writeFile(uploadPath(config, uuid, 'png'), noisyBuffer);
}
await createJob(prisma, {
uuid: lowUuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'jpg',
originalFilename: 'noisy.png',
inputPath: `${lowUuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: noisyBuffer.length,
expiresAt: new Date(Date.now() + 3600 * 1000),
quality: 5,
});
await createJob(prisma, {
uuid: defaultUuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'jpg',
originalFilename: 'noisy.png',
inputPath: `${defaultUuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: noisyBuffer.length,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
await processPendingJobs(prisma, { ...config, workerConcurrency: 2 });
const lowJob = await getJobByUuid(prisma, lowUuid);
const defaultJob = await getJobByUuid(prisma, defaultUuid);
expect(lowJob.status).toBe('done');
expect(defaultJob.status).toBe('done');
expect(lowJob.outputSizeBytes).toBeLessThan(defaultJob.outputSizeBytes);
});
it('marks a job failed with a safe message and a detailed log when the converter throws', async () => {
const uuid = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'missing.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
// Note: input file is intentionally never written, so sharp will throw ENOENT.
await processPendingJobs(prisma, config);
const job = await getJobByUuid(prisma, uuid);
expect(job.status).toBe('failed');
expect(job.errorMessage).toBe('Conversion failed, please try again.');
const errorLog = await getJobErrorLog(prisma, job.id);
expect(errorLog).toMatch(/input file is missing/i);
});
it('only picks up as many jobs as workerConcurrency allows', async () => {
await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp');
await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp');
await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp');
const limitedConfig = { ...config, workerConcurrency: 2 };
const processedCount = await processPendingJobs(prisma, limitedConfig);
expect(processedCount).toBe(2);
});
});
```
- [ ] **Step 9: Update `test/cleanup.test.js`**
Replace the full file content of `test/cleanup.test.js`:
```js
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { getPrismaClient, closePrismaClient } from '../src/db.js';
import { loadConfig } from '../src/config.js';
import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js';
import { createJob, markDone, getJobByUuid } from '../src/jobs/jobRepository.js';
import { runCleanup } from '../src/cleanup.js';
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
describe('runCleanup', () => {
it('marks an expired done job cleaned and removes its input and output files, keeping the row', async () => {
const uuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee';
await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes');
await fs.writeFile(outputPath(config, uuid, 'webp'), 'output bytes');
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const created = await getJobByUuid(prisma, uuid);
await markDone(prisma, created.id, {
outputPath: `${uuid}.webp`,
outputMimeType: 'image/webp',
outputSizeBytes: 12,
conversionDurationSeconds: 0.5,
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).not.toBeNull();
await expect(fs.stat(uploadPath(config, uuid, 'png'))).rejects.toThrow();
await expect(fs.stat(outputPath(config, uuid, 'webp'))).rejects.toThrow();
});
it('marks an expired pending job (with no output file) cleaned without throwing', async () => {
const uuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff';
await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes');
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).not.toBeNull();
});
it('leaves non-expired jobs untouched', async () => {
const uuid = '12121212-1212-4212-8212-121212121212';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(0);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).toBeNull();
});
it('does not re-process an already cleaned expired job', async () => {
const uuid = '13131313-1313-4313-8313-131313131313';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const firstRun = await runCleanup(prisma, config);
const secondRun = await runCleanup(prisma, config);
expect(firstRun).toBe(1);
expect(secondRun).toBe(0);
});
});
```
- [ ] **Step 10: Run the full test suite**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run`
Expected: all tests pass except the two pre-existing documented failures (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, clock/timezone related — verify against `main` if unsure whether a failure is new).
- [ ] **Step 11: Commit**
```bash
git add src/app.js src/server.js src/worker.js src/cleanup.js test/api/jobs.test.js test/api/jobStatus.test.js test/api/endToEnd.test.js test/worker.test.js test/cleanup.test.js
git commit -m "Wire the app, worker, and cleanup entry points onto Prisma Client"
```
---
### Task 6: Remove the mariadb dependency and the old schema file
**Files:**
- Modify: `package.json`
- Delete: `db/schema.sql`
**Interfaces:** none (cleanup only — no code depends on `mariadb` or `db/schema.sql` after Task 5).
- [ ] **Step 1: Delete the old schema file**
Run: `git rm db/schema.sql`
- [ ] **Step 2: Remove the `mariadb` dependency**
Edit `package.json`, removing the `"mariadb"` line from `dependencies` (keep the rest of the dependency list and its alphabetical ordering intact).
- [ ] **Step 3: Update the lockfile**
Run: `npm install`
Expected: `package-lock.json` updates to remove `mariadb` and its transitive dependencies; no other dependency versions change.
- [ ] **Step 4: Run the full test suite**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run`
Expected: same result as Task 5 Step 10 — all pass except the two pre-existing documented failures.
- [ ] **Step 5: Commit**
```bash
git add package.json package-lock.json db/schema.sql
git commit -m "Remove the mariadb dependency and the superseded schema.sql"
```
---
### Task 7: Final verification
**Files:** none (verification only).
- [ ] **Step 1: Run the full test suite one more time**
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run`
Expected: identical results to Task 6 Step 4.
- [ ] **Step 2: Confirm the frontend build is unaffected**
Run: `npm run build`
Expected: succeeds, unrelated to the DB layer change.
- [ ] **Step 3: Confirm `prisma generate` runs cleanly on a fresh install**
Run: `rm -rf node_modules && npm install`
Expected: `postinstall` runs `prisma generate` automatically as part of `npm install`, with no errors.
No commit needed for this task if all checks pass — it is verification only.