Task 1's reviewer found the original schema.prisma didn't reproduce db/schema.sql's DDL (DATETIME(3) vs DATETIME, auto-generated index names, no explicit precision on defaults). Verified the real DDL via SHOW CREATE TABLE and empirically confirmed @db.DateTime(0) plus explicit map: names close the gap. Two remaining differences (table collation, no DB-level ON UPDATE for updatedAt) are Prisma/MySQL provider limitations with no schema-level fix; documented as accepted, per user decision, since the baseline never executes this SQL against the real database and all writes go through Prisma. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
61 KiB
Prisma ORM and DB Migrations Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Replace the hand-rolled mariadb pool and raw-SQL repository with Prisma Client for data access, and introduce Prisma Migrate (with a no-data-loss baseline) as the schema migration system for conversion_jobs.
Architecture: prisma/schema.prisma becomes the single source of truth for the conversion_jobs table (replacing db/schema.sql), with prisma/migrations/ as its version history. src/db.js swaps its exported mariadb pool for a singleton PrismaClient. src/jobs/jobRepository.js is rewritten onto Prisma's model API, keeping the same exported function names and (client, ...) argument shape so src/app.js, src/worker.js, and src/cleanup.js only need a pool → prisma rename, not a logic change.
Tech Stack: Prisma ORM (prisma, @prisma/client) against the existing MariaDB instance via Prisma's mysql provider. Node.js ESM, Vitest, Supertest — all unchanged.
Global Constraints
prisma/schema.prismamust reproducedb/schema.sqlcolumn-for-column, so the initial migration is a no-op against databases that already have the table (spec: "Schema"). Two documented, accepted exceptions where Prisma's MySQL provider has no schema-level equivalent (verified empirically against the real DB'sSHOW CREATE TABLE conversion_jobs, and confirmed acceptable by the user on 2026-07-31): (1) table collation — the live MariaDB server defaults toutf8mb4_uca1400_ai_ci(a MariaDB-specific collation), but Prisma always emitsutf8mb4_unicode_ciin generated DDL with no schema attribute to override it; (2)updatedAt'sON UPDATE CURRENT_TIMESTAMP— Prisma's@updatedAtis implemented client-side only for themysqlprovider and never emits a DB-levelON UPDATEclause. Neither has practical impact here: the baseline (Task 3) marks the initial migration applied without ever executing its SQL against the real database, and every write toconversion_jobsafter this migration goes through Prisma Client, which setsupdatedAtitself. All other columns (types, defaults, index/constraint names) must match exactly — see Task 1's schema, which uses@db.DateTime(0)and explicitmap:names for this reason.DATABASE_URLis never written into.envor.env.local— it is always computed on demand from the existingDB_HOST/DB_USER/DB_PASSWORD/DB_NAMEvariables (spec: "Running Prisma CLI locally")..envholds production credentials and must never be loaded for local work (CLAUDE.md).- The connection string includes
?connection_limit=10, preserving the current pool'sconnectionLimit: 10(spec: "Running Prisma CLI locally"). getJobByUuid,findPendingJobs, andfindExpiredJobsmust never exposeerrorLog— onlygetJobErrorLogdoes (spec: "Data access layer").src/jobs/jobRepository.jsexported function names and their(client, ...)first-argument shape do not change (spec: "Data access layer").- The local test convention of passing
DB_HOST/DB_USER/DB_PASSWORD/DB_NAMEinline (documented in CLAUDE.md) must keep working unchanged after this migration. - After full implementation, the only acceptable test failures are the two already documented in CLAUDE.md (
test/cleanup.test.jsandtest/jobs/jobRepository.test.js, clock/timezone related) — no new regressions (spec: "Behavior change to watch for").
Task 1: Add Prisma dependencies and the schema definition
Files:
- Modify:
package.json - Create:
prisma/schema.prisma
Interfaces:
-
Produces:
prisma/schema.prismadefining modelConversionJob(Prisma Client generates typed methods from this in Task 4). -
Step 1: Install the Prisma packages
Run: npm install @prisma/client
Expected: package.json dependencies gains an "@prisma/client" entry; package-lock.json updated.
Run: npm install --save-dev prisma
Expected: package.json devDependencies gains a "prisma" entry; package-lock.json updated.
- Step 2: Create the Prisma schema
Create prisma/schema.prisma:
datasource db {
provider = "mysql"
url = env("DATABASE_URL")
}
generator client {
provider = "prisma-client-js"
}
enum JobStatus {
pending
processing
done
failed
}
model ConversionJob {
id Int @id @default(autoincrement()) @db.UnsignedInt
uuid String @unique(map: "uniq_uuid") @db.Char(36)
status JobStatus @default(pending)
family String @db.VarChar(32)
sourceFormat String @map("source_format") @db.VarChar(16)
targetFormat String @map("target_format") @db.VarChar(16)
originalFilename String @map("original_filename") @db.VarChar(255)
inputPath String @map("input_path") @db.VarChar(255)
outputPath String? @map("output_path") @db.VarChar(255)
inputMimeType String @map("input_mime_type") @db.VarChar(128)
outputMimeType String? @map("output_mime_type") @db.VarChar(128)
inputSizeBytes Int @map("input_size_bytes") @db.UnsignedInt
outputSizeBytes Int? @map("output_size_bytes") @db.UnsignedInt
quality Int? @db.UnsignedSmallInt
conversionDurationSeconds Decimal? @map("conversion_duration_seconds") @db.Decimal(10, 3)
errorMessage String? @map("error_message") @db.VarChar(255)
errorLog String? @map("error_log") @db.Text
createdAt DateTime @default(now()) @map("created_at") @db.DateTime(0)
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.DateTime(0)
expiresAt DateTime @map("expires_at") @db.DateTime(0)
cleanedAt DateTime? @map("cleaned_at") @db.DateTime(0)
@@index([status], map: "idx_status")
@@index([expiresAt], map: "idx_expires_at")
@@map("conversion_jobs")
}
@db.DateTime(0) matches the real table's datetime columns (no fractional seconds) — without it Prisma defaults to DATETIME(3)/CURRENT_TIMESTAMP(3), which would both mismatch the existing column type and risk an "Invalid default value" error on a fresh prisma migrate deploy (MySQL/MariaDB require the CURRENT_TIMESTAMP(n) default's precision to match the column's own precision). The map: "..." arguments make Prisma reuse the existing constraint/index names (uniq_uuid, idx_status, idx_expires_at) instead of auto-generating new ones. Verified empirically via prisma migrate diff --from-empty --to-schema-datamodel in a scratch directory on 2026-07-31.
- Step 3: Add the postinstall script
Edit package.json:
old:
"scripts": {
"start": "node src/server.js",
"worker": "node src/worker.js",
"cleanup": "node src/cleanup.js",
"test": "vitest run",
"build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend"
},
new:
"scripts": {
"start": "node src/server.js",
"worker": "node src/worker.js",
"cleanup": "node src/cleanup.js",
"test": "vitest run",
"build": "npm install --prefix frontend --include=dev && npm run build --prefix frontend",
"postinstall": "prisma generate"
},
- Step 4: Validate the schema
Run: DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma validate
Expected: The schema at prisma/schema.prisma is valid 🚀 (the connection string here is a placeholder only used to satisfy env("DATABASE_URL") resolution — validate does not connect to a database).
- Step 5: Generate the client
Run: DATABASE_URL="mysql://user:pass@localhost:3306/db" npx prisma generate
Expected: output confirms the client was generated, e.g. Generated Prisma Client ... to ./node_modules/@prisma/client.
- Step 6: Commit
git add package.json package-lock.json prisma/schema.prisma
git commit -m "Add Prisma schema and client generation"
Task 2: Add buildDatabaseUrl and the printDatabaseUrl CLI helper
Files:
- Modify:
src/db.js - Modify:
test/db.test.js - Create:
scripts/printDatabaseUrl.js
Interfaces:
- Consumes:
config.db = { host, user, password, database }shape fromloadConfig()(src/config.js, unchanged). - Produces:
buildDatabaseUrl(config): string— used by Task 4'sgetPrismaClientand byscripts/printDatabaseUrl.js.
This task is purely additive: getPool/closePool and the existing mariadb pool are untouched, so the whole app and test suite keep working exactly as today after this task.
- Step 1: Write the failing tests
Edit test/db.test.js, adding a new describe block above the existing describe('getPool', ...):
import { describe, it, expect, afterAll } from 'vitest';
import { getPool, closePool, buildDatabaseUrl } from '../src/db.js';
import { loadConfig } from '../src/config.js';
describe('buildDatabaseUrl', () => {
it('builds a mysql connection string from db config', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' },
});
expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10');
});
it('percent-encodes special characters in user and password', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' },
});
expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10');
});
});
describe('getPool', () => {
afterAll(async () => {
await closePool();
});
it('returns a working pool that can run a query', async () => {
const config = loadConfig();
const pool = getPool(config);
const rows = await pool.query('SELECT 1 AS value');
expect(Number(rows[0].value)).toBe(1);
});
it('returns the same pool instance on repeated calls', () => {
const config = loadConfig();
const poolA = getPool(config);
const poolB = getPool(config);
expect(poolA).toBe(poolB);
});
});
- Step 2: Run the test file to verify the new tests fail
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js
Expected: FAIL — buildDatabaseUrl is not a function (or import error), the two new tests fail; the two pre-existing getPool tests still pass.
- Step 3: Implement
buildDatabaseUrl
Edit src/db.js:
old:
import mariadb from 'mariadb';
let pool;
export function getPool(config) {
new:
import mariadb from 'mariadb';
let pool;
export function buildDatabaseUrl(config) {
const { host, user, password, database } = config.db;
return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`;
}
export function getPool(config) {
- Step 4: Run the test file to verify it passes
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js
Expected: PASS — all 4 tests green.
- Step 5: Create the CLI helper
Create scripts/printDatabaseUrl.js:
import { loadConfig } from '../src/config.js';
import { buildDatabaseUrl } from '../src/db.js';
console.log(buildDatabaseUrl(loadConfig()));
- Step 6: Verify the helper manually
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js
Expected stdout: mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10
- Step 7: Commit
git add src/db.js test/db.test.js scripts/printDatabaseUrl.js
git commit -m "Add buildDatabaseUrl and a CLI helper to print it"
Task 3: Baseline Prisma Migrate against the existing database
Files:
- Create:
prisma/migrations/migration_lock.toml - Create:
prisma/migrations/0_init/migration.sql - Modify:
CLAUDE.md
Interfaces:
- Consumes:
scripts/printDatabaseUrl.js(Task 2). - Produces: a migration history that Prisma considers already applied on the local dev DB, with
conversion_jobsand its rows untouched.
This task only touches Prisma's own bookkeeping (prisma/migrations/, and the _prisma_migrations tracking table it creates in the target database) — it must not run any SQL against conversion_jobs itself.
- Step 1: Record the current row count as a safety check
Run: mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"
Note the number returned — it must be identical after Step 5.
- Step 2: Confirm the diff flag name for this installed Prisma version
Run: npx prisma migrate diff --help
Expected: the help text lists a flag for rendering a schema file's data model as the diff target without connecting to a live database — documented in Prisma's migration-diffing workflow as --to-schema-datamodel <path>. Confirm this exact flag name in the output before proceeding; if this installed version names it differently, substitute the flag --help reports in Step 4.
- Step 3: Create the migration folder and lock file
Run: mkdir -p prisma/migrations/0_init
Create prisma/migrations/migration_lock.toml:
# Please do not edit this file manually
# It should be added in your version-control system (e.g., Git)
provider = "mysql"
- Step 4: Generate the migration SQL from the schema file (no database connection needed for this)
Run:
npx prisma migrate diff \
--from-empty \
--to-schema-datamodel prisma/schema.prisma \
--script > prisma/migrations/0_init/migration.sql
Expected: prisma/migrations/0_init/migration.sql is created containing a single CREATE TABLE conversion_jobs (...) statement (plus its indexes) — no DROP or ALTER statements.
- Step 5: Read the generated SQL and confirm it matches
db/schema.sql
Read prisma/migrations/0_init/migration.sql and compare column names, types, and constraints against db/schema.sql. They must describe the same table. If Prisma rendered a type differently than expected (e.g. a default value clause), stop and reconcile prisma/schema.prisma in Task 1 before continuing — do not hand-edit the generated SQL, regenerate it after fixing the schema.
- Step 6: Mark the migration as applied (bookkeeping only, does not run the SQL)
Run:
DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \
npx prisma migrate resolve --applied 0_init
Expected: Migration 0_init marked as applied.
- Step 7: Verify migration status and that data is untouched
Run:
DATABASE_URL=$(DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage node scripts/printDatabaseUrl.js) \
npx prisma migrate status
Expected: Database schema is up to date!
Run: mysql -h 127.0.0.1 -u convert_user -pchange_me file_converter -e "SELECT COUNT(*) FROM conversion_jobs;"
Expected: the same row count recorded in Step 1.
- Step 8: Document the migration workflow in CLAUDE.md
Edit CLAUDE.md, adding a new section after the existing "Local environment / running tests" section:
## Database schema & migrations
- `prisma/schema.prisma` is the source of truth for the `conversion_jobs` schema; `prisma/migrations/` is its version history. `db/schema.sql` no longer exists.
- Prisma CLI commands (`prisma migrate dev`, `prisma migrate deploy`, `prisma generate`) need `DATABASE_URL` in their own process environment, separate from the app. Compute it from the same `DB_HOST`/`DB_USER`/`DB_PASSWORD`/`DB_NAME` values used for local tests, via `node scripts/printDatabaseUrl.js` — never write `DATABASE_URL` into `.env` or `.env.local`.
- To create a new migration locally after editing `prisma/schema.prisma`:
DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate dev --name
- To apply pending migrations in production: with the real `DB_*` values loaded from `.env`, run `DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate deploy` before starting the app.
- Step 9: Commit
git add prisma/migrations CLAUDE.md
git commit -m "Baseline Prisma Migrate against the existing conversion_jobs table"
Task 4: Rewrite the data access layer onto Prisma Client
Files:
- Modify:
src/db.js - Modify:
src/jobs/jobRepository.js - Modify:
test/db.test.js - Modify:
test/jobs/jobRepository.test.js
Interfaces:
- Produces:
getPrismaClient(config): PrismaClient,closePrismaClient(): Promise<void>(replacinggetPool/closePool);createJob,getJobByUuid,getJobErrorLog,markProcessing,markDone,markFailed,findPendingJobs,findExpiredJobs,markCleaned— all still(prisma, ...)first-argument, same names, now backed by Prisma.
After this task, test/db.test.js and test/jobs/jobRepository.test.js pass in isolation. Every other file that still imports getPool/closePool from src/db.js (src/app.js, src/server.js, src/worker.js, src/cleanup.js, and 5 remaining test files) will fail to import until Task 5 lands — this is expected and addressed next.
- Step 1: Rewrite
src/db.js
Replace the full file content of src/db.js:
import { PrismaClient } from '@prisma/client';
let prisma;
export function buildDatabaseUrl(config) {
const { host, user, password, database } = config.db;
return `mysql://${encodeURIComponent(user)}:${encodeURIComponent(password)}@${host}:3306/${database}?connection_limit=10`;
}
export function getPrismaClient(config) {
if (!prisma) {
process.env.DATABASE_URL = buildDatabaseUrl(config);
prisma = new PrismaClient();
}
return prisma;
}
export async function closePrismaClient() {
if (prisma) {
await prisma.$disconnect();
prisma = undefined;
}
}
- Step 2: Rewrite
src/jobs/jobRepository.js
Replace the full file content of src/jobs/jobRepository.js:
const jobSelect = {
id: true,
uuid: true,
status: true,
family: true,
sourceFormat: true,
targetFormat: true,
originalFilename: true,
inputPath: true,
outputPath: true,
inputMimeType: true,
outputMimeType: true,
inputSizeBytes: true,
outputSizeBytes: true,
quality: true,
conversionDurationSeconds: true,
errorMessage: true,
createdAt: true,
updatedAt: true,
expiresAt: true,
cleanedAt: true,
};
export async function createJob(prisma, job) {
await prisma.conversionJob.create({
data: {
uuid: job.uuid,
family: job.family,
sourceFormat: job.sourceFormat,
targetFormat: job.targetFormat,
originalFilename: job.originalFilename,
inputPath: job.inputPath,
inputMimeType: job.inputMimeType,
inputSizeBytes: job.inputSizeBytes,
expiresAt: job.expiresAt,
quality: job.quality ?? null,
},
});
}
export async function getJobByUuid(prisma, uuid) {
return prisma.conversionJob.findUnique({ where: { uuid }, select: jobSelect });
}
export async function getJobErrorLog(prisma, id) {
const job = await prisma.conversionJob.findUnique({ where: { id }, select: { errorLog: true } });
return job?.errorLog ?? null;
}
export async function markProcessing(prisma, id) {
await prisma.conversionJob.update({ where: { id }, data: { status: 'processing' } });
}
export async function markDone(prisma, id, { outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds }) {
await prisma.conversionJob.update({
where: { id },
data: { status: 'done', outputPath, outputMimeType, outputSizeBytes, conversionDurationSeconds },
});
}
export async function markFailed(prisma, id, { errorMessage, errorLog }) {
await prisma.conversionJob.update({
where: { id },
data: { status: 'failed', errorMessage, errorLog },
});
}
export async function findPendingJobs(prisma, limit) {
return prisma.conversionJob.findMany({
where: { status: 'pending' },
orderBy: { createdAt: 'asc' },
take: limit,
select: jobSelect,
});
}
export async function findExpiredJobs(prisma) {
return prisma.conversionJob.findMany({
where: { expiresAt: { lt: new Date() }, cleanedAt: null },
select: jobSelect,
});
}
export async function markCleaned(prisma, id) {
await prisma.conversionJob.update({ where: { id }, data: { cleanedAt: new Date() } });
}
- Step 3: Rewrite
test/db.test.js
Replace the full file content of test/db.test.js:
import { describe, it, expect, afterAll } from 'vitest';
import { getPrismaClient, closePrismaClient, buildDatabaseUrl } from '../src/db.js';
import { loadConfig } from '../src/config.js';
describe('buildDatabaseUrl', () => {
it('builds a mysql connection string from db config', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'convert_user', password: 'change_me', database: 'file_converter' },
});
expect(url).toBe('mysql://convert_user:change_me@127.0.0.1:3306/file_converter?connection_limit=10');
});
it('percent-encodes special characters in user and password', () => {
const url = buildDatabaseUrl({
db: { host: '127.0.0.1', user: 'a@b', password: 'p@ss:word', database: 'file_converter' },
});
expect(url).toBe('mysql://a%40b:p%40ss%3Aword@127.0.0.1:3306/file_converter?connection_limit=10');
});
});
describe('getPrismaClient', () => {
afterAll(async () => {
await closePrismaClient();
});
it('returns a working client that can run a query', async () => {
const config = loadConfig();
const prisma = getPrismaClient(config);
const rows = await prisma.$queryRaw`SELECT 1 AS value`;
expect(Number(rows[0].value)).toBe(1);
});
it('returns the same client instance on repeated calls', () => {
const config = loadConfig();
const clientA = getPrismaClient(config);
const clientB = getPrismaClient(config);
expect(clientA).toBe(clientB);
});
});
- Step 4: Rewrite
test/jobs/jobRepository.test.js
Replace the full file content of test/jobs/jobRepository.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import {
createJob,
getJobByUuid,
getJobErrorLog,
markProcessing,
markDone,
markFailed,
findPendingJobs,
findExpiredJobs,
markCleaned,
} from '../../src/jobs/jobRepository.js';
let prisma;
beforeAll(() => {
prisma = getPrismaClient(loadConfig());
});
afterAll(async () => {
await closePrismaClient();
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
function baseJob(overrides = {}) {
return {
uuid: overrides.uuid ?? '11111111-1111-4111-8111-111111111111',
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${overrides.uuid ?? '11111111-1111-4111-8111-111111111111'}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 1024,
expiresAt: new Date(Date.now() + 3600 * 1000),
...overrides,
};
}
describe('jobRepository', () => {
it('creates and retrieves a pending job', async () => {
await createJob(prisma, baseJob());
const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(job.id).toEqual(expect.any(Number));
expect(job.uuid).toBe('11111111-1111-4111-8111-111111111111');
expect(job.status).toBe('pending');
expect(job.family).toBe('image');
expect(job.sourceFormat).toBe('png');
expect(job.targetFormat).toBe('webp');
expect(job.originalFilename).toBe('photo.png');
expect(job.inputMimeType).toBe('image/png');
expect(job.inputSizeBytes).toBe(1024);
expect(job.outputPath).toBeNull();
expect(job.outputSizeBytes).toBeNull();
expect(job.conversionDurationSeconds).toBeNull();
expect(job.errorMessage).toBeNull();
expect(job.cleanedAt).toBeNull();
expect(job.quality).toBeNull();
});
it('stores and retrieves a numeric quality value', async () => {
await createJob(prisma, baseJob({ uuid: '66666666-6666-4666-8666-666666666666', quality: 82 }));
const job = await getJobByUuid(prisma, '66666666-6666-4666-8666-666666666666');
expect(job.quality).toBe(82);
});
it('returns null for an unknown uuid', async () => {
const job = await getJobByUuid(prisma, '22222222-2222-4222-8222-222222222222');
expect(job).toBeNull();
});
it('transitions a job through processing to done', async () => {
await createJob(prisma, baseJob());
await markProcessing(prisma, (await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111')).id);
const processing = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(processing.status).toBe('processing');
await markDone(prisma, processing.id, {
outputPath: '11111111-1111-4111-8111-111111111111.webp',
outputMimeType: 'image/webp',
outputSizeBytes: 2048,
conversionDurationSeconds: 1.5,
});
const done = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(done.status).toBe('done');
expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp');
expect(done.outputMimeType).toBe('image/webp');
expect(done.outputSizeBytes).toBe(2048);
expect(Number(done.conversionDurationSeconds)).toBe(1.5);
});
it('marks a job failed with a short message and a separate detailed log', async () => {
await createJob(prisma, baseJob());
const created = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
await markFailed(prisma, created.id, {
errorMessage: 'Conversion failed, please try again',
errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)',
});
const job = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(job.status).toBe('failed');
expect(job.errorMessage).toBe('Conversion failed, please try again');
expect(job.errorLog).toBeUndefined();
const errorLog = await getJobErrorLog(prisma, created.id);
expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)');
});
it('finds pending jobs oldest first, up to a limit', async () => {
await createJob(prisma, baseJob({ uuid: '33333333-3333-4333-8333-333333333333' }));
await createJob(prisma, baseJob({ uuid: '44444444-4444-4444-8444-444444444444' }));
await createJob(prisma, baseJob({ uuid: '55555555-5555-4555-8555-555555555555' }));
const jobs = await findPendingJobs(prisma, 2);
expect(jobs).toHaveLength(2);
expect(jobs[0].uuid).toBe('33333333-3333-4333-8333-333333333333');
expect(jobs[1].uuid).toBe('44444444-4444-4444-8444-444444444444');
});
it('finds expired jobs and allows marking them cleaned without deleting the row', async () => {
await createJob(prisma, baseJob({ expiresAt: new Date(Date.now() - 1000) }));
const expired = await findExpiredJobs(prisma);
expect(expired).toHaveLength(1);
expect(expired[0].uuid).toBe('11111111-1111-4111-8111-111111111111');
await markCleaned(prisma, expired[0].id);
const stillPresent = await getJobByUuid(prisma, '11111111-1111-4111-8111-111111111111');
expect(stillPresent).not.toBeNull();
expect(stillPresent.cleanedAt).not.toBeNull();
const afterCleaning = await findExpiredJobs(prisma);
expect(afterCleaning).toHaveLength(0);
});
});
- Step 5: Run the two updated test files
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage npx vitest run test/db.test.js test/jobs/jobRepository.test.js
Expected: PASS — all tests green. (Running the full suite at this point will show import errors in other files; that's expected until Task 5.)
- Step 6: Commit
git add src/db.js src/jobs/jobRepository.js test/db.test.js test/jobs/jobRepository.test.js
git commit -m "Rewrite the data access layer onto Prisma Client"
Task 5: Wire up the remaining call sites and tests
Files:
- Modify:
src/app.js - Modify:
src/server.js - Modify:
src/worker.js - Modify:
src/cleanup.js - Modify:
test/api/jobs.test.js - Modify:
test/api/jobStatus.test.js - Modify:
test/api/endToEnd.test.js - Modify:
test/worker.test.js - Modify:
test/cleanup.test.js
Interfaces:
- Consumes:
getPrismaClient,closePrismaClient(Task 4,src/db.js);createJob,getJobByUuid,markProcessing,markDone,markFailed,findPendingJobs,findExpiredJobs,markCleaned(Task 4,src/jobs/jobRepository.js). - Produces:
createApp(config, prisma)(renamed second parameter, same behavior).
After this task the full test suite should pass except the two pre-existing documented failures.
- Step 1: Update
src/app.js
Edit src/app.js:
old:
export function createApp(config, pool) {
new:
export function createApp(config, prisma) {
old:
const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000);
await createJob(pool, {
new:
const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000);
await createJob(prisma, {
old:
app.get('/api/jobs/:id', async (req, res) => {
const job = await getJobByUuid(pool, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
new:
app.get('/api/jobs/:id', async (req, res) => {
const job = await getJobByUuid(prisma, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
old:
app.get('/api/jobs/:id/download', async (req, res) => {
const job = await getJobByUuid(pool, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
if (job.status !== 'done') {
new:
app.get('/api/jobs/:id/download', async (req, res) => {
const job = await getJobByUuid(prisma, req.params.id);
if (!job) {
return res.status(404).json({ error: 'Job not found' });
}
if (job.status !== 'done') {
- Step 2: Update
src/server.js
Replace the full file content of src/server.js:
import { loadConfig } from './config.js';
import { getPrismaClient } from './db.js';
import { ensureStorageDirs } from './storage.js';
import { createApp } from './app.js';
async function main() {
const config = loadConfig();
await ensureStorageDirs(config);
const prisma = getPrismaClient(config);
const app = createApp(config, prisma);
app.listen(config.port, () => {
console.log(`File converter API listening on port ${config.port}`);
});
}
main().catch((error) => {
console.error('Failed to start server:', error);
process.exit(1);
});
- Step 3: Update
src/worker.js
Replace the full file content of src/worker.js:
import { pathToFileURL } from 'node:url';
import fs from 'node:fs/promises';
import { loadConfig } from './config.js';
import { getPrismaClient } from './db.js';
import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js';
import { outputMimeType } from './mime.js';
import { resolve as resolveConverter } from './converters/registry.js';
import { registerImageConverters } from './converters/image.js';
import { registerImageToPdfConverter } from './converters/imageToPdf.js';
import { registerDocumentConverters } from './converters/document.js';
import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js';
const JOB_TIMEOUT_MS = 60000;
function withTimeout(promise, ms) {
let timeoutId;
const timeout = new Promise((_, reject) => {
timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms);
});
return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId));
}
async function processJob(prisma, config, job) {
await markProcessing(prisma, job.id);
const inputFilePath = uploadPath(config, job.uuid, job.sourceFormat);
const outputFilePath = outputPath(config, job.uuid, job.targetFormat);
try {
const entry = resolveConverter(job.sourceFormat, job.targetFormat);
if (!entry) {
throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`);
}
const startedAt = Date.now();
await withTimeout(entry.convert(inputFilePath, outputFilePath, { quality: job.quality }), JOB_TIMEOUT_MS);
const conversionDurationSeconds = (Date.now() - startedAt) / 1000;
const { size: outputSizeBytes } = await fs.stat(outputFilePath);
await markDone(prisma, job.id, {
outputPath: `${job.uuid}.${job.targetFormat}`,
outputMimeType: outputMimeType(job.targetFormat),
outputSizeBytes,
conversionDurationSeconds,
});
} catch (error) {
await deleteIfExists(outputFilePath);
await markFailed(prisma, job.id, {
errorMessage: 'Conversion failed, please try again.',
errorLog: error.stack ?? String(error),
});
}
}
export async function processPendingJobs(prisma, config) {
const jobs = await findPendingJobs(prisma, config.workerConcurrency);
await Promise.all(jobs.map((job) => processJob(prisma, config, job)));
return jobs.length;
}
export function startWorker(prisma, config) {
const interval = setInterval(() => {
processPendingJobs(prisma, config).catch((error) => {
console.error('Error while processing pending jobs:', error);
});
}, config.workerPollIntervalMs);
return () => clearInterval(interval);
}
async function main() {
const config = loadConfig();
await ensureStorageDirs(config);
const prisma = getPrismaClient(config);
registerImageConverters();
registerImageToPdfConverter();
registerDocumentConverters();
startWorker(prisma, config);
console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`);
}
main();
- Step 4: Update
src/cleanup.js
Replace the full file content of src/cleanup.js:
import { pathToFileURL } from 'node:url';
import { loadConfig } from './config.js';
import { getPrismaClient, closePrismaClient } from './db.js';
import { uploadPath, outputPath, deleteIfExists } from './storage.js';
import { findExpiredJobs, markCleaned } from './jobs/jobRepository.js';
export async function runCleanup(prisma, config) {
const expiredJobs = await findExpiredJobs(prisma);
for (const job of expiredJobs) {
await deleteIfExists(uploadPath(config, job.uuid, job.sourceFormat));
await deleteIfExists(outputPath(config, job.uuid, job.targetFormat));
await markCleaned(prisma, job.id);
}
return expiredJobs.length;
}
async function main() {
const config = loadConfig();
const prisma = getPrismaClient(config);
const deletedCount = await runCleanup(prisma, config);
console.log(`Cleanup: removed ${deletedCount} expired job(s).`);
await closePrismaClient();
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
main();
}
- Step 5: Update
test/api/jobs.test.js
Replace the full file content of test/api/jobs.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { getJobByUuid } from '../../src/jobs/jobRepository.js';
import { ensureStorageDirs } from '../../src/storage.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
describe('GET /api/formats', () => {
it('lists valid target formats for a known source format', async () => {
const response = await request(app).get('/api/formats').query({ source: 'png' });
expect(response.status).toBe(200);
expect(response.body.targets).toContain('webp');
expect(response.body.targets).toContain('pdf');
});
it('returns an empty list for an unknown source format', async () => {
const response = await request(app).get('/api/formats').query({ source: 'made-up' });
expect(response.status).toBe(200);
expect(response.body.targets).toEqual([]);
});
});
describe('POST /api/jobs', () => {
it('creates a pending job for a valid image upload', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs).toHaveLength(1);
expect(response.body.jobs[0].status).toBe('pending');
expect(response.body.jobs[0].file).toBe('photo.png');
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
expect(job.status).toBe('pending');
expect(job.sourceFormat).toBe('png');
expect(job.targetFormat).toBe('webp');
expect(job.originalFilename).toBe('photo.png');
expect(job.inputMimeType).toBe('image/png');
expect(job.inputSizeBytes).toBeGreaterThan(0);
expect(job.quality).toBeNull();
});
it('creates a pending job with a quality level for a supported format', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([45]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].status).toBe('pending');
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
expect(job.quality).toBe(45);
});
it('rejects an out-of-range quality without failing the whole batch', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([500]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
});
it('rejects a quality value for a format that does not support one', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['gif']))
.field('qualities', JSON.stringify([50]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
});
it('returns 400 when qualities length does not match the number of files', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['jpg']))
.field('qualities', JSON.stringify([10, 20]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(400);
});
it('rejects a file whose content does not match its extension, without failing the whole batch', async () => {
const fakePath = path.join(config.storageDir, 'fake.png');
await fs.writeFile(fakePath, 'not actually a png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fakePath, 'fake.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/does not match/);
await fs.unlink(fakePath);
});
it('rejects an unsupported source/target pair', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['made-up-format']))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(201);
expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/);
});
it('returns 400 when targetFormats length does not match the number of files', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify([]))
.attach('files', fixturePath, 'photo.png');
expect(response.status).toBe(400);
});
it('returns 400 when no files are uploaded', async () => {
const response = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify([]));
expect(response.status).toBe(400);
});
});
- Step 6: Update
test/api/jobStatus.test.js
Replace the full file content of test/api/jobStatus.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { createJob, getJobByUuid, markDone } from '../../src/jobs/jobRepository.js';
import { ensureStorageDirs, outputPath } from '../../src/storage.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
function baseJob(uuid) {
return {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'holiday photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
};
}
describe('GET /api/jobs/:id', () => {
it('returns job status without the error log field', async () => {
const uuid = '66666666-6666-4666-8666-666666666666';
await createJob(prisma, baseJob(uuid));
const response = await request(app).get(`/api/jobs/${uuid}`);
expect(response.status).toBe(200);
expect(response.body.id).toBe(uuid);
expect(response.body.status).toBe('pending');
expect(response.body.originalFilename).toBe('holiday photo.png');
expect(response.body.errorLog).toBeUndefined();
});
it('returns 404 for an unknown id', async () => {
const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000');
expect(response.status).toBe(404);
});
});
describe('GET /api/jobs/:id/download', () => {
it('streams the converted file with correct headers once done', async () => {
const uuid = '77777777-7777-4777-8777-777777777777';
await createJob(prisma, baseJob(uuid));
const filePath = outputPath(config, uuid, 'webp');
await fs.writeFile(filePath, Buffer.from('fake webp bytes'));
const created = await getJobByUuid(prisma, uuid);
await markDone(prisma, created.id, {
outputPath: `${uuid}.webp`,
outputMimeType: 'image/webp',
outputSizeBytes: 16,
conversionDurationSeconds: 0.2,
});
const response = await request(app).get(`/api/jobs/${uuid}/download`);
expect(response.status).toBe(200);
expect(response.headers['content-type']).toBe('image/webp');
expect(response.headers['content-disposition']).toContain('holiday photo.webp');
expect(response.body ?? response.text).toBeDefined();
});
it('returns 409 when the job is not done yet', async () => {
const uuid = '88888888-8888-4888-8888-888888888888';
await createJob(prisma, baseJob(uuid));
const response = await request(app).get(`/api/jobs/${uuid}/download`);
expect(response.status).toBe(409);
});
it('returns 404 for an unknown id', async () => {
const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download');
expect(response.status).toBe(404);
});
});
describe('unexpected server errors', () => {
it('returns a generic 500 without leaking internal error details', async () => {
const originalFindUnique = prisma.conversionJob.findUnique.bind(prisma.conversionJob);
prisma.conversionJob.findUnique = () => Promise.reject(new Error('connection reset by peer'));
const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999');
prisma.conversionJob.findUnique = originalFindUnique;
expect(response.status).toBe(500);
expect(response.body.error).toBe('Internal server error');
expect(JSON.stringify(response.body)).not.toContain('connection reset by peer');
});
});
- Step 7: Update
test/api/endToEnd.test.js
Replace the full file content of test/api/endToEnd.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import request from 'supertest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { Document, Paragraph, TextRun, Packer } from 'docx';
import { createApp } from '../../src/app.js';
import { getPrismaClient, closePrismaClient } from '../../src/db.js';
import { loadConfig } from '../../src/config.js';
import { ensureStorageDirs } from '../../src/storage.js';
import { processPendingJobs } from '../../src/worker.js';
let app;
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
app = createApp(config, prisma);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
async function waitForDone(id, maxAttempts = 20) {
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
await processPendingJobs(prisma, config);
const response = await request(app).get(`/api/jobs/${id}`);
if (response.body.status === 'done' || response.body.status === 'failed') {
return response.body;
}
}
throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`);
}
describe('end-to-end: image family', () => {
it('uploads a PNG, converts it to WebP, and downloads the result', async () => {
const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png');
const uploadResponse = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['webp']))
.attach('files', fixturePath, 'photo.png');
const { id } = uploadResponse.body.jobs[0];
const finalStatus = await waitForDone(id);
expect(finalStatus.status).toBe('done');
const downloadResponse = await request(app).get(`/api/jobs/${id}/download`);
expect(downloadResponse.status).toBe(200);
expect(downloadResponse.headers['content-type']).toBe('image/webp');
expect(downloadResponse.headers['content-disposition']).toContain('photo.webp');
}, 20000);
});
describe('end-to-end: document family', () => {
it('uploads a DOCX, converts it to PDF, and downloads the result', async () => {
const doc = new Document({
sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }],
});
const buffer = await Packer.toBuffer(doc);
const docxPath = path.join(config.storageDir, 'e2e-fixture.docx');
await fs.writeFile(docxPath, buffer);
const uploadResponse = await request(app)
.post('/api/jobs')
.field('targetFormats', JSON.stringify(['pdf']))
.attach('files', docxPath, 'report.docx');
const { id } = uploadResponse.body.jobs[0];
const finalStatus = await waitForDone(id);
expect(finalStatus.status).toBe('done');
const downloadResponse = await request(app).get(`/api/jobs/${id}/download`);
expect(downloadResponse.status).toBe(200);
expect(downloadResponse.headers['content-type']).toBe('application/pdf');
expect(downloadResponse.headers['content-disposition']).toContain('report.pdf');
await fs.unlink(docxPath);
}, 20000);
});
- Step 8: Update
test/worker.test.js
Replace the full file content of test/worker.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import sharp from 'sharp';
import { getPrismaClient, closePrismaClient } from '../src/db.js';
import { loadConfig } from '../src/config.js';
import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js';
import { createJob, getJobByUuid, getJobErrorLog } from '../src/jobs/jobRepository.js';
import { registerImageConverters } from '../src/converters/image.js';
import { processPendingJobs } from '../src/worker.js';
let prisma;
let config;
beforeAll(async () => {
registerImageConverters();
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
async function createPendingImageJob(uuid, sourceFormat, targetFormat) {
const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png');
const inputFilePath = uploadPath(config, uuid, sourceFormat);
await fs.copyFile(fixturePath, inputFilePath);
const { size: inputSizeBytes } = await fs.stat(inputFilePath);
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat,
targetFormat,
originalFilename: `photo.${sourceFormat}`,
inputPath: `${uuid}.${sourceFormat}`,
inputMimeType: 'image/png',
inputSizeBytes,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
}
describe('processPendingJobs', () => {
it('converts a pending image job to done', async () => {
const uuid = '99999999-9999-4999-8999-999999999999';
await createPendingImageJob(uuid, 'png', 'webp');
const processedCount = await processPendingJobs(prisma, config);
expect(processedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job.status).toBe('done');
expect(job.outputPath).toBe(`${uuid}.webp`);
expect(job.outputMimeType).toBe('image/webp');
expect(job.outputSizeBytes).toBeGreaterThan(0);
expect(Number(job.conversionDurationSeconds)).toBeGreaterThanOrEqual(0);
const stat = await fs.stat(outputPath(config, uuid, 'webp'));
expect(stat.size).toBeGreaterThan(0);
});
it('passes the job quality through to the converter, shrinking output for a low quality value', async () => {
const lowUuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee';
const defaultUuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff';
const noisyBuffer = await sharp({
create: {
width: 256,
height: 256,
channels: 3,
noise: { type: 'gaussian', mean: 128, sigma: 40 },
},
})
.png()
.toBuffer();
for (const uuid of [lowUuid, defaultUuid]) {
await fs.writeFile(uploadPath(config, uuid, 'png'), noisyBuffer);
}
await createJob(prisma, {
uuid: lowUuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'jpg',
originalFilename: 'noisy.png',
inputPath: `${lowUuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: noisyBuffer.length,
expiresAt: new Date(Date.now() + 3600 * 1000),
quality: 5,
});
await createJob(prisma, {
uuid: defaultUuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'jpg',
originalFilename: 'noisy.png',
inputPath: `${defaultUuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: noisyBuffer.length,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
await processPendingJobs(prisma, { ...config, workerConcurrency: 2 });
const lowJob = await getJobByUuid(prisma, lowUuid);
const defaultJob = await getJobByUuid(prisma, defaultUuid);
expect(lowJob.status).toBe('done');
expect(defaultJob.status).toBe('done');
expect(lowJob.outputSizeBytes).toBeLessThan(defaultJob.outputSizeBytes);
});
it('marks a job failed with a safe message and a detailed log when the converter throws', async () => {
const uuid = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'missing.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
// Note: input file is intentionally never written, so sharp will throw ENOENT.
await processPendingJobs(prisma, config);
const job = await getJobByUuid(prisma, uuid);
expect(job.status).toBe('failed');
expect(job.errorMessage).toBe('Conversion failed, please try again.');
const errorLog = await getJobErrorLog(prisma, job.id);
expect(errorLog).toMatch(/input file is missing/i);
});
it('only picks up as many jobs as workerConcurrency allows', async () => {
await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp');
await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp');
await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp');
const limitedConfig = { ...config, workerConcurrency: 2 };
const processedCount = await processPendingJobs(prisma, limitedConfig);
expect(processedCount).toBe(2);
});
});
- Step 9: Update
test/cleanup.test.js
Replace the full file content of test/cleanup.test.js:
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest';
import fs from 'node:fs/promises';
import path from 'node:path';
import os from 'node:os';
import { getPrismaClient, closePrismaClient } from '../src/db.js';
import { loadConfig } from '../src/config.js';
import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js';
import { createJob, markDone, getJobByUuid } from '../src/jobs/jobRepository.js';
import { runCleanup } from '../src/cleanup.js';
let prisma;
let config;
beforeAll(async () => {
config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) };
await ensureStorageDirs(config);
prisma = getPrismaClient(config);
});
afterAll(async () => {
await closePrismaClient();
await fs.rm(config.storageDir, { recursive: true, force: true });
});
beforeEach(async () => {
await prisma.conversionJob.deleteMany();
});
describe('runCleanup', () => {
it('marks an expired done job cleaned and removes its input and output files, keeping the row', async () => {
const uuid = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee';
await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes');
await fs.writeFile(outputPath(config, uuid, 'webp'), 'output bytes');
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const created = await getJobByUuid(prisma, uuid);
await markDone(prisma, created.id, {
outputPath: `${uuid}.webp`,
outputMimeType: 'image/webp',
outputSizeBytes: 12,
conversionDurationSeconds: 0.5,
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).not.toBeNull();
await expect(fs.stat(uploadPath(config, uuid, 'png'))).rejects.toThrow();
await expect(fs.stat(outputPath(config, uuid, 'webp'))).rejects.toThrow();
});
it('marks an expired pending job (with no output file) cleaned without throwing', async () => {
const uuid = 'ffffffff-ffff-4fff-8fff-ffffffffffff';
await fs.writeFile(uploadPath(config, uuid, 'png'), 'input bytes');
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(1);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).not.toBeNull();
});
it('leaves non-expired jobs untouched', async () => {
const uuid = '12121212-1212-4212-8212-121212121212';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() + 3600 * 1000),
});
const cleanedCount = await runCleanup(prisma, config);
expect(cleanedCount).toBe(0);
const job = await getJobByUuid(prisma, uuid);
expect(job).not.toBeNull();
expect(job.cleanedAt).toBeNull();
});
it('does not re-process an already cleaned expired job', async () => {
const uuid = '13131313-1313-4313-8313-131313131313';
await createJob(prisma, {
uuid,
family: 'image',
sourceFormat: 'png',
targetFormat: 'webp',
originalFilename: 'photo.png',
inputPath: `${uuid}.png`,
inputMimeType: 'image/png',
inputSizeBytes: 11,
expiresAt: new Date(Date.now() - 1000),
});
const firstRun = await runCleanup(prisma, config);
const secondRun = await runCleanup(prisma, config);
expect(firstRun).toBe(1);
expect(secondRun).toBe(0);
});
});
- Step 10: Run the full test suite
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run
Expected: all tests pass except the two pre-existing documented failures (test/cleanup.test.js and test/jobs/jobRepository.test.js, clock/timezone related — verify against main if unsure whether a failure is new).
- Step 11: Commit
git add src/app.js src/server.js src/worker.js src/cleanup.js test/api/jobs.test.js test/api/jobStatus.test.js test/api/endToEnd.test.js test/worker.test.js test/cleanup.test.js
git commit -m "Wire the app, worker, and cleanup entry points onto Prisma Client"
Task 6: Remove the mariadb dependency and the old schema file
Files:
- Modify:
package.json - Delete:
db/schema.sql
Interfaces: none (cleanup only — no code depends on mariadb or db/schema.sql after Task 5).
- Step 1: Delete the old schema file
Run: git rm db/schema.sql
- Step 2: Remove the
mariadbdependency
Edit package.json, removing the "mariadb" line from dependencies (keep the rest of the dependency list and its alphabetical ordering intact).
- Step 3: Update the lockfile
Run: npm install
Expected: package-lock.json updates to remove mariadb and its transitive dependencies; no other dependency versions change.
- Step 4: Run the full test suite
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run
Expected: same result as Task 5 Step 10 — all pass except the two pre-existing documented failures.
- Step 5: Commit
git add package.json package-lock.json db/schema.sql
git commit -m "Remove the mariadb dependency and the superseded schema.sql"
Task 7: Final verification
Files: none (verification only).
- Step 1: Run the full test suite one more time
Run: DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run
Expected: identical results to Task 6 Step 4.
- Step 2: Confirm the frontend build is unaffected
Run: npm run build
Expected: succeeds, unrelated to the DB layer change.
- Step 3: Confirm
prisma generateruns cleanly on a fresh install
Run: rm -rf node_modules && npm install
Expected: postinstall runs prisma generate automatically as part of npm install, with no errors.
No commit needed for this task if all checks pass — it is verification only.