# File Converter Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build the core (upload → convert → download) pipeline of a Convertio-like file converter, covering the Images and Documents format families, deployable on o2switch shared hosting. **Architecture:** An Express API process (behind Passenger) accepts uploads and writes `pending` jobs to a MariaDB `conversion_jobs` table; a separate worker process (run via `pm2`, outside Passenger) polls that table, runs the matching converter from a central format registry, and writes the result to disk. A React SPA drives uploads and polls job status. A cron-triggered cleanup script deletes expired jobs and their files. **Tech Stack:** Node.js (ESM, `"type": "module"`), Express, MariaDB (`mariadb` npm connector), `multer` (upload), `sharp` (images), `pdf-lib` + `puppeteer` + `mammoth` + `pdfjs-dist` + `docx` (documents), `file-type` (MIME sniffing), `express-rate-limit`, `uuid`, React (Vite) frontend, `vitest` + `supertest` for tests. ## Global Constraints - No native binaries installed at the system level (no apt-get, no compilation). Only pure-JS npm packages or npm packages that auto-download a precompiled binary during `npm install` (e.g. Puppeteer's Chromium) are allowed. - `npm install` must never run with `--ignore-scripts` in deployment — Puppeteer's Chromium download depends on its postinstall script running. - Max upload size: `MAX_FILE_SIZE_MB`, default 100. - Retention: `RETENTION_HOURS`, default 1 — expired jobs (row + files) are deleted by the cleanup script. - Job IDs are UUID v4, used directly in public URLs (`/api/jobs/:id`, `/api/jobs/:id/download`). Non-enumerable by design — no sequential IDs anywhere in the public API. - Files on disk are named `.` directly under `STORAGE_DIR/uploads/` and `STORAGE_DIR/outputs/` — no subfolders, no original filename in the path. - `input_mime_type` is always detected from magic bytes (`file-type`), never trusted from the client's declared `Content-Type` or file extension. A mismatch with the declared `source_format` is rejected (HTTP 422) before a job is created. - `error_message` (short, user-safe) and `error_log` (full technical detail, never returned by the API) are separate columns — the API must never leak `error_log` contents to a client. - Worker concurrency is bounded by `WORKER_CONCURRENCY` — jobs beyond that stay `pending` rather than being spawned unbounded. - Spec reference: `docs/superpowers/specs/2026-07-28-file-converter-design.md`. --- ## Task 1: Project scaffolding & config **Files:** - Create: `package.json` - Create: `.env.example` - Create: `.gitignore` (append `node_modules/`, `.env`, `storage/`, `dist/` if not already present) - Create: `src/config.js` - Test: `test/config.test.js` **Interfaces:** - Produces: `export function loadConfig()` in `src/config.js`, returning: ```js { port: number, storageDir: string, db: { host: string, user: string, password: string, database: string }, maxFileSizeMb: number, retentionHours: number, workerPollIntervalMs: number, workerConcurrency: number, rateLimitMaxJobs: number, rateLimitWindowMinutes: number, } ``` Throws an `Error` listing every missing required variable if any of `STORAGE_DIR`, `DB_HOST`, `DB_USER`, `DB_PASSWORD`, `DB_NAME` is unset. - [ ] **Step 1: Create `package.json`** ```bash npm init -y ``` Then edit `package.json` to set: ```json { "name": "file-converter", "version": "0.1.0", "private": true, "type": "module", "scripts": { "start": "node src/app.js", "worker": "node src/worker.js", "cleanup": "node src/cleanup.js", "test": "vitest run" } } ``` - [ ] **Step 2: Install dependencies** ```bash npm install express multer mariadb uuid file-type sharp pdf-lib puppeteer mammoth docx pdfjs-dist express-rate-limit dotenv npm install --save-dev vitest supertest ``` Expected: `npm install` completes without error (Puppeteer's postinstall will download Chromium — this can take a minute). - [ ] **Step 3: Create `.env.example`** ```dotenv PORT=3000 STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter MAX_FILE_SIZE_MB=100 RETENTION_HOURS=1 WORKER_POLL_INTERVAL_MS=1500 WORKER_CONCURRENCY=3 RATE_LIMIT_MAX_JOBS=20 RATE_LIMIT_WINDOW_MINUTES=10 ``` - [ ] **Step 4: Update `.gitignore`** Add these lines if not already present: ```gitignore node_modules/ .env storage/ dist/ ``` - [ ] **Step 5: Write the failing test for config loading** Create `test/config.test.js`: ```js import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { loadConfig } from '../src/config.js'; const REQUIRED_VARS = ['STORAGE_DIR', 'DB_HOST', 'DB_USER', 'DB_PASSWORD', 'DB_NAME']; let savedEnv; beforeEach(() => { savedEnv = { ...process.env }; }); afterEach(() => { process.env = savedEnv; }); describe('loadConfig', () => { it('returns parsed config when all required vars are set', () => { process.env.STORAGE_DIR = './storage'; process.env.DB_HOST = 'localhost'; process.env.DB_USER = 'user'; process.env.DB_PASSWORD = 'pass'; process.env.DB_NAME = 'db'; process.env.PORT = '4000'; process.env.MAX_FILE_SIZE_MB = '50'; const config = loadConfig(); expect(config.port).toBe(4000); expect(config.storageDir).toBe('./storage'); expect(config.db).toEqual({ host: 'localhost', user: 'user', password: 'pass', database: 'db' }); expect(config.maxFileSizeMb).toBe(50); }); it('applies defaults for optional vars', () => { process.env.STORAGE_DIR = './storage'; process.env.DB_HOST = 'localhost'; process.env.DB_USER = 'user'; process.env.DB_PASSWORD = 'pass'; process.env.DB_NAME = 'db'; delete process.env.PORT; delete process.env.MAX_FILE_SIZE_MB; delete process.env.RETENTION_HOURS; delete process.env.WORKER_CONCURRENCY; const config = loadConfig(); expect(config.port).toBe(3000); expect(config.maxFileSizeMb).toBe(100); expect(config.retentionHours).toBe(1); expect(config.workerConcurrency).toBe(3); }); it('throws listing every missing required variable', () => { for (const key of REQUIRED_VARS) delete process.env[key]; expect(() => loadConfig()).toThrowError(/STORAGE_DIR.*DB_HOST.*DB_USER.*DB_PASSWORD.*DB_NAME/s); }); }); ``` - [ ] **Step 6: Run test to verify it fails** Run: `npx vitest run test/config.test.js` Expected: FAIL — `src/config.js` does not exist yet. - [ ] **Step 7: Implement `src/config.js`** ```js import 'dotenv/config'; const REQUIRED_VARS = ['STORAGE_DIR', 'DB_HOST', 'DB_USER', 'DB_PASSWORD', 'DB_NAME']; export function loadConfig() { const missing = REQUIRED_VARS.filter((key) => !process.env[key]); if (missing.length > 0) { throw new Error(`Missing required environment variables: ${missing.join(', ')}`); } return { port: Number(process.env.PORT ?? 3000), storageDir: process.env.STORAGE_DIR, db: { host: process.env.DB_HOST, user: process.env.DB_USER, password: process.env.DB_PASSWORD, database: process.env.DB_NAME, }, maxFileSizeMb: Number(process.env.MAX_FILE_SIZE_MB ?? 100), retentionHours: Number(process.env.RETENTION_HOURS ?? 1), workerPollIntervalMs: Number(process.env.WORKER_POLL_INTERVAL_MS ?? 1500), workerConcurrency: Number(process.env.WORKER_CONCURRENCY ?? 3), rateLimitMaxJobs: Number(process.env.RATE_LIMIT_MAX_JOBS ?? 20), rateLimitWindowMinutes: Number(process.env.RATE_LIMIT_WINDOW_MINUTES ?? 10), }; } ``` - [ ] **Step 8: Run test to verify it passes** Run: `npx vitest run test/config.test.js` Expected: PASS (3 tests) - [ ] **Step 9: Commit** ```bash git add package.json package-lock.json .env.example .gitignore src/config.js test/config.test.js git commit -m "feat: project scaffolding and env config loader" ``` --- ## Task 2: MariaDB schema & connection pool **Files:** - Create: `db/schema.sql` - Create: `src/db.js` - Test: `test/db.test.js` **Interfaces:** - Consumes: `loadConfig().db` from Task 1 (`{ host, user, password, database }`). - Produces: `export function getPool(config)` in `src/db.js` — returns a singleton `mariadb` pool built from `config.db` (creates it on first call, returns the same instance on subsequent calls). Also exports `export async function closePool()` to end the pool (used in tests and graceful shutdown). - [ ] **Step 1: Write the schema file** Create `db/schema.sql`: ```sql CREATE TABLE IF NOT EXISTS conversion_jobs ( id CHAR(36) NOT NULL PRIMARY KEY, status ENUM('pending', 'processing', 'done', 'failed') NOT NULL DEFAULT 'pending', family VARCHAR(32) NOT NULL, source_format VARCHAR(16) NOT NULL, target_format VARCHAR(16) NOT NULL, original_filename VARCHAR(255) NOT NULL, input_path VARCHAR(255) NOT NULL, output_path VARCHAR(255) NULL, input_mime_type VARCHAR(128) NOT NULL, output_mime_type VARCHAR(128) NULL, error_message VARCHAR(255) NULL, error_log TEXT NULL, created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, expires_at DATETIME NOT NULL, INDEX idx_status (status), INDEX idx_expires_at (expires_at) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` - [ ] **Step 2: Apply the schema to a local/test MariaDB database** Run (adjust host/user/db to your local MariaDB instance used for development): ```bash mysql -h 127.0.0.1 -u convert_user -p file_converter < db/schema.sql ``` Expected: no error, `conversion_jobs` table exists (`SHOW TABLES;` lists it). - [ ] **Step 3: Write the failing test for the pool** Create `test/db.test.js`: ```js import { describe, it, expect, afterAll } from 'vitest'; import { getPool, closePool } from '../src/db.js'; import { loadConfig } from '../src/config.js'; describe('getPool', () => { afterAll(async () => { await closePool(); }); it('returns a working pool that can run a query', async () => { const config = loadConfig(); const pool = getPool(config); const rows = await pool.query('SELECT 1 AS value'); expect(Number(rows[0].value)).toBe(1); }); it('returns the same pool instance on repeated calls', () => { const config = loadConfig(); const poolA = getPool(config); const poolB = getPool(config); expect(poolA).toBe(poolB); }); }); ``` Note: this test requires a reachable MariaDB instance matching the env vars used to run tests: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/db.test.js`. - [ ] **Step 4: Run test to verify it fails** Run: `npx vitest run test/db.test.js` Expected: FAIL — `src/db.js` does not exist yet. - [ ] **Step 5: Implement `src/db.js`** ```js import mariadb from 'mariadb'; let pool; export function getPool(config) { if (!pool) { pool = mariadb.createPool({ host: config.db.host, user: config.db.user, password: config.db.password, database: config.db.database, connectionLimit: 10, }); } return pool; } export async function closePool() { if (pool) { await pool.end(); pool = undefined; } } ``` - [ ] **Step 6: Run test to verify it passes** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/db.test.js` Expected: PASS (2 tests) - [ ] **Step 7: Commit** ```bash git add db/schema.sql src/db.js test/db.test.js git commit -m "feat: add MariaDB schema and connection pool" ``` --- ## Task 3: Storage utilities **Files:** - Create: `src/storage.js` - Test: `test/storage.test.js` **Interfaces:** - Consumes: `config.storageDir` from Task 1. - Produces: - `export async function ensureStorageDirs(config)` — creates `/uploads` and `/outputs` if missing. - `export function uploadPath(config, id, ext)` — returns `/uploads/.` (absolute path). - `export function outputPath(config, id, ext)` — returns `/outputs/.` (absolute path). - `export async function deleteIfExists(filePath)` — deletes a file, silently no-ops if it doesn't exist. - [ ] **Step 1: Write the failing tests** Create `test/storage.test.js`: ```js import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from '../src/storage.js'; let tmpDir; let config; beforeEach(async () => { tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-storage-')); config = { storageDir: tmpDir }; }); afterEach(async () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); describe('storage', () => { it('creates uploads and outputs directories', async () => { await ensureStorageDirs(config); const uploadsStat = await fs.stat(path.join(tmpDir, 'uploads')); const outputsStat = await fs.stat(path.join(tmpDir, 'outputs')); expect(uploadsStat.isDirectory()).toBe(true); expect(outputsStat.isDirectory()).toBe(true); }); it('builds upload and output paths as .', () => { const id = '11111111-1111-4111-8111-111111111111'; expect(uploadPath(config, id, 'png')).toBe(path.join(tmpDir, 'uploads', `${id}.png`)); expect(outputPath(config, id, 'pdf')).toBe(path.join(tmpDir, 'outputs', `${id}.pdf`)); }); it('deletes an existing file', async () => { await ensureStorageDirs(config); const filePath = uploadPath(config, 'file-to-delete', 'txt'); await fs.writeFile(filePath, 'content'); await deleteIfExists(filePath); await expect(fs.stat(filePath)).rejects.toThrow(); }); it('does not throw when deleting a missing file', async () => { const filePath = uploadPath(config, 'does-not-exist', 'txt'); await expect(deleteIfExists(filePath)).resolves.toBeUndefined(); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `npx vitest run test/storage.test.js` Expected: FAIL — `src/storage.js` does not exist yet. - [ ] **Step 3: Implement `src/storage.js`** ```js import fs from 'node:fs/promises'; import path from 'node:path'; export async function ensureStorageDirs(config) { await fs.mkdir(path.join(config.storageDir, 'uploads'), { recursive: true }); await fs.mkdir(path.join(config.storageDir, 'outputs'), { recursive: true }); } export function uploadPath(config, id, ext) { return path.join(config.storageDir, 'uploads', `${id}.${ext}`); } export function outputPath(config, id, ext) { return path.join(config.storageDir, 'outputs', `${id}.${ext}`); } export async function deleteIfExists(filePath) { try { await fs.unlink(filePath); } catch (error) { if (error.code !== 'ENOENT') throw error; } } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `npx vitest run test/storage.test.js` Expected: PASS (4 tests) - [ ] **Step 5: Commit** ```bash git add src/storage.js test/storage.test.js git commit -m "feat: add storage path helpers" ``` --- ## Task 4: MIME detection utilities **Files:** - Create: `src/mime.js` - Create: `test/fixtures/sample.png` (tiny valid PNG used as a fixture) - Test: `test/mime.test.js` **Interfaces:** - Produces: - `export async function detectInputMime(filePath)` — sniffs magic bytes via `file-type`'s `fileTypeFromFile`, returns `{ ext, mime }` or `null` if undetectable. - `export function outputMimeType(targetFormat)` — looks up a static map, throws if `targetFormat` is unknown. - [ ] **Step 1: Create the PNG fixture** Create `test/fixtures/sample.png` with this exact 67-byte 1x1 transparent PNG (write as binary — do not treat as text): ```js // One-off script to generate the fixture (run once, not part of the app): // node -e "require('fs').writeFileSync('test/fixtures/sample.png', Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGNgAAIAAAUAAen63NgAAAAASUVORK5CYII=', 'base64'))" ``` Run that command from the project root. - [ ] **Step 2: Write the failing tests** Create `test/mime.test.js`: ```js import { describe, it, expect } from 'vitest'; import path from 'node:path'; import { detectInputMime, outputMimeType } from '../src/mime.js'; describe('detectInputMime', () => { it('detects PNG from magic bytes regardless of file extension', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const result = await detectInputMime(fixturePath); expect(result).toEqual({ ext: 'png', mime: 'image/png' }); }); }); describe('outputMimeType', () => { it('returns the MIME type for a known target format', () => { expect(outputMimeType('pdf')).toBe('application/pdf'); expect(outputMimeType('png')).toBe('image/png'); expect(outputMimeType('docx')).toBe( 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' ); }); it('throws for an unknown target format', () => { expect(() => outputMimeType('made-up-format')).toThrowError(/made-up-format/); }); }); ``` - [ ] **Step 3: Run tests to verify they fail** Run: `npx vitest run test/mime.test.js` Expected: FAIL — `src/mime.js` does not exist yet. - [ ] **Step 4: Implement `src/mime.js`** ```js import { fileTypeFromFile } from 'file-type'; const OUTPUT_MIME_TYPES = { jpg: 'image/jpeg', jpeg: 'image/jpeg', png: 'image/png', webp: 'image/webp', gif: 'image/gif', tiff: 'image/tiff', avif: 'image/avif', bmp: 'image/bmp', pdf: 'application/pdf', html: 'text/html', txt: 'text/plain', docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', }; export async function detectInputMime(filePath) { const result = await fileTypeFromFile(filePath); return result ?? null; } export function outputMimeType(targetFormat) { const mime = OUTPUT_MIME_TYPES[targetFormat]; if (!mime) { throw new Error(`No known MIME type for target format "${targetFormat}"`); } return mime; } ``` - [ ] **Step 5: Run tests to verify they pass** Run: `npx vitest run test/mime.test.js` Expected: PASS (3 tests) - [ ] **Step 6: Commit** ```bash git add src/mime.js test/mime.test.js test/fixtures/sample.png git commit -m "feat: add MIME sniffing and output MIME lookup" ``` --- ## Task 5: Job repository **Files:** - Create: `src/jobs/jobRepository.js` - Test: `test/jobs/jobRepository.test.js` **Interfaces:** - Consumes: `getPool(config)` from Task 2. - Produces (all functions take `pool` as first argument): - `async function createJob(pool, { id, family, sourceFormat, targetFormat, originalFilename, inputPath, inputMimeType, expiresAt })` — inserts a `pending` row, returns nothing. - `async function getJobById(pool, id)` — returns the row as a plain object (camelCase keys: `id, status, family, sourceFormat, targetFormat, originalFilename, inputPath, outputPath, inputMimeType, outputMimeType, errorMessage, createdAt, updatedAt, expiresAt`) or `null` if not found. `error_log` is intentionally NOT selected by this function — callers that need it must use `getJobErrorLog`. - `async function getJobErrorLog(pool, id)` — returns the raw `error_log` string or `null`. Used only by the worker/debug tooling, never by the public API. - `async function markProcessing(pool, id)` — sets `status = 'processing'`. - `async function markDone(pool, id, { outputPath, outputMimeType })` — sets `status = 'done'` and those two columns. - `async function markFailed(pool, id, { errorMessage, errorLog })` — sets `status = 'failed'` and both error columns. - `async function findPendingJobs(pool, limit)` — returns up to `limit` rows with `status = 'pending'`, oldest first, each mapped like `getJobById`. - `async function findExpiredJobs(pool)` — returns all rows where `expires_at < NOW()`, mapped like `getJobById`. - `async function deleteJob(pool, id)` — deletes the row. - [ ] **Step 1: Write the failing tests** Create `test/jobs/jobRepository.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import { getPool, closePool } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { createJob, getJobById, getJobErrorLog, markProcessing, markDone, markFailed, findPendingJobs, findExpiredJobs, deleteJob, } from '../../src/jobs/jobRepository.js'; let pool; beforeAll(() => { pool = getPool(loadConfig()); }); afterAll(async () => { await closePool(); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); function baseJob(overrides = {}) { return { id: overrides.id ?? '11111111-1111-4111-8111-111111111111', family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${overrides.id ?? '11111111-1111-4111-8111-111111111111'}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() + 3600 * 1000), ...overrides, }; } describe('jobRepository', () => { it('creates and retrieves a pending job', async () => { await createJob(pool, baseJob()); const job = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); expect(job.status).toBe('pending'); expect(job.family).toBe('image'); expect(job.sourceFormat).toBe('png'); expect(job.targetFormat).toBe('webp'); expect(job.originalFilename).toBe('photo.png'); expect(job.inputMimeType).toBe('image/png'); expect(job.outputPath).toBeNull(); expect(job.errorMessage).toBeNull(); }); it('returns null for an unknown id', async () => { const job = await getJobById(pool, '22222222-2222-4222-8222-222222222222'); expect(job).toBeNull(); }); it('transitions a job through processing to done', async () => { await createJob(pool, baseJob()); await markProcessing(pool, '11111111-1111-4111-8111-111111111111'); const processing = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); expect(processing.status).toBe('processing'); await markDone(pool, '11111111-1111-4111-8111-111111111111', { outputPath: '11111111-1111-4111-8111-111111111111.webp', outputMimeType: 'image/webp', }); const done = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); expect(done.status).toBe('done'); expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp'); expect(done.outputMimeType).toBe('image/webp'); }); it('marks a job failed with a short message and a separate detailed log', async () => { await createJob(pool, baseJob()); await markFailed(pool, '11111111-1111-4111-8111-111111111111', { errorMessage: 'Conversion failed, please try again', errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)', }); const job = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); expect(job.status).toBe('failed'); expect(job.errorMessage).toBe('Conversion failed, please try again'); expect(job.errorLog).toBeUndefined(); const errorLog = await getJobErrorLog(pool, '11111111-1111-4111-8111-111111111111'); expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)'); }); it('finds pending jobs oldest first, up to a limit', async () => { await createJob(pool, baseJob({ id: '33333333-3333-4333-8333-333333333333' })); await createJob(pool, baseJob({ id: '44444444-4444-4444-8444-444444444444' })); await createJob(pool, baseJob({ id: '55555555-5555-4555-8555-555555555555' })); const jobs = await findPendingJobs(pool, 2); expect(jobs).toHaveLength(2); expect(jobs[0].id).toBe('33333333-3333-4333-8333-333333333333'); expect(jobs[1].id).toBe('44444444-4444-4444-8444-444444444444'); }); it('finds expired jobs and allows deleting them', async () => { await createJob(pool, baseJob({ expiresAt: new Date(Date.now() - 1000) })); const expired = await findExpiredJobs(pool); expect(expired).toHaveLength(1); expect(expired[0].id).toBe('11111111-1111-4111-8111-111111111111'); await deleteJob(pool, '11111111-1111-4111-8111-111111111111'); const afterDelete = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); expect(afterDelete).toBeNull(); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/jobs/jobRepository.test.js` Expected: FAIL — `src/jobs/jobRepository.js` does not exist yet. - [ ] **Step 3: Implement `src/jobs/jobRepository.js`** ```js function toCamelJob(row) { if (!row) return null; return { id: row.id, status: row.status, family: row.family, sourceFormat: row.source_format, targetFormat: row.target_format, originalFilename: row.original_filename, inputPath: row.input_path, outputPath: row.output_path, inputMimeType: row.input_mime_type, outputMimeType: row.output_mime_type, errorMessage: row.error_message, createdAt: row.created_at, updatedAt: row.updated_at, expiresAt: row.expires_at, }; } export async function createJob(pool, job) { await pool.query( `INSERT INTO conversion_jobs (id, status, family, source_format, target_format, original_filename, input_path, input_mime_type, expires_at) VALUES (?, 'pending', ?, ?, ?, ?, ?, ?, ?)`, [ job.id, job.family, job.sourceFormat, job.targetFormat, job.originalFilename, job.inputPath, job.inputMimeType, job.expiresAt, ] ); } export async function getJobById(pool, id) { const rows = await pool.query( `SELECT id, status, family, source_format, target_format, original_filename, input_path, output_path, input_mime_type, output_mime_type, error_message, created_at, updated_at, expires_at FROM conversion_jobs WHERE id = ?`, [id] ); return toCamelJob(rows[0]); } export async function getJobErrorLog(pool, id) { const rows = await pool.query('SELECT error_log FROM conversion_jobs WHERE id = ?', [id]); return rows[0]?.error_log ?? null; } export async function markProcessing(pool, id) { await pool.query("UPDATE conversion_jobs SET status = 'processing' WHERE id = ?", [id]); } export async function markDone(pool, id, { outputPath, outputMimeType }) { await pool.query( "UPDATE conversion_jobs SET status = 'done', output_path = ?, output_mime_type = ? WHERE id = ?", [outputPath, outputMimeType, id] ); } export async function markFailed(pool, id, { errorMessage, errorLog }) { await pool.query( "UPDATE conversion_jobs SET status = 'failed', error_message = ?, error_log = ? WHERE id = ?", [errorMessage, errorLog, id] ); } export async function findPendingJobs(pool, limit) { const rows = await pool.query( `SELECT id, status, family, source_format, target_format, original_filename, input_path, output_path, input_mime_type, output_mime_type, error_message, created_at, updated_at, expires_at FROM conversion_jobs WHERE status = 'pending' ORDER BY created_at ASC LIMIT ?`, [limit] ); return rows.map(toCamelJob); } export async function findExpiredJobs(pool) { const rows = await pool.query( `SELECT id, status, family, source_format, target_format, original_filename, input_path, output_path, input_mime_type, output_mime_type, error_message, created_at, updated_at, expires_at FROM conversion_jobs WHERE expires_at < NOW()` ); return rows.map(toCamelJob); } export async function deleteJob(pool, id) { await pool.query('DELETE FROM conversion_jobs WHERE id = ?', [id]); } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/jobs/jobRepository.test.js` Expected: PASS (6 tests) - [ ] **Step 5: Commit** ```bash git add src/jobs/jobRepository.js test/jobs/jobRepository.test.js git commit -m "feat: add conversion_jobs repository" ``` --- ## Task 6: Converter registry **Files:** - Create: `src/converters/registry.js` - Test: `test/converters/registry.test.js` **Interfaces:** - Produces: - `export function register({ family, sourceFormat, targetFormat, convert })` — adds an entry. `convert` has signature `async (inputPath, outputPath) => void`. - `export function resolve(sourceFormat, targetFormat)` — returns the registered `{ family, convert }` or `null` if no converter exists for that pair. - `export function listTargetFormats(sourceFormat)` — returns an array of every `targetFormat` registered for that `sourceFormat` (empty array if none). - `export function _resetForTests()` — clears the registry (test-only helper). This module holds no conversion logic itself — Tasks 7-10 call `register` at import time to populate it. - [ ] **Step 1: Write the failing tests** Create `test/converters/registry.test.js`: ```js import { describe, it, expect, beforeEach } from 'vitest'; import { register, resolve, listTargetFormats, _resetForTests } from '../../src/converters/registry.js'; beforeEach(() => { _resetForTests(); }); describe('converter registry', () => { it('resolves a registered source/target pair', () => { const convert = async () => {}; register({ family: 'image', sourceFormat: 'png', targetFormat: 'webp', convert }); const entry = resolve('png', 'webp'); expect(entry.family).toBe('image'); expect(entry.convert).toBe(convert); }); it('returns null for an unregistered pair', () => { expect(resolve('png', 'made-up')).toBeNull(); }); it('lists all target formats registered for a source format', () => { register({ family: 'image', sourceFormat: 'png', targetFormat: 'webp', convert: async () => {} }); register({ family: 'image', sourceFormat: 'png', targetFormat: 'jpg', convert: async () => {} }); register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: async () => {} }); expect(listTargetFormats('png').sort()).toEqual(['jpg', 'webp']); expect(listTargetFormats('docx')).toEqual(['pdf']); expect(listTargetFormats('unknown-format')).toEqual([]); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `npx vitest run test/converters/registry.test.js` Expected: FAIL — `src/converters/registry.js` does not exist yet. - [ ] **Step 3: Implement `src/converters/registry.js`** ```js let entries = new Map(); function key(sourceFormat, targetFormat) { return `${sourceFormat}->${targetFormat}`; } export function register({ family, sourceFormat, targetFormat, convert }) { entries.set(key(sourceFormat, targetFormat), { family, convert }); } export function resolve(sourceFormat, targetFormat) { return entries.get(key(sourceFormat, targetFormat)) ?? null; } export function listTargetFormats(sourceFormat) { const prefix = `${sourceFormat}->`; return [...entries.keys()] .filter((k) => k.startsWith(prefix)) .map((k) => k.slice(prefix.length)); } export function _resetForTests() { entries = new Map(); } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `npx vitest run test/converters/registry.test.js` Expected: PASS (3 tests) - [ ] **Step 5: Commit** ```bash git add src/converters/registry.js test/converters/registry.test.js git commit -m "feat: add converter registry" ``` --- ## Task 7: Image converter family (sharp) **Files:** - Create: `src/converters/image.js` - Test: `test/converters/image.test.js` **Interfaces:** - Consumes: `register` from Task 6's `src/converters/registry.js`. - Produces: `export function registerImageConverters()` — call once at process startup (from `app.js` and `worker.js` in later tasks) to populate the registry with every `{sourceFormat, targetFormat}` pair among `jpg, jpeg, png, webp, gif, tiff, avif` (source ≠ target), family `'image'`. Each entry's `convert(inputPath, outputPath)` runs `sharp(inputPath).toFormat().toFile(outputPath)`. Note: `bmp` is intentionally excluded from this task — `sharp` does not support BMP as an output format (confirmed against the official docs: output formats are JPEG, PNG, WebP, GIF, AVIF, TIFF). BMP support is out of scope for v1 per the design spec. - [ ] **Step 1: Write the failing test** Create `test/converters/image.test.js`: ```js import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { registerImageConverters } from '../../src/converters/image.js'; import { resolve, listTargetFormats } from '../../src/converters/registry.js'; import { detectInputMime } from '../../src/mime.js'; let tmpDir; beforeAll(async () => { registerImageConverters(); tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-image-')); }); afterAll(async () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); describe('image converters', () => { it('registers every pair among the supported formats', () => { const targets = listTargetFormats('png').sort(); expect(targets).toEqual(['avif', 'gif', 'jpeg', 'jpg', 'tiff', 'webp']); }); it('converts a PNG fixture to WebP', async () => { const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const outputPath = path.join(tmpDir, 'output.webp'); const entry = resolve('png', 'webp'); await entry.convert(inputPath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('image/webp'); }); it('converts a PNG fixture to JPG', async () => { const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const outputPath = path.join(tmpDir, 'output.jpg'); const entry = resolve('png', 'jpg'); await entry.convert(inputPath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('image/jpeg'); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npx vitest run test/converters/image.test.js` Expected: FAIL — `src/converters/image.js` does not exist yet. - [ ] **Step 3: Implement `src/converters/image.js`** ```js import sharp from 'sharp'; import { register } from './registry.js'; const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif']; function sharpFormatName(format) { return format === 'jpg' ? 'jpeg' : format; } export function registerImageConverters() { for (const sourceFormat of IMAGE_FORMATS) { for (const targetFormat of IMAGE_FORMATS) { if (sourceFormat === targetFormat) continue; register({ family: 'image', sourceFormat, targetFormat, convert: async (inputPath, outputPath) => { await sharp(inputPath).toFormat(sharpFormatName(targetFormat)).toFile(outputPath); }, }); } } } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npx vitest run test/converters/image.test.js` Expected: PASS (3 tests) - [ ] **Step 5: Commit** ```bash git add src/converters/image.js test/converters/image.test.js git commit -m "feat: add image converter family (sharp)" ``` --- ## Task 8: Image → PDF converter **Files:** - Create: `src/converters/imageToPdf.js` - Test: `test/converters/imageToPdf.test.js` **Interfaces:** - Consumes: `register` from Task 6, `sharp` (already a dependency from Task 7). - Produces: `export function registerImageToPdfConverter()` — registers `{sourceFormat, targetFormat: 'pdf'}` for every format in `jpg, jpeg, png, webp, gif, tiff, avif`, family `'image'`. `convert(inputPath, outputPath)` normalizes the input to a PNG buffer via `sharp(inputPath).png().toBuffer()`, then creates a one-page PDF sized to the image via `pdf-lib`, embeds the PNG, and writes it with `fs.writeFile`. - [ ] **Step 1: Write the failing test** Create `test/converters/imageToPdf.test.js`: ```js import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { registerImageToPdfConverter } from '../../src/converters/imageToPdf.js'; import { resolve, listTargetFormats } from '../../src/converters/registry.js'; import { detectInputMime } from '../../src/mime.js'; let tmpDir; beforeAll(async () => { registerImageToPdfConverter(); tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-image-to-pdf-')); }); afterAll(async () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); describe('image to PDF converter', () => { it('registers pdf as a target for every image format', () => { expect(listTargetFormats('png')).toContain('pdf'); expect(listTargetFormats('jpg')).toContain('pdf'); }); it('converts a PNG fixture into a valid PDF', async () => { const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const outputPath = path.join(tmpDir, 'output.pdf'); const entry = resolve('png', 'pdf'); await entry.convert(inputPath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('application/pdf'); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npx vitest run test/converters/imageToPdf.test.js` Expected: FAIL — `src/converters/imageToPdf.js` does not exist yet. - [ ] **Step 3: Implement `src/converters/imageToPdf.js`** ```js import fs from 'node:fs/promises'; import sharp from 'sharp'; import { PDFDocument } from 'pdf-lib'; import { register } from './registry.js'; const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif']; async function convert(inputPath, outputPath) { const pngBuffer = await sharp(inputPath).png().toBuffer(); const metadata = await sharp(pngBuffer).metadata(); const pdfDoc = await PDFDocument.create(); const page = pdfDoc.addPage([metadata.width, metadata.height]); const embeddedImage = await pdfDoc.embedPng(pngBuffer); page.drawImage(embeddedImage, { x: 0, y: 0, width: metadata.width, height: metadata.height, }); const pdfBytes = await pdfDoc.save(); await fs.writeFile(outputPath, pdfBytes); } export function registerImageToPdfConverter() { for (const sourceFormat of IMAGE_FORMATS) { register({ family: 'image', sourceFormat, targetFormat: 'pdf', convert }); } } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npx vitest run test/converters/imageToPdf.test.js` Expected: PASS (2 tests) - [ ] **Step 5: Commit** ```bash git add src/converters/imageToPdf.js test/converters/imageToPdf.test.js git commit -m "feat: add image to PDF converter" ``` --- ## Task 9: Document converters, part A — DOCX→HTML, TXT/HTML→PDF, DOCX→PDF **Files:** - Create: `src/converters/document.js` - Test: `test/converters/document.test.js` **Interfaces:** - Consumes: `register` from Task 6. - Produces (all added to `src/converters/document.js`, all called from one exported `export function registerDocumentConverters()`): - `docx → html` via `mammoth.convertToHtml({ path: inputPath })`, writing `result.value` to `outputPath`. - `txt → pdf` via reading the text file, wrapping it in a minimal HTML template (escaping `<`, `>`, `&`), and rendering with Puppeteer. - `html → pdf` via reading the HTML file as-is and rendering with Puppeteer. - `docx → pdf` by chaining: `mammoth.convertToHtml` then the same Puppeteer rendering used for `html → pdf`. - A private helper `renderHtmlToPdf(html, outputPath)` shared by the three PDF-producing paths above, using `puppeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox'] })`, `page.setContent(html, { waitUntil: 'networkidle0' })`, `page.pdf({ path: outputPath, format: 'A4', printBackground: true })`, always closing the browser in a `finally` block. This task does not yet cover `pdf → text/html` or `pdf → docx` — those are Task 10. - [ ] **Step 1: Write the failing test** Create `test/converters/document.test.js`: ```js import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { Document, Paragraph, TextRun, Packer } from 'docx'; import { registerDocumentConverters } from '../../src/converters/document.js'; import { resolve } from '../../src/converters/registry.js'; import { detectInputMime } from '../../src/mime.js'; let tmpDir; let docxFixturePath; beforeAll(async () => { registerDocumentConverters(); tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-document-')); const doc = new Document({ sections: [ { children: [new Paragraph({ children: [new TextRun('Hello from the fixture document')] })], }, ], }); const buffer = await Packer.toBuffer(doc); docxFixturePath = path.join(tmpDir, 'fixture.docx'); await fs.writeFile(docxFixturePath, buffer); }); afterAll(async () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); describe('document converters', () => { it('converts DOCX to HTML containing the source text', async () => { const outputPath = path.join(tmpDir, 'output.html'); const entry = resolve('docx', 'html'); await entry.convert(docxFixturePath, outputPath); const html = await fs.readFile(outputPath, 'utf8'); expect(html).toContain('Hello from the fixture document'); }, 20000); it('converts TXT to a valid PDF', async () => { const inputPath = path.join(tmpDir, 'fixture.txt'); await fs.writeFile(inputPath, 'Plain text content for the PDF'); const outputPath = path.join(tmpDir, 'from-txt.pdf'); const entry = resolve('txt', 'pdf'); await entry.convert(inputPath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('application/pdf'); }, 20000); it('converts HTML to a valid PDF', async () => { const inputPath = path.join(tmpDir, 'fixture.html'); await fs.writeFile(inputPath, '

Hello HTML

'); const outputPath = path.join(tmpDir, 'from-html.pdf'); const entry = resolve('html', 'pdf'); await entry.convert(inputPath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('application/pdf'); }, 20000); it('converts DOCX to a valid PDF', async () => { const outputPath = path.join(tmpDir, 'from-docx.pdf'); const entry = resolve('docx', 'pdf'); await entry.convert(docxFixturePath, outputPath); const detected = await detectInputMime(outputPath); expect(detected.mime).toBe('application/pdf'); }, 20000); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npx vitest run test/converters/document.test.js` Expected: FAIL — `src/converters/document.js` does not exist yet. - [ ] **Step 3: Implement the Puppeteer rendering helper and DOCX/TXT/HTML converters in `src/converters/document.js`** ```js import fs from 'node:fs/promises'; import mammoth from 'mammoth'; import puppeteer from 'puppeteer'; import { register } from './registry.js'; async function renderHtmlToPdf(html, outputPath) { const browser = await puppeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox'], }); try { const page = await browser.newPage(); await page.setContent(html, { waitUntil: 'networkidle0' }); await page.pdf({ path: outputPath, format: 'A4', printBackground: true }); } finally { await browser.close(); } } function escapeHtml(text) { return text.replace(/&/g, '&').replace(//g, '>'); } async function convertDocxToHtml(inputPath, outputPath) { const result = await mammoth.convertToHtml({ path: inputPath }); await fs.writeFile(outputPath, result.value); } async function convertTxtToPdf(inputPath, outputPath) { const text = await fs.readFile(inputPath, 'utf8'); const html = `
${escapeHtml(text)}
`; await renderHtmlToPdf(html, outputPath); } async function convertHtmlToPdf(inputPath, outputPath) { const html = await fs.readFile(inputPath, 'utf8'); await renderHtmlToPdf(html, outputPath); } async function convertDocxToPdf(inputPath, outputPath) { const result = await mammoth.convertToHtml({ path: inputPath }); await renderHtmlToPdf(result.value, outputPath); } export function registerDocumentConverters() { register({ family: 'document', sourceFormat: 'docx', targetFormat: 'html', convert: convertDocxToHtml }); register({ family: 'document', sourceFormat: 'txt', targetFormat: 'pdf', convert: convertTxtToPdf }); register({ family: 'document', sourceFormat: 'html', targetFormat: 'pdf', convert: convertHtmlToPdf }); register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: convertDocxToPdf }); } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npx vitest run test/converters/document.test.js` Expected: PASS (4 tests). Puppeteer launches a real headless Chromium per test — this is slower than the other unit tests (hence the 20000ms timeouts); this is expected. - [ ] **Step 5: Commit** ```bash git add src/converters/document.js test/converters/document.test.js git commit -m "feat: add DOCX/TXT/HTML document converters" ``` --- ## Task 10: Document converters, part B — PDF→text/HTML, PDF→DOCX (best-effort) **Files:** - Modify: `src/converters/document.js` (add to the same file and to `registerDocumentConverters()`) - Test: `test/converters/documentFromPdf.test.js` **Interfaces:** - Consumes: `register` from Task 6, the fixture-generation pattern from Task 9 (generate inputs at test time instead of committing binary fixtures). - Produces, added to `src/converters/document.js`: - A private helper `async function extractPdfPageTexts(inputPath)` returning `string[]`, one entry per page, using `pdfjs-dist/legacy/build/pdf.mjs`'s `getDocument({ data }).promise` and `page.getTextContent()`. - `pdf → txt`: joins page texts with `\n\n` and writes as plain text. - `pdf → html`: wraps each page's text in a `

` tag (HTML-escaped) and writes a minimal HTML document. - `pdf → docx`: builds one `docx` `Paragraph` per page text and writes via `Packer.toBuffer`. This is explicitly a **best-effort text reconstruction** — no attempt is made to preserve original layout, tables, or image positions, matching the limitation documented in the design spec. - [ ] **Step 1: Write the failing test** Create `test/converters/documentFromPdf.test.js`: ```js import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { PDFDocument, StandardFonts } from 'pdf-lib'; import { registerDocumentConverters } from '../../src/converters/document.js'; import { resolve } from '../../src/converters/registry.js'; let tmpDir; let pdfFixturePath; beforeAll(async () => { registerDocumentConverters(); tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-document-from-pdf-')); const pdfDoc = await PDFDocument.create(); const page = pdfDoc.addPage([600, 400]); const font = await pdfDoc.embedFont(StandardFonts.Helvetica); page.drawText('Extractable fixture text', { x: 50, y: 350, size: 24, font }); const bytes = await pdfDoc.save(); pdfFixturePath = path.join(tmpDir, 'fixture.pdf'); await fs.writeFile(pdfFixturePath, bytes); }); afterAll(async () => { await fs.rm(tmpDir, { recursive: true, force: true }); }); describe('PDF source document converters', () => { it('extracts text from PDF to TXT', async () => { const outputPath = path.join(tmpDir, 'output.txt'); const entry = resolve('pdf', 'txt'); await entry.convert(pdfFixturePath, outputPath); const text = await fs.readFile(outputPath, 'utf8'); expect(text).toContain('Extractable fixture text'); }); it('extracts text from PDF to HTML', async () => { const outputPath = path.join(tmpDir, 'output.html'); const entry = resolve('pdf', 'html'); await entry.convert(pdfFixturePath, outputPath); const html = await fs.readFile(outputPath, 'utf8'); expect(html).toContain('Extractable fixture text'); expect(html).toContain('

'); }); it('reconstructs PDF text into a DOCX (best-effort)', async () => { const outputPath = path.join(tmpDir, 'output.docx'); const entry = resolve('pdf', 'docx'); await entry.convert(pdfFixturePath, outputPath); const stat = await fs.stat(outputPath); expect(stat.size).toBeGreaterThan(0); const mammoth = await import('mammoth'); const result = await mammoth.default.convertToHtml({ path: outputPath }); expect(result.value).toContain('Extractable fixture text'); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npx vitest run test/converters/documentFromPdf.test.js` Expected: FAIL — `pdf` is not yet a registered source format. - [ ] **Step 3: Add the PDF-source converters to `src/converters/document.js`** Add these imports at the top of `src/converters/document.js` (alongside the existing ones): ```js import * as pdfjsLib from 'pdfjs-dist/legacy/build/pdf.mjs'; import { Document, Paragraph, TextRun, Packer } from 'docx'; ``` Add these functions to the same file: ```js async function extractPdfPageTexts(inputPath) { const data = new Uint8Array(await fs.readFile(inputPath)); const doc = await pdfjsLib.getDocument({ data }).promise; const pageTexts = []; for (let pageNum = 1; pageNum <= doc.numPages; pageNum += 1) { const page = await doc.getPage(pageNum); const content = await page.getTextContent(); pageTexts.push(content.items.map((item) => item.str).join(' ')); } return pageTexts; } async function convertPdfToTxt(inputPath, outputPath) { const pageTexts = await extractPdfPageTexts(inputPath); await fs.writeFile(outputPath, pageTexts.join('\n\n')); } async function convertPdfToHtml(inputPath, outputPath) { const pageTexts = await extractPdfPageTexts(inputPath); const body = pageTexts.map((text) => `

${escapeHtml(text)}

`).join('\n'); await fs.writeFile(outputPath, `${body}`); } async function convertPdfToDocx(inputPath, outputPath) { const pageTexts = await extractPdfPageTexts(inputPath); const doc = new Document({ sections: [ { children: pageTexts.map((text) => new Paragraph({ children: [new TextRun(text)] })), }, ], }); const buffer = await Packer.toBuffer(doc); await fs.writeFile(outputPath, buffer); } ``` Update `registerDocumentConverters()` to also register these three: ```js export function registerDocumentConverters() { register({ family: 'document', sourceFormat: 'docx', targetFormat: 'html', convert: convertDocxToHtml }); register({ family: 'document', sourceFormat: 'txt', targetFormat: 'pdf', convert: convertTxtToPdf }); register({ family: 'document', sourceFormat: 'html', targetFormat: 'pdf', convert: convertHtmlToPdf }); register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: convertDocxToPdf }); register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'txt', convert: convertPdfToTxt }); register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'html', convert: convertPdfToHtml }); register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'docx', convert: convertPdfToDocx }); } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npx vitest run test/converters/documentFromPdf.test.js` Expected: PASS (3 tests) - [ ] **Step 5: Run the full document test suite to confirm no regression** Run: `npx vitest run test/converters/document.test.js test/converters/documentFromPdf.test.js` Expected: PASS (7 tests total) - [ ] **Step 6: Commit** ```bash git add src/converters/document.js test/converters/documentFromPdf.test.js git commit -m "feat: add best-effort PDF source converters (txt, html, docx)" ``` --- ## Task 11: Input format validation helper (magic-byte vs. declared extension) **Files:** - Modify: `src/mime.js` - Test: `test/mime.test.js` (extend) **Interfaces:** - Produces: `export async function resolveInputFormat(filePath, declaredFormat)` in `src/mime.js` — returns `{ mime: string, valid: boolean }`. - Sniffs magic bytes via `detectInputMime`. - If a signature is found, `valid` is `true` only when the detected extension matches `declaredFormat` (treating `jpg`/`jpeg` as equivalent — `file-type` returns `jpg` for JPEG, but both are valid declared extensions in this app). - If no signature is found (expected for `txt` and `html`, which have no magic bytes), falls back to trusting `declaredFormat` **only** when it is `txt` or `html`, returning the matching static MIME type. Any other undetectable format is `valid: false`. - [ ] **Step 1: Write the failing tests (append to `test/mime.test.js`)** Add to `test/mime.test.js`: ```js import { resolveInputFormat } from '../src/mime.js'; ``` (add this import alongside the existing ones at the top of the file), then add: ```js describe('resolveInputFormat', () => { it('accepts a PNG file declared as png', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const result = await resolveInputFormat(fixturePath, 'png'); expect(result).toEqual({ mime: 'image/png', valid: true }); }); it('rejects a PNG file declared as a different format', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const result = await resolveInputFormat(fixturePath, 'pdf'); expect(result.valid).toBe(false); }); it('treats jpg and jpeg as equivalent declared formats', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const result = await resolveInputFormat(fixturePath, 'png'); expect(result.valid).toBe(true); }); it('trusts the declared format for undetectable txt files', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.txt'); await fs.writeFile(fixturePath, 'plain text, no magic bytes'); const result = await resolveInputFormat(fixturePath, 'txt'); expect(result).toEqual({ mime: 'text/plain', valid: true }); await fs.unlink(fixturePath); }); it('rejects an undetectable file declared as a binary format', async () => { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample-fake.png'); await fs.writeFile(fixturePath, 'this is not really a PNG'); const result = await resolveInputFormat(fixturePath, 'png'); expect(result.valid).toBe(false); await fs.unlink(fixturePath); }); }); ``` Also add `import fs from 'node:fs/promises';` at the top of `test/mime.test.js` if not already present. - [ ] **Step 2: Run tests to verify they fail** Run: `npx vitest run test/mime.test.js` Expected: FAIL — `resolveInputFormat` is not exported yet. - [ ] **Step 3: Implement `resolveInputFormat` in `src/mime.js`** Add to `src/mime.js`: ```js const UNDETECTABLE_TEXT_FORMATS = { txt: 'text/plain', html: 'text/html', }; function normalizeFormat(format) { return format === 'jpg' ? 'jpeg' : format; } export async function resolveInputFormat(filePath, declaredFormat) { const detected = await detectInputMime(filePath); if (!detected) { const fallbackMime = UNDETECTABLE_TEXT_FORMATS[declaredFormat]; if (fallbackMime) { return { mime: fallbackMime, valid: true }; } return { mime: null, valid: false }; } const valid = normalizeFormat(detected.ext) === normalizeFormat(declaredFormat); return { mime: detected.mime, valid }; } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `npx vitest run test/mime.test.js` Expected: PASS (8 tests total) - [ ] **Step 5: Commit** ```bash git add src/mime.js test/mime.test.js git commit -m "feat: validate declared source format against sniffed magic bytes" ``` --- ## Task 12: Express app — `POST /api/jobs` and `GET /api/formats` **Files:** - Create: `src/app.js` - Test: `test/api/jobs.test.js` **Interfaces:** - Consumes: `resolveInputFormat` (Task 11), `resolve`/`listTargetFormats` (Task 6), `registerImageConverters`/`registerImageToPdfConverter`/`registerDocumentConverters` (Tasks 7-10), `createJob`/`getJobById` (Task 5), `ensureStorageDirs`/`deleteIfExists` (Task 3), `getPool` (Task 2). - Produces: `export function createApp(config, pool)` — returns a configured Express `app` (does not call `.listen`). Registers all converters as a side effect the first time it's called (safe to call once per process). **Request contract for `POST /api/jobs`:** - `multipart/form-data` with a `files` field (one or more files, field name repeated) and a `targetFormats` field: a JSON-encoded array of strings, same length and order as `files`. - Response `201`: `{ "jobs": [ { "file": "photo.png", "id": "", "status": "pending" } | { "file": "photo.png", "error": "" }, ... ] }` — one entry per uploaded file, in the same order. A file-level problem never fails the whole request; only a malformed request (no files, `targetFormats` missing/wrong length) returns `400` with no jobs created. - [ ] **Step 1: Write the failing tests** Create `test/api/jobs.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { createApp } from '../../src/app.js'; import { getPool, closePool } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { getJobById } from '../../src/jobs/jobRepository.js'; import { ensureStorageDirs } from '../../src/storage.js'; let app; let pool; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) }; await ensureStorageDirs(config); pool = getPool(config); app = createApp(config, pool); }); afterAll(async () => { await closePool(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); describe('GET /api/formats', () => { it('lists valid target formats for a known source format', async () => { const response = await request(app).get('/api/formats').query({ source: 'png' }); expect(response.status).toBe(200); expect(response.body.targets).toContain('webp'); expect(response.body.targets).toContain('pdf'); }); it('returns an empty list for an unknown source format', async () => { const response = await request(app).get('/api/formats').query({ source: 'made-up' }); expect(response.status).toBe(200); expect(response.body.targets).toEqual([]); }); }); describe('POST /api/jobs', () => { it('creates a pending job for a valid image upload', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs).toHaveLength(1); expect(response.body.jobs[0].status).toBe('pending'); expect(response.body.jobs[0].file).toBe('photo.png'); const job = await getJobById(pool, response.body.jobs[0].id); expect(job.status).toBe('pending'); expect(job.sourceFormat).toBe('png'); expect(job.targetFormat).toBe('webp'); expect(job.originalFilename).toBe('photo.png'); expect(job.inputMimeType).toBe('image/png'); }); it('rejects a file whose content does not match its extension, without failing the whole batch', async () => { const fakePath = path.join(config.storageDir, 'fake.png'); await fs.writeFile(fakePath, 'not actually a png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fakePath, 'fake.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/does not match/); await fs.unlink(fakePath); }); it('rejects an unsupported source/target pair', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['made-up-format'])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(201); expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/); }); it('returns 400 when targetFormats length does not match the number of files', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify([])) .attach('files', fixturePath, 'photo.png'); expect(response.status).toBe(400); }); it('returns 400 when no files are uploaded', async () => { const response = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify([])); expect(response.status).toBe(400); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobs.test.js` Expected: FAIL — `src/app.js` does not exist yet. - [ ] **Step 3: Implement `src/app.js`** ```js import path from 'node:path'; import express from 'express'; import multer from 'multer'; import { rateLimit } from 'express-rate-limit'; import { v4 as uuidv4 } from 'uuid'; import { resolve as resolveConverter, listTargetFormats } from './converters/registry.js'; import { registerImageConverters } from './converters/image.js'; import { registerImageToPdfConverter } from './converters/imageToPdf.js'; import { registerDocumentConverters } from './converters/document.js'; import { resolveInputFormat } from './mime.js'; import { deleteIfExists } from './storage.js'; import { createJob, getJobById } from './jobs/jobRepository.js'; let convertersRegistered = false; function registerAllConverters() { if (convertersRegistered) return; registerImageConverters(); registerImageToPdfConverter(); registerDocumentConverters(); convertersRegistered = true; } export function createApp(config, pool) { registerAllConverters(); const app = express(); const storage = multer.diskStorage({ destination: (req, file, cb) => cb(null, path.join(config.storageDir, 'uploads')), filename: (req, file, cb) => { const ext = path.extname(file.originalname).slice(1).toLowerCase(); cb(null, `${uuidv4()}.${ext}`); }, }); const upload = multer({ storage, limits: { fileSize: config.maxFileSizeMb * 1024 * 1024, files: 10, }, }); const jobsRateLimiter = rateLimit({ windowMs: config.rateLimitWindowMinutes * 60 * 1000, limit: config.rateLimitMaxJobs, standardHeaders: 'draft-8', legacyHeaders: false, }); app.get('/api/formats', (req, res) => { const source = String(req.query.source ?? '').toLowerCase(); res.json({ targets: listTargetFormats(source) }); }); app.post('/api/jobs', jobsRateLimiter, upload.array('files', 10), async (req, res) => { if (!req.files || req.files.length === 0) { return res.status(400).json({ error: 'No files uploaded' }); } let targetFormats; try { targetFormats = JSON.parse(req.body.targetFormats ?? '[]'); } catch { return res.status(400).json({ error: 'targetFormats must be a JSON array' }); } if (!Array.isArray(targetFormats) || targetFormats.length !== req.files.length) { return res.status(400).json({ error: 'targetFormats must have one entry per uploaded file' }); } const results = []; for (let i = 0; i < req.files.length; i += 1) { const file = req.files[i]; const targetFormat = targetFormats[i]; const id = path.basename(file.filename, path.extname(file.filename)); const sourceFormat = path.extname(file.filename).slice(1).toLowerCase(); const { mime, valid } = await resolveInputFormat(file.path, sourceFormat); if (!valid) { await deleteIfExists(file.path); results.push({ file: file.originalname, error: 'File content does not match its extension' }); continue; } const registryEntry = resolveConverter(sourceFormat, targetFormat); if (!registryEntry) { await deleteIfExists(file.path); results.push({ file: file.originalname, error: `Unsupported conversion: ${sourceFormat} to ${targetFormat}`, }); continue; } const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); await createJob(pool, { id, family: registryEntry.family, sourceFormat, targetFormat, originalFilename: file.originalname, inputPath: file.filename, inputMimeType: mime, expiresAt, }); results.push({ file: file.originalname, id, status: 'pending' }); } res.status(201).json({ jobs: results }); }); return app; } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobs.test.js` Expected: PASS (7 tests) - [ ] **Step 5: Commit** ```bash git add src/app.js test/api/jobs.test.js git commit -m "feat: add POST /api/jobs and GET /api/formats endpoints" ``` --- ## Task 13: `GET /api/jobs/:id` and `GET /api/jobs/:id/download` **Files:** - Modify: `src/app.js` (add two routes) - Test: `test/api/jobStatus.test.js` **Interfaces:** - Consumes: `getJobById` (Task 5), `outputPath` (Task 3). - `GET /api/jobs/:id` → `200` with `{ id, status, originalFilename, sourceFormat, targetFormat, errorMessage }` (never `errorLog`), or `404` if the id doesn't exist. - `GET /api/jobs/:id/download` → streams the file with `Content-Type` set from `output_mime_type` and `Content-Disposition: attachment` carrying `original_filename` (RFC 5987-encoded to safely support non-ASCII names and prevent header injection). `404` if the id doesn't exist, `409` if the job exists but isn't `done` yet. - [ ] **Step 1: Write the failing tests** Create `test/api/jobStatus.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { createApp } from '../../src/app.js'; import { getPool, closePool } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { createJob, markDone } from '../../src/jobs/jobRepository.js'; import { ensureStorageDirs, outputPath } from '../../src/storage.js'; let app; let pool; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) }; await ensureStorageDirs(config); pool = getPool(config); app = createApp(config, pool); }); afterAll(async () => { await closePool(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); function baseJob(id) { return { id, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'holiday photo.png', inputPath: `${id}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() + 3600 * 1000), }; } describe('GET /api/jobs/:id', () => { it('returns job status without the error log field', async () => { const id = '66666666-6666-4666-8666-666666666666'; await createJob(pool, baseJob(id)); const response = await request(app).get(`/api/jobs/${id}`); expect(response.status).toBe(200); expect(response.body.status).toBe('pending'); expect(response.body.originalFilename).toBe('holiday photo.png'); expect(response.body.errorLog).toBeUndefined(); }); it('returns 404 for an unknown id', async () => { const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000'); expect(response.status).toBe(404); }); }); describe('GET /api/jobs/:id/download', () => { it('streams the converted file with correct headers once done', async () => { const id = '77777777-7777-4777-8777-777777777777'; await createJob(pool, baseJob(id)); const filePath = outputPath(config, id, 'webp'); await fs.writeFile(filePath, Buffer.from('fake webp bytes')); await markDone(pool, id, { outputPath: `${id}.webp`, outputMimeType: 'image/webp' }); const response = await request(app).get(`/api/jobs/${id}/download`); expect(response.status).toBe(200); expect(response.headers['content-type']).toBe('image/webp'); expect(response.headers['content-disposition']).toContain('holiday photo.png'); expect(response.body ?? response.text).toBeDefined(); }); it('returns 409 when the job is not done yet', async () => { const id = '88888888-8888-4888-8888-888888888888'; await createJob(pool, baseJob(id)); const response = await request(app).get(`/api/jobs/${id}/download`); expect(response.status).toBe(409); }); it('returns 404 for an unknown id', async () => { const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download'); expect(response.status).toBe(404); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobStatus.test.js` Expected: FAIL — routes don't exist yet. - [ ] **Step 3: Add the two routes to `src/app.js`** Add this import at the top of `src/app.js`: ```js import fs from 'node:fs'; import { outputPath } from './storage.js'; ``` Add this helper function and the two routes inside `createApp`, before `return app;`: ```js function contentDispositionHeader(filename) { const asciiFallback = filename.replace(/[^\x20-\x7E]/g, '_').replace(/"/g, "'"); const encoded = encodeURIComponent(filename); return `attachment; filename="${asciiFallback}"; filename*=UTF-8''${encoded}`; } ``` ```js app.get('/api/jobs/:id', async (req, res) => { const job = await getJobById(pool, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } res.json({ id: job.id, status: job.status, originalFilename: job.originalFilename, sourceFormat: job.sourceFormat, targetFormat: job.targetFormat, errorMessage: job.errorMessage, }); }); app.get('/api/jobs/:id/download', async (req, res) => { const job = await getJobById(pool, req.params.id); if (!job) { return res.status(404).json({ error: 'Job not found' }); } if (job.status !== 'done') { return res.status(409).json({ error: `Job is not ready yet (status: ${job.status})` }); } const filePath = outputPath(config, job.id, job.targetFormat); res.set('Content-Type', job.outputMimeType); res.set('Content-Disposition', contentDispositionHeader(job.originalFilename)); fs.createReadStream(filePath).pipe(res); }); ``` **Note on error handling:** `npm install express` (Task 1) installs the current major version, Express 5, where an async route handler's rejected promise is automatically forwarded to error-handling middleware (no manual `try/catch`-to-`next` plumbing needed) — confirmed against the official Express 5 migration guide. Add the error-handling middleware itself now, as the last thing registered in `createApp`, so it can catch errors from every route defined so far (Task 17 will insert its static-file serving *before* this middleware, never after — an error handler registered before a route can't catch that route's errors): ```js app.use((err, req, res, next) => { console.error('Unhandled API error:', err); res.status(500).json({ error: 'Internal server error' }); }); ``` Add this immediately before `return app;` at the end of `createApp`. Add this test to `test/api/jobStatus.test.js`, in a new `describe` block: ```js describe('unexpected server errors', () => { it('returns a generic 500 without leaking internal error details', async () => { const originalQuery = pool.query.bind(pool); pool.query = () => Promise.reject(new Error('connection reset by peer')); const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999'); pool.query = originalQuery; expect(response.status).toBe(500); expect(response.body.error).toBe('Internal server error'); expect(JSON.stringify(response.body)).not.toContain('connection reset by peer'); }); }); ``` - [ ] **Step 4: Run tests to verify they pass** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobStatus.test.js` Expected: PASS (6 tests) - [ ] **Step 5: Run the full API test suite to confirm no regression** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api` Expected: PASS (13 tests total) - [ ] **Step 6: Commit** ```bash git add src/app.js test/api/jobStatus.test.js git commit -m "feat: add job status and download endpoints" ``` --- ## Task 14: Server entry point **Files:** - Create: `src/server.js` - Modify: `package.json` (fix the `start` script from Task 1, which pointed at a file that was never created) **Interfaces:** - Consumes: `loadConfig` (Task 1), `ensureStorageDirs` (Task 3), `getPool` (Task 2), `createApp` (Task 12). - Produces: a runnable entry point with no exports of its own — this is the process Passenger starts. - [ ] **Step 1: Create `src/server.js`** ```js import { loadConfig } from './config.js'; import { getPool } from './db.js'; import { ensureStorageDirs } from './storage.js'; import { createApp } from './app.js'; async function main() { const config = loadConfig(); await ensureStorageDirs(config); const pool = getPool(config); const app = createApp(config, pool); app.listen(config.port, () => { console.log(`File converter API listening on port ${config.port}`); }); } main().catch((error) => { console.error('Failed to start server:', error); process.exit(1); }); ``` - [ ] **Step 2: Update `package.json`'s `start` script** Change: ```json "start": "node src/app.js", ``` to: ```json "start": "node src/server.js", ``` - [ ] **Step 3: Verify the server starts** Run (with a reachable MariaDB and matching `.env`, copy `.env.example` to `.env` and adjust values first): ```bash npm start ``` Expected: console prints `File converter API listening on port 3000` (or your configured port), process stays running. Stop with Ctrl+C. - [ ] **Step 4: Commit** ```bash git add src/server.js package.json git commit -m "feat: add server entry point" ``` --- ## Task 15: Worker process **Files:** - Create: `src/worker.js` - Test: `test/worker.test.js` **Interfaces:** - Consumes: `findPendingJobs`/`markProcessing`/`markDone`/`markFailed` (Task 5), `resolve` (Task 6), `uploadPath`/`outputPath`/`deleteIfExists` (Task 3), `outputMimeType` (Task 4), the three `register*Converters` functions (Tasks 7-10). - Produces: - `export async function processPendingJobs(pool, config)` — fetches up to `config.workerConcurrency` pending jobs and processes them concurrently; returns the number of jobs it picked up. **This is the function the test suite calls directly** — it does one pass and returns, making it usable both by the real polling loop and by tests/Task 18's end-to-end test, without needing to wait on a timer. - `export function startWorker(pool, config)` — sets up a `setInterval` calling `processPendingJobs` every `config.workerPollIntervalMs`, returns a stop function. - A `main()` that registers all converters and calls `startWorker`, run only when this file is executed directly (not on import). - Each job conversion is bounded by a 60-second timeout; a timeout or thrown error marks the job `failed` with a generic `errorMessage` and the full error in `errorLog`, and any partial output file is deleted. - [ ] **Step 1: Write the failing tests** Create `test/worker.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { getPool, closePool } from '../src/db.js'; import { loadConfig } from '../src/config.js'; import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; import { createJob, getJobById, getJobErrorLog } from '../src/jobs/jobRepository.js'; import { registerImageConverters } from '../src/converters/image.js'; import { processPendingJobs } from '../src/worker.js'; let pool; let config; beforeAll(async () => { registerImageConverters(); config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) }; await ensureStorageDirs(config); pool = getPool(config); }); afterAll(async () => { await closePool(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); async function createPendingImageJob(id, sourceFormat, targetFormat) { const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); const inputFilePath = uploadPath(config, id, sourceFormat); await fs.copyFile(fixturePath, inputFilePath); await createJob(pool, { id, family: 'image', sourceFormat, targetFormat, originalFilename: `photo.${sourceFormat}`, inputPath: `${id}.${sourceFormat}`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() + 3600 * 1000), }); } describe('processPendingJobs', () => { it('converts a pending image job to done', async () => { const id = '99999999-9999-4999-8999-999999999999'; await createPendingImageJob(id, 'png', 'webp'); const processedCount = await processPendingJobs(pool, config); expect(processedCount).toBe(1); const job = await getJobById(pool, id); expect(job.status).toBe('done'); expect(job.outputPath).toBe(`${id}.webp`); expect(job.outputMimeType).toBe('image/webp'); const stat = await fs.stat(outputPath(config, id, 'webp')); expect(stat.size).toBeGreaterThan(0); }); it('marks a job failed with a safe message and a detailed log when the converter throws', async () => { const id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'; await createJob(pool, { id, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'missing.png', inputPath: `${id}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() + 3600 * 1000), }); // Note: input file is intentionally never written, so sharp will throw ENOENT. await processPendingJobs(pool, config); const job = await getJobById(pool, id); expect(job.status).toBe('failed'); expect(job.errorMessage).toBe('Conversion failed, please try again.'); const errorLog = await getJobErrorLog(pool, id); expect(errorLog).toMatch(/ENOENT|no such file/i); }); it('only picks up as many jobs as workerConcurrency allows', async () => { await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp'); await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp'); await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp'); const limitedConfig = { ...config, workerConcurrency: 2 }; const processedCount = await processPendingJobs(pool, limitedConfig); expect(processedCount).toBe(2); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/worker.test.js` Expected: FAIL — `src/worker.js` does not exist yet. - [ ] **Step 3: Implement `src/worker.js`** ```js import { loadConfig } from './config.js'; import { getPool } from './db.js'; import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js'; import { outputMimeType } from './mime.js'; import { resolve as resolveConverter } from './converters/registry.js'; import { registerImageConverters } from './converters/image.js'; import { registerImageToPdfConverter } from './converters/imageToPdf.js'; import { registerDocumentConverters } from './converters/document.js'; import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js'; const JOB_TIMEOUT_MS = 60000; function withTimeout(promise, ms) { let timeoutId; const timeout = new Promise((_, reject) => { timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms); }); return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId)); } async function processJob(pool, config, job) { await markProcessing(pool, job.id); const inputFilePath = uploadPath(config, job.id, job.sourceFormat); const outputFilePath = outputPath(config, job.id, job.targetFormat); try { const entry = resolveConverter(job.sourceFormat, job.targetFormat); if (!entry) { throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`); } await withTimeout(entry.convert(inputFilePath, outputFilePath), JOB_TIMEOUT_MS); await markDone(pool, job.id, { outputPath: `${job.id}.${job.targetFormat}`, outputMimeType: outputMimeType(job.targetFormat), }); } catch (error) { await deleteIfExists(outputFilePath); await markFailed(pool, job.id, { errorMessage: 'Conversion failed, please try again.', errorLog: error.stack ?? String(error), }); } } export async function processPendingJobs(pool, config) { const jobs = await findPendingJobs(pool, config.workerConcurrency); await Promise.all(jobs.map((job) => processJob(pool, config, job))); return jobs.length; } export function startWorker(pool, config) { const interval = setInterval(() => { processPendingJobs(pool, config).catch((error) => { console.error('Error while processing pending jobs:', error); }); }, config.workerPollIntervalMs); return () => clearInterval(interval); } async function main() { const config = loadConfig(); await ensureStorageDirs(config); const pool = getPool(config); registerImageConverters(); registerImageToPdfConverter(); registerDocumentConverters(); startWorker(pool, config); console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`); } if (import.meta.url === `file://${process.argv[1]}`) { main(); } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/worker.test.js` Expected: PASS (3 tests) - [ ] **Step 5: Commit** ```bash git add src/worker.js test/worker.test.js git commit -m "feat: add worker process with bounded concurrency and per-job timeout" ``` --- ## Task 16: Cleanup script and deployment files **Files:** - Create: `src/cleanup.js` - Create: `ecosystem.config.cjs` - Create: `README.md` (deployment section) - Test: `test/cleanup.test.js` **Interfaces:** - Consumes: `findExpiredJobs`/`deleteJob` (Task 5), `uploadPath`/`outputPath`/`deleteIfExists` (Task 3). - Produces: `export async function runCleanup(pool, config)` — deletes every expired job's row and its upload/output files (tolerating a missing output file for jobs that never finished), returns the number of jobs removed. A `main()` runs it once and exits, for use from cron. - [ ] **Step 1: Write the failing tests** Create `test/cleanup.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { getPool, closePool } from '../src/db.js'; import { loadConfig } from '../src/config.js'; import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; import { createJob, markDone, getJobById } from '../src/jobs/jobRepository.js'; import { runCleanup } from '../src/cleanup.js'; let pool; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) }; await ensureStorageDirs(config); pool = getPool(config); }); afterAll(async () => { await closePool(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); describe('runCleanup', () => { it('deletes an expired done job, its input file, and its output file', async () => { const id = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; await fs.writeFile(uploadPath(config, id, 'png'), 'input bytes'); await fs.writeFile(outputPath(config, id, 'webp'), 'output bytes'); await createJob(pool, { id, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${id}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() - 1000), }); await markDone(pool, id, { outputPath: `${id}.webp`, outputMimeType: 'image/webp' }); const deletedCount = await runCleanup(pool, config); expect(deletedCount).toBe(1); expect(await getJobById(pool, id)).toBeNull(); await expect(fs.stat(uploadPath(config, id, 'png'))).rejects.toThrow(); await expect(fs.stat(outputPath(config, id, 'webp'))).rejects.toThrow(); }); it('deletes an expired pending job (with no output file) without throwing', async () => { const id = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; await fs.writeFile(uploadPath(config, id, 'png'), 'input bytes'); await createJob(pool, { id, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${id}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() - 1000), }); const deletedCount = await runCleanup(pool, config); expect(deletedCount).toBe(1); expect(await getJobById(pool, id)).toBeNull(); }); it('leaves non-expired jobs untouched', async () => { const id = '12121212-1212-4212-8212-121212121212'; await createJob(pool, { id, family: 'image', sourceFormat: 'png', targetFormat: 'webp', originalFilename: 'photo.png', inputPath: `${id}.png`, inputMimeType: 'image/png', expiresAt: new Date(Date.now() + 3600 * 1000), }); const deletedCount = await runCleanup(pool, config); expect(deletedCount).toBe(0); expect(await getJobById(pool, id)).not.toBeNull(); }); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/cleanup.test.js` Expected: FAIL — `src/cleanup.js` does not exist yet. - [ ] **Step 3: Implement `src/cleanup.js`** ```js import { loadConfig } from './config.js'; import { getPool, closePool } from './db.js'; import { uploadPath, outputPath, deleteIfExists } from './storage.js'; import { findExpiredJobs, deleteJob } from './jobs/jobRepository.js'; export async function runCleanup(pool, config) { const expiredJobs = await findExpiredJobs(pool); for (const job of expiredJobs) { await deleteIfExists(uploadPath(config, job.id, job.sourceFormat)); await deleteIfExists(outputPath(config, job.id, job.targetFormat)); await deleteJob(pool, job.id); } return expiredJobs.length; } async function main() { const config = loadConfig(); const pool = getPool(config); const deletedCount = await runCleanup(pool, config); console.log(`Cleanup: removed ${deletedCount} expired job(s).`); await closePool(); } if (import.meta.url === `file://${process.argv[1]}`) { main(); } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/cleanup.test.js` Expected: PASS (3 tests) - [ ] **Step 5: Add the `cleanup` script check to `package.json`** Confirm `package.json` already has (added in Task 1): ```json "cleanup": "node src/cleanup.js" ``` - [ ] **Step 6: Create the pm2 ecosystem file for the worker** Create `ecosystem.config.cjs`: ```js module.exports = { apps: [ { name: 'convert-worker', script: 'src/worker.js', interpreter: 'node', env: { NODE_ENV: 'production', }, }, ], }; ``` - [ ] **Step 7: Document deployment steps in `README.md`** Create `README.md`: ```markdown # File Converter ## Local development 1. Copy `.env.example` to `.env` and fill in your local MariaDB credentials. 2. Apply the schema: `mysql -h -u -p < db/schema.sql` 3. Install dependencies: `npm install` 4. Run the API: `npm start` 5. Run the worker (separate terminal): `npm run worker` 6. Run tests: `npm test` (requires the same MariaDB reachable via your `.env` vars, exported into the shell) ## Deployment on o2switch 1. Upload the project (excluding `node_modules/`) via SSH/Git. 2. Create/adjust `.env` on the server with production values (`STORAGE_DIR` pointing to a writable path under your account, MariaDB credentials from cPanel). 3. Apply `db/schema.sql` to the MariaDB database created in cPanel. 4. `npm install` (never with `--ignore-scripts` — Puppeteer needs its postinstall step to download Chromium). 5. Configure the app in cPanel "Setup Node.js App", pointing its entry point at `src/server.js`. Passenger manages this process (start/stop/restart). 6. Start the worker independently of Passenger, over SSH: `pm2 start ecosystem.config.cjs`, then `pm2 save`. Try `pm2 startup` to survive a server reboot; if that's not permitted without root on this account, fall back to a cPanel cron job every 5 minutes that runs `pm2 resurrect` (or checks `pm2 list` and restarts the app if absent) — validate which option this hosting plan actually allows once connected over SSH. 7. Add a cPanel cron job to run the cleanup script periodically, e.g. every 15 minutes: `*/15 * * * * cd /home// && /usr/bin/node src/cleanup.js >> logs/cleanup.log 2>&1` (adjust the path and node binary location to match your actual account — check with `which node` over SSH). 8. On every subsequent deployment: pull changes, `npm install`, `npm run build` (frontend, once Task 17 exists), then restart the Passenger app from cPanel and `pm2 restart convert-worker`. ``` - [ ] **Step 8: Commit** ```bash git add src/cleanup.js test/cleanup.test.js ecosystem.config.cjs README.md git commit -m "feat: add cleanup script and o2switch deployment docs" ``` --- ## Task 17: Frontend (React SPA) **Files:** - Create: `frontend/` (scaffolded by Vite, then edited) - Modify: `frontend/vite.config.js` - Create: `frontend/src/api.js` - Create: `frontend/src/FileCard.jsx` - Modify: `frontend/src/App.jsx` - Modify: `frontend/src/App.css` - Modify: `src/app.js` (serve the built frontend) - Modify: `package.json` (root `build` script) This task is implementation-focused rather than TDD — the design spec's test strategy scopes automated tests to converters/registry/API (Tasks 1-16), not UI components. Verification here is manual: run the dev server and the built app in a browser. - [ ] **Step 1: Scaffold the Vite React app** From the project root: ```bash npm create vite@latest frontend -- --template react cd frontend && npm install && cd .. ``` - [ ] **Step 2: Configure the dev proxy so the Vite dev server forwards `/api` to Express** Replace the contents of `frontend/vite.config.js`: ```js import { defineConfig } from 'vite'; import react from '@vitejs/plugin-react'; export default defineConfig({ plugins: [react()], server: { proxy: { '/api': 'http://localhost:3000', }, }, }); ``` - [ ] **Step 3: Create `frontend/src/api.js`** ```js export async function fetchFormats(source) { const response = await fetch(`/api/formats?source=${encodeURIComponent(source)}`); const data = await response.json(); return data.targets; } export async function uploadFiles(items) { const formData = new FormData(); const targetFormats = []; for (const item of items) { formData.append('files', item.file); targetFormats.push(item.targetFormat); } formData.append('targetFormats', JSON.stringify(targetFormats)); const response = await fetch('/api/jobs', { method: 'POST', body: formData }); const data = await response.json(); return data.jobs; } export async function fetchJobStatus(id) { const response = await fetch(`/api/jobs/${id}`); return response.json(); } export function downloadUrl(id) { return `/api/jobs/${id}/download`; } ``` - [ ] **Step 4: Create `frontend/src/FileCard.jsx`** ```jsx import { useEffect, useState } from 'react'; import { fetchJobStatus, downloadUrl } from './api.js'; export function FileCard({ fileName, jobId, initialError }) { const [status, setStatus] = useState(initialError ? 'failed' : 'pending'); const [errorMessage, setErrorMessage] = useState(initialError ?? null); useEffect(() => { if (!jobId || initialError) return undefined; let cancelled = false; const interval = setInterval(async () => { const job = await fetchJobStatus(jobId); if (cancelled) return; setStatus(job.status); if (job.status === 'failed') setErrorMessage(job.errorMessage); if (job.status === 'done' || job.status === 'failed') clearInterval(interval); }, 1500); return () => { cancelled = true; clearInterval(interval); }; }, [jobId, initialError]); return (
  • {fileName} {(status === 'pending' || status === 'processing') && Conversion en cours...} {status === 'done' && Télécharger} {status === 'failed' && {errorMessage}}
  • ); } ``` - [ ] **Step 5: Replace `frontend/src/App.jsx`** ```jsx import { useState } from 'react'; import { fetchFormats, uploadFiles } from './api.js'; import { FileCard } from './FileCard.jsx'; import './App.css'; function extensionOf(fileName) { return fileName.split('.').pop().toLowerCase(); } export default function App() { const [pendingFiles, setPendingFiles] = useState([]); const [submittedJobs, setSubmittedJobs] = useState([]); async function handleFilesSelected(fileList) { const files = Array.from(fileList); const withTargets = await Promise.all( files.map(async (file) => { const targets = await fetchFormats(extensionOf(file.name)); return { file, targets, targetFormat: targets[0] ?? null }; }) ); setPendingFiles(withTargets); } function updateTargetFormat(index, targetFormat) { setPendingFiles((current) => current.map((item, i) => (i === index ? { ...item, targetFormat } : item))); } async function handleConvert() { const validItems = pendingFiles.filter((item) => item.targetFormat); const jobs = await uploadFiles(validItems); setSubmittedJobs((current) => [...current, ...jobs]); setPendingFiles([]); } return (

    Convertisseur de fichiers

    handleFilesSelected(event.target.files)} /> {pendingFiles.length > 0 && (
      {pendingFiles.map((item, index) => (
    • {item.file.name} {item.targets.length > 0 ? ( ) : ( Format non supporté )}
    • ))}
    )}
      {submittedJobs.map((job, index) => job.id ? ( ) : ( ) )}
    ); } ``` - [ ] **Step 6: Replace `frontend/src/App.css` with minimal layout styles** ```css main { max-width: 640px; margin: 2rem auto; font-family: system-ui, sans-serif; } ul { list-style: none; padding: 0; } li { display: flex; align-items: center; gap: 0.75rem; padding: 0.5rem 0; } .error { color: #b00020; } ``` - [ ] **Step 7: Serve the built frontend from Express** Add to `src/app.js`, near the top (with the other imports): ```js import path from 'node:path'; ``` (skip if already imported by Task 13). Add this after the API routes but **before** the error-handling middleware Task 13 added at the end of `createApp` (an error handler registered before a route never sees that route's errors, so the static/catch-all route must come first): ```js const frontendDist = path.join(import.meta.dirname, '..', 'frontend', 'dist'); app.use(express.static(frontendDist)); app.get(/^\/(?!api\/).*/, (req, res) => { res.sendFile(path.join(frontendDist, 'index.html')); }); ``` - [ ] **Step 8: Add a root-level `build` script** Add to the root `package.json` `scripts`: ```json "build": "npm install --prefix frontend && npm run build --prefix frontend" ``` - [ ] **Step 9: Manual verification — dev mode** Terminal 1: `npm start` (Express API on port 3000) Terminal 2: `cd frontend && npm run dev` (Vite dev server, typically port 5173) Open the Vite dev server URL in a browser. Drop a PNG and a DOCX file simultaneously, pick target formats, click "Convertir", confirm both file cards poll independently and eventually show a working "Télécharger" link (requires the worker from Task 15 running too: `npm run worker` in a third terminal). - [ ] **Step 10: Manual verification — production build served by Express** ```bash npm run build npm start ``` Open `http://localhost:3000` directly (no Vite dev server this time) and repeat the same upload/convert/download flow. - [ ] **Step 11: Commit** ```bash git add frontend package.json src/app.js git commit -m "feat: add React frontend and serve it from Express" ``` --- ## Task 18: End-to-end pipeline test (one case per family) **Files:** - Create: `test/api/endToEnd.test.js` **Interfaces:** - Consumes: `createApp` (Task 12), `processPendingJobs` (Task 15) — called directly instead of `startWorker`'s timer, so the test drives the worker deterministically instead of racing a `setInterval`. This closes the loop the spec's test strategy asked for: upload → status → download, for one image case and one document case, through the real HTTP API and the real worker function (no mocks). - [ ] **Step 1: Write the test** Create `test/api/endToEnd.test.js`: ```js import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; import request from 'supertest'; import fs from 'node:fs/promises'; import path from 'node:path'; import os from 'node:os'; import { Document, Paragraph, TextRun, Packer } from 'docx'; import { createApp } from '../../src/app.js'; import { getPool, closePool } from '../../src/db.js'; import { loadConfig } from '../../src/config.js'; import { ensureStorageDirs } from '../../src/storage.js'; import { processPendingJobs } from '../../src/worker.js'; let app; let pool; let config; beforeAll(async () => { config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) }; await ensureStorageDirs(config); pool = getPool(config); app = createApp(config, pool); }); afterAll(async () => { await closePool(); await fs.rm(config.storageDir, { recursive: true, force: true }); }); beforeEach(async () => { await pool.query('DELETE FROM conversion_jobs'); }); async function waitForDone(id, maxAttempts = 20) { for (let attempt = 0; attempt < maxAttempts; attempt += 1) { await processPendingJobs(pool, config); const response = await request(app).get(`/api/jobs/${id}`); if (response.body.status === 'done' || response.body.status === 'failed') { return response.body; } } throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`); } describe('end-to-end: image family', () => { it('uploads a PNG, converts it to WebP, and downloads the result', async () => { const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); const uploadResponse = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['webp'])) .attach('files', fixturePath, 'photo.png'); const { id } = uploadResponse.body.jobs[0]; const finalStatus = await waitForDone(id); expect(finalStatus.status).toBe('done'); const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); expect(downloadResponse.status).toBe(200); expect(downloadResponse.headers['content-type']).toBe('image/webp'); expect(downloadResponse.headers['content-disposition']).toContain('photo.png'); }, 20000); }); describe('end-to-end: document family', () => { it('uploads a DOCX, converts it to PDF, and downloads the result', async () => { const doc = new Document({ sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }], }); const buffer = await Packer.toBuffer(doc); const docxPath = path.join(config.storageDir, 'e2e-fixture.docx'); await fs.writeFile(docxPath, buffer); const uploadResponse = await request(app) .post('/api/jobs') .field('targetFormats', JSON.stringify(['pdf'])) .attach('files', docxPath, 'report.docx'); const { id } = uploadResponse.body.jobs[0]; const finalStatus = await waitForDone(id); expect(finalStatus.status).toBe('done'); const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); expect(downloadResponse.status).toBe(200); expect(downloadResponse.headers['content-type']).toBe('application/pdf'); expect(downloadResponse.headers['content-disposition']).toContain('report.docx'); await fs.unlink(docxPath); }, 20000); }); ``` - [ ] **Step 2: Run the test to verify it fails initially if the worker isn't wired correctly** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/endToEnd.test.js` Expected at this point: PASS, since Tasks 1-17 already implemented everything this test exercises — this step is a regression check, not new implementation. If it fails, the bug is in the interaction between tasks (most likely a mismatch between how `app.js` names uploaded files and how `worker.js` reconstructs paths), not in a single file — investigate before continuing. - [ ] **Step 3: Run the entire test suite** Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run` Expected: PASS, all tests across all previous tasks plus this one. - [ ] **Step 4: Commit** ```bash git add test/api/endToEnd.test.js git commit -m "test: add end-to-end upload-convert-download coverage for image and document families" ``` --- ## Out of scope for this plan Audio, Video, Presentations, Fonts, Ebook, Archives — to be added later as new converter family modules registered the same way (`register*Converters()` functions consumed by `app.js` and `worker.js`), following the pattern established in Tasks 7-10. BMP output support (sharp does not support it) is also deferred pending confirmation it's actually needed.