From fe93139c6ec74afa4e815e07f983b25c05de2468 Mon Sep 17 00:00:00 2001 From: Anthony GAEREMYNCK <1@anthony.sh> Date: Wed, 29 Jul 2026 13:46:16 +0200 Subject: [PATCH] Add implementation plan for file converter v1 (core + Images + Documents) 18 bite-sized TDD tasks from project scaffolding through the React frontend and an end-to-end pipeline test, following the approved design spec. Co-Authored-By: Claude Sonnet 5 --- .../plans/2026-07-28-file-converter.md | 3054 +++++++++++++++++ 1 file changed, 3054 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-28-file-converter.md diff --git a/docs/superpowers/plans/2026-07-28-file-converter.md b/docs/superpowers/plans/2026-07-28-file-converter.md new file mode 100644 index 0000000..4f7545b --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-file-converter.md @@ -0,0 +1,3054 @@ +# File Converter Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the core (upload → convert → download) pipeline of a Convertio-like file converter, covering the Images and Documents format families, deployable on o2switch shared hosting. + +**Architecture:** An Express API process (behind Passenger) accepts uploads and writes `pending` jobs to a MariaDB `conversion_jobs` table; a separate worker process (run via `pm2`, outside Passenger) polls that table, runs the matching converter from a central format registry, and writes the result to disk. A React SPA drives uploads and polls job status. A cron-triggered cleanup script deletes expired jobs and their files. + +**Tech Stack:** Node.js (ESM, `"type": "module"`), Express, MariaDB (`mariadb` npm connector), `multer` (upload), `sharp` (images), `pdf-lib` + `puppeteer` + `mammoth` + `pdfjs-dist` + `docx` (documents), `file-type` (MIME sniffing), `express-rate-limit`, `uuid`, React (Vite) frontend, `vitest` + `supertest` for tests. + +## Global Constraints + +- No native binaries installed at the system level (no apt-get, no compilation). Only pure-JS npm packages or npm packages that auto-download a precompiled binary during `npm install` (e.g. Puppeteer's Chromium) are allowed. +- `npm install` must never run with `--ignore-scripts` in deployment — Puppeteer's Chromium download depends on its postinstall script running. +- Max upload size: `MAX_FILE_SIZE_MB`, default 100. +- Retention: `RETENTION_HOURS`, default 1 — expired jobs (row + files) are deleted by the cleanup script. +- Job IDs are UUID v4, used directly in public URLs (`/api/jobs/:id`, `/api/jobs/:id/download`). Non-enumerable by design — no sequential IDs anywhere in the public API. +- Files on disk are named `.` directly under `STORAGE_DIR/uploads/` and `STORAGE_DIR/outputs/` — no subfolders, no original filename in the path. +- `input_mime_type` is always detected from magic bytes (`file-type`), never trusted from the client's declared `Content-Type` or file extension. A mismatch with the declared `source_format` is rejected (HTTP 422) before a job is created. +- `error_message` (short, user-safe) and `error_log` (full technical detail, never returned by the API) are separate columns — the API must never leak `error_log` contents to a client. +- Worker concurrency is bounded by `WORKER_CONCURRENCY` — jobs beyond that stay `pending` rather than being spawned unbounded. +- Spec reference: `docs/superpowers/specs/2026-07-28-file-converter-design.md`. + +--- + +## Task 1: Project scaffolding & config + +**Files:** +- Create: `package.json` +- Create: `.env.example` +- Create: `.gitignore` (append `node_modules/`, `.env`, `storage/`, `dist/` if not already present) +- Create: `src/config.js` +- Test: `test/config.test.js` + +**Interfaces:** +- Produces: `export function loadConfig()` in `src/config.js`, returning: + ```js + { + port: number, + storageDir: string, + db: { host: string, user: string, password: string, database: string }, + maxFileSizeMb: number, + retentionHours: number, + workerPollIntervalMs: number, + workerConcurrency: number, + rateLimitMaxJobs: number, + rateLimitWindowMinutes: number, + } + ``` + Throws an `Error` listing every missing required variable if any of `STORAGE_DIR`, `DB_HOST`, `DB_USER`, `DB_PASSWORD`, `DB_NAME` is unset. + +- [ ] **Step 1: Create `package.json`** + +```bash +npm init -y +``` + +Then edit `package.json` to set: + +```json +{ + "name": "file-converter", + "version": "0.1.0", + "private": true, + "type": "module", + "scripts": { + "start": "node src/app.js", + "worker": "node src/worker.js", + "cleanup": "node src/cleanup.js", + "test": "vitest run" + } +} +``` + +- [ ] **Step 2: Install dependencies** + +```bash +npm install express multer mariadb uuid file-type sharp pdf-lib puppeteer mammoth docx pdfjs-dist express-rate-limit dotenv +npm install --save-dev vitest supertest +``` + +Expected: `npm install` completes without error (Puppeteer's postinstall will download Chromium — this can take a minute). + +- [ ] **Step 3: Create `.env.example`** + +```dotenv +PORT=3000 +STORAGE_DIR=./storage +DB_HOST=127.0.0.1 +DB_USER=convert_user +DB_PASSWORD=change_me +DB_NAME=file_converter +MAX_FILE_SIZE_MB=100 +RETENTION_HOURS=1 +WORKER_POLL_INTERVAL_MS=1500 +WORKER_CONCURRENCY=3 +RATE_LIMIT_MAX_JOBS=20 +RATE_LIMIT_WINDOW_MINUTES=10 +``` + +- [ ] **Step 4: Update `.gitignore`** + +Add these lines if not already present: + +```gitignore +node_modules/ +.env +storage/ +dist/ +``` + +- [ ] **Step 5: Write the failing test for config loading** + +Create `test/config.test.js`: + +```js +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { loadConfig } from '../src/config.js'; + +const REQUIRED_VARS = ['STORAGE_DIR', 'DB_HOST', 'DB_USER', 'DB_PASSWORD', 'DB_NAME']; +let savedEnv; + +beforeEach(() => { + savedEnv = { ...process.env }; +}); + +afterEach(() => { + process.env = savedEnv; +}); + +describe('loadConfig', () => { + it('returns parsed config when all required vars are set', () => { + process.env.STORAGE_DIR = './storage'; + process.env.DB_HOST = 'localhost'; + process.env.DB_USER = 'user'; + process.env.DB_PASSWORD = 'pass'; + process.env.DB_NAME = 'db'; + process.env.PORT = '4000'; + process.env.MAX_FILE_SIZE_MB = '50'; + + const config = loadConfig(); + + expect(config.port).toBe(4000); + expect(config.storageDir).toBe('./storage'); + expect(config.db).toEqual({ host: 'localhost', user: 'user', password: 'pass', database: 'db' }); + expect(config.maxFileSizeMb).toBe(50); + }); + + it('applies defaults for optional vars', () => { + process.env.STORAGE_DIR = './storage'; + process.env.DB_HOST = 'localhost'; + process.env.DB_USER = 'user'; + process.env.DB_PASSWORD = 'pass'; + process.env.DB_NAME = 'db'; + delete process.env.PORT; + delete process.env.MAX_FILE_SIZE_MB; + delete process.env.RETENTION_HOURS; + delete process.env.WORKER_CONCURRENCY; + + const config = loadConfig(); + + expect(config.port).toBe(3000); + expect(config.maxFileSizeMb).toBe(100); + expect(config.retentionHours).toBe(1); + expect(config.workerConcurrency).toBe(3); + }); + + it('throws listing every missing required variable', () => { + for (const key of REQUIRED_VARS) delete process.env[key]; + + expect(() => loadConfig()).toThrowError(/STORAGE_DIR.*DB_HOST.*DB_USER.*DB_PASSWORD.*DB_NAME/s); + }); +}); +``` + +- [ ] **Step 6: Run test to verify it fails** + +Run: `npx vitest run test/config.test.js` +Expected: FAIL — `src/config.js` does not exist yet. + +- [ ] **Step 7: Implement `src/config.js`** + +```js +import 'dotenv/config'; + +const REQUIRED_VARS = ['STORAGE_DIR', 'DB_HOST', 'DB_USER', 'DB_PASSWORD', 'DB_NAME']; + +export function loadConfig() { + const missing = REQUIRED_VARS.filter((key) => !process.env[key]); + if (missing.length > 0) { + throw new Error(`Missing required environment variables: ${missing.join(', ')}`); + } + + return { + port: Number(process.env.PORT ?? 3000), + storageDir: process.env.STORAGE_DIR, + db: { + host: process.env.DB_HOST, + user: process.env.DB_USER, + password: process.env.DB_PASSWORD, + database: process.env.DB_NAME, + }, + maxFileSizeMb: Number(process.env.MAX_FILE_SIZE_MB ?? 100), + retentionHours: Number(process.env.RETENTION_HOURS ?? 1), + workerPollIntervalMs: Number(process.env.WORKER_POLL_INTERVAL_MS ?? 1500), + workerConcurrency: Number(process.env.WORKER_CONCURRENCY ?? 3), + rateLimitMaxJobs: Number(process.env.RATE_LIMIT_MAX_JOBS ?? 20), + rateLimitWindowMinutes: Number(process.env.RATE_LIMIT_WINDOW_MINUTES ?? 10), + }; +} +``` + +- [ ] **Step 8: Run test to verify it passes** + +Run: `npx vitest run test/config.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 9: Commit** + +```bash +git add package.json package-lock.json .env.example .gitignore src/config.js test/config.test.js +git commit -m "feat: project scaffolding and env config loader" +``` + +--- + +## Task 2: MariaDB schema & connection pool + +**Files:** +- Create: `db/schema.sql` +- Create: `src/db.js` +- Test: `test/db.test.js` + +**Interfaces:** +- Consumes: `loadConfig().db` from Task 1 (`{ host, user, password, database }`). +- Produces: `export function getPool(config)` in `src/db.js` — returns a singleton `mariadb` pool built from `config.db` (creates it on first call, returns the same instance on subsequent calls). Also exports `export async function closePool()` to end the pool (used in tests and graceful shutdown). + +- [ ] **Step 1: Write the schema file** + +Create `db/schema.sql`: + +```sql +CREATE TABLE IF NOT EXISTS conversion_jobs ( + id CHAR(36) NOT NULL PRIMARY KEY, + status ENUM('pending', 'processing', 'done', 'failed') NOT NULL DEFAULT 'pending', + family VARCHAR(32) NOT NULL, + source_format VARCHAR(16) NOT NULL, + target_format VARCHAR(16) NOT NULL, + original_filename VARCHAR(255) NOT NULL, + input_path VARCHAR(255) NOT NULL, + output_path VARCHAR(255) NULL, + input_mime_type VARCHAR(128) NOT NULL, + output_mime_type VARCHAR(128) NULL, + error_message VARCHAR(255) NULL, + error_log TEXT NULL, + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + expires_at DATETIME NOT NULL, + INDEX idx_status (status), + INDEX idx_expires_at (expires_at) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; +``` + +- [ ] **Step 2: Apply the schema to a local/test MariaDB database** + +Run (adjust host/user/db to your local MariaDB instance used for development): + +```bash +mysql -h 127.0.0.1 -u convert_user -p file_converter < db/schema.sql +``` + +Expected: no error, `conversion_jobs` table exists (`SHOW TABLES;` lists it). + +- [ ] **Step 3: Write the failing test for the pool** + +Create `test/db.test.js`: + +```js +import { describe, it, expect, afterAll } from 'vitest'; +import { getPool, closePool } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; + +describe('getPool', () => { + afterAll(async () => { + await closePool(); + }); + + it('returns a working pool that can run a query', async () => { + const config = loadConfig(); + const pool = getPool(config); + + const rows = await pool.query('SELECT 1 AS value'); + + expect(Number(rows[0].value)).toBe(1); + }); + + it('returns the same pool instance on repeated calls', () => { + const config = loadConfig(); + const poolA = getPool(config); + const poolB = getPool(config); + + expect(poolA).toBe(poolB); + }); +}); +``` + +Note: this test requires a reachable MariaDB instance matching the env vars used to run tests: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/db.test.js`. + +- [ ] **Step 4: Run test to verify it fails** + +Run: `npx vitest run test/db.test.js` +Expected: FAIL — `src/db.js` does not exist yet. + +- [ ] **Step 5: Implement `src/db.js`** + +```js +import mariadb from 'mariadb'; + +let pool; + +export function getPool(config) { + if (!pool) { + pool = mariadb.createPool({ + host: config.db.host, + user: config.db.user, + password: config.db.password, + database: config.db.database, + connectionLimit: 10, + }); + } + return pool; +} + +export async function closePool() { + if (pool) { + await pool.end(); + pool = undefined; + } +} +``` + +- [ ] **Step 6: Run test to verify it passes** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/db.test.js` +Expected: PASS (2 tests) + +- [ ] **Step 7: Commit** + +```bash +git add db/schema.sql src/db.js test/db.test.js +git commit -m "feat: add MariaDB schema and connection pool" +``` + +--- + +## Task 3: Storage utilities + +**Files:** +- Create: `src/storage.js` +- Test: `test/storage.test.js` + +**Interfaces:** +- Consumes: `config.storageDir` from Task 1. +- Produces: + - `export async function ensureStorageDirs(config)` — creates `/uploads` and `/outputs` if missing. + - `export function uploadPath(config, id, ext)` — returns `/uploads/.` (absolute path). + - `export function outputPath(config, id, ext)` — returns `/outputs/.` (absolute path). + - `export async function deleteIfExists(filePath)` — deletes a file, silently no-ops if it doesn't exist. + +- [ ] **Step 1: Write the failing tests** + +Create `test/storage.test.js`: + +```js +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from '../src/storage.js'; + +let tmpDir; +let config; + +beforeEach(async () => { + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-storage-')); + config = { storageDir: tmpDir }; +}); + +afterEach(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +describe('storage', () => { + it('creates uploads and outputs directories', async () => { + await ensureStorageDirs(config); + + const uploadsStat = await fs.stat(path.join(tmpDir, 'uploads')); + const outputsStat = await fs.stat(path.join(tmpDir, 'outputs')); + + expect(uploadsStat.isDirectory()).toBe(true); + expect(outputsStat.isDirectory()).toBe(true); + }); + + it('builds upload and output paths as .', () => { + const id = '11111111-1111-4111-8111-111111111111'; + + expect(uploadPath(config, id, 'png')).toBe(path.join(tmpDir, 'uploads', `${id}.png`)); + expect(outputPath(config, id, 'pdf')).toBe(path.join(tmpDir, 'outputs', `${id}.pdf`)); + }); + + it('deletes an existing file', async () => { + await ensureStorageDirs(config); + const filePath = uploadPath(config, 'file-to-delete', 'txt'); + await fs.writeFile(filePath, 'content'); + + await deleteIfExists(filePath); + + await expect(fs.stat(filePath)).rejects.toThrow(); + }); + + it('does not throw when deleting a missing file', async () => { + const filePath = uploadPath(config, 'does-not-exist', 'txt'); + + await expect(deleteIfExists(filePath)).resolves.toBeUndefined(); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx vitest run test/storage.test.js` +Expected: FAIL — `src/storage.js` does not exist yet. + +- [ ] **Step 3: Implement `src/storage.js`** + +```js +import fs from 'node:fs/promises'; +import path from 'node:path'; + +export async function ensureStorageDirs(config) { + await fs.mkdir(path.join(config.storageDir, 'uploads'), { recursive: true }); + await fs.mkdir(path.join(config.storageDir, 'outputs'), { recursive: true }); +} + +export function uploadPath(config, id, ext) { + return path.join(config.storageDir, 'uploads', `${id}.${ext}`); +} + +export function outputPath(config, id, ext) { + return path.join(config.storageDir, 'outputs', `${id}.${ext}`); +} + +export async function deleteIfExists(filePath) { + try { + await fs.unlink(filePath); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx vitest run test/storage.test.js` +Expected: PASS (4 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/storage.js test/storage.test.js +git commit -m "feat: add storage path helpers" +``` + +--- + +## Task 4: MIME detection utilities + +**Files:** +- Create: `src/mime.js` +- Create: `test/fixtures/sample.png` (tiny valid PNG used as a fixture) +- Test: `test/mime.test.js` + +**Interfaces:** +- Produces: + - `export async function detectInputMime(filePath)` — sniffs magic bytes via `file-type`'s `fileTypeFromFile`, returns `{ ext, mime }` or `null` if undetectable. + - `export function outputMimeType(targetFormat)` — looks up a static map, throws if `targetFormat` is unknown. + +- [ ] **Step 1: Create the PNG fixture** + +Create `test/fixtures/sample.png` with this exact 67-byte 1x1 transparent PNG (write as binary — do not treat as text): + +```js +// One-off script to generate the fixture (run once, not part of the app): +// node -e "require('fs').writeFileSync('test/fixtures/sample.png', Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGNgAAIAAAUAAen63NgAAAAASUVORK5CYII=', 'base64'))" +``` + +Run that command from the project root. + +- [ ] **Step 2: Write the failing tests** + +Create `test/mime.test.js`: + +```js +import { describe, it, expect } from 'vitest'; +import path from 'node:path'; +import { detectInputMime, outputMimeType } from '../src/mime.js'; + +describe('detectInputMime', () => { + it('detects PNG from magic bytes regardless of file extension', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + + const result = await detectInputMime(fixturePath); + + expect(result).toEqual({ ext: 'png', mime: 'image/png' }); + }); +}); + +describe('outputMimeType', () => { + it('returns the MIME type for a known target format', () => { + expect(outputMimeType('pdf')).toBe('application/pdf'); + expect(outputMimeType('png')).toBe('image/png'); + expect(outputMimeType('docx')).toBe( + 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' + ); + }); + + it('throws for an unknown target format', () => { + expect(() => outputMimeType('made-up-format')).toThrowError(/made-up-format/); + }); +}); +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `npx vitest run test/mime.test.js` +Expected: FAIL — `src/mime.js` does not exist yet. + +- [ ] **Step 4: Implement `src/mime.js`** + +```js +import { fileTypeFromFile } from 'file-type'; + +const OUTPUT_MIME_TYPES = { + jpg: 'image/jpeg', + jpeg: 'image/jpeg', + png: 'image/png', + webp: 'image/webp', + gif: 'image/gif', + tiff: 'image/tiff', + avif: 'image/avif', + bmp: 'image/bmp', + pdf: 'application/pdf', + html: 'text/html', + txt: 'text/plain', + docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', +}; + +export async function detectInputMime(filePath) { + const result = await fileTypeFromFile(filePath); + return result ?? null; +} + +export function outputMimeType(targetFormat) { + const mime = OUTPUT_MIME_TYPES[targetFormat]; + if (!mime) { + throw new Error(`No known MIME type for target format "${targetFormat}"`); + } + return mime; +} +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `npx vitest run test/mime.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 6: Commit** + +```bash +git add src/mime.js test/mime.test.js test/fixtures/sample.png +git commit -m "feat: add MIME sniffing and output MIME lookup" +``` + +--- + +## Task 5: Job repository + +**Files:** +- Create: `src/jobs/jobRepository.js` +- Test: `test/jobs/jobRepository.test.js` + +**Interfaces:** +- Consumes: `getPool(config)` from Task 2. +- Produces (all functions take `pool` as first argument): + - `async function createJob(pool, { id, family, sourceFormat, targetFormat, originalFilename, inputPath, inputMimeType, expiresAt })` — inserts a `pending` row, returns nothing. + - `async function getJobById(pool, id)` — returns the row as a plain object (camelCase keys: `id, status, family, sourceFormat, targetFormat, originalFilename, inputPath, outputPath, inputMimeType, outputMimeType, errorMessage, createdAt, updatedAt, expiresAt`) or `null` if not found. `error_log` is intentionally NOT selected by this function — callers that need it must use `getJobErrorLog`. + - `async function getJobErrorLog(pool, id)` — returns the raw `error_log` string or `null`. Used only by the worker/debug tooling, never by the public API. + - `async function markProcessing(pool, id)` — sets `status = 'processing'`. + - `async function markDone(pool, id, { outputPath, outputMimeType })` — sets `status = 'done'` and those two columns. + - `async function markFailed(pool, id, { errorMessage, errorLog })` — sets `status = 'failed'` and both error columns. + - `async function findPendingJobs(pool, limit)` — returns up to `limit` rows with `status = 'pending'`, oldest first, each mapped like `getJobById`. + - `async function findExpiredJobs(pool)` — returns all rows where `expires_at < NOW()`, mapped like `getJobById`. + - `async function deleteJob(pool, id)` — deletes the row. + +- [ ] **Step 1: Write the failing tests** + +Create `test/jobs/jobRepository.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { getPool, closePool } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { + createJob, + getJobById, + getJobErrorLog, + markProcessing, + markDone, + markFailed, + findPendingJobs, + findExpiredJobs, + deleteJob, +} from '../../src/jobs/jobRepository.js'; + +let pool; + +beforeAll(() => { + pool = getPool(loadConfig()); +}); + +afterAll(async () => { + await closePool(); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +function baseJob(overrides = {}) { + return { + id: overrides.id ?? '11111111-1111-4111-8111-111111111111', + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${overrides.id ?? '11111111-1111-4111-8111-111111111111'}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() + 3600 * 1000), + ...overrides, + }; +} + +describe('jobRepository', () => { + it('creates and retrieves a pending job', async () => { + await createJob(pool, baseJob()); + + const job = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); + + expect(job.status).toBe('pending'); + expect(job.family).toBe('image'); + expect(job.sourceFormat).toBe('png'); + expect(job.targetFormat).toBe('webp'); + expect(job.originalFilename).toBe('photo.png'); + expect(job.inputMimeType).toBe('image/png'); + expect(job.outputPath).toBeNull(); + expect(job.errorMessage).toBeNull(); + }); + + it('returns null for an unknown id', async () => { + const job = await getJobById(pool, '22222222-2222-4222-8222-222222222222'); + expect(job).toBeNull(); + }); + + it('transitions a job through processing to done', async () => { + await createJob(pool, baseJob()); + + await markProcessing(pool, '11111111-1111-4111-8111-111111111111'); + const processing = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); + expect(processing.status).toBe('processing'); + + await markDone(pool, '11111111-1111-4111-8111-111111111111', { + outputPath: '11111111-1111-4111-8111-111111111111.webp', + outputMimeType: 'image/webp', + }); + const done = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); + expect(done.status).toBe('done'); + expect(done.outputPath).toBe('11111111-1111-4111-8111-111111111111.webp'); + expect(done.outputMimeType).toBe('image/webp'); + }); + + it('marks a job failed with a short message and a separate detailed log', async () => { + await createJob(pool, baseJob()); + + await markFailed(pool, '11111111-1111-4111-8111-111111111111', { + errorMessage: 'Conversion failed, please try again', + errorLog: 'Error: sharp threw at line 42\n at convert (image.js:10:5)', + }); + + const job = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); + expect(job.status).toBe('failed'); + expect(job.errorMessage).toBe('Conversion failed, please try again'); + expect(job.errorLog).toBeUndefined(); + + const errorLog = await getJobErrorLog(pool, '11111111-1111-4111-8111-111111111111'); + expect(errorLog).toBe('Error: sharp threw at line 42\n at convert (image.js:10:5)'); + }); + + it('finds pending jobs oldest first, up to a limit', async () => { + await createJob(pool, baseJob({ id: '33333333-3333-4333-8333-333333333333' })); + await createJob(pool, baseJob({ id: '44444444-4444-4444-8444-444444444444' })); + await createJob(pool, baseJob({ id: '55555555-5555-4555-8555-555555555555' })); + + const jobs = await findPendingJobs(pool, 2); + + expect(jobs).toHaveLength(2); + expect(jobs[0].id).toBe('33333333-3333-4333-8333-333333333333'); + expect(jobs[1].id).toBe('44444444-4444-4444-8444-444444444444'); + }); + + it('finds expired jobs and allows deleting them', async () => { + await createJob(pool, baseJob({ expiresAt: new Date(Date.now() - 1000) })); + + const expired = await findExpiredJobs(pool); + expect(expired).toHaveLength(1); + expect(expired[0].id).toBe('11111111-1111-4111-8111-111111111111'); + + await deleteJob(pool, '11111111-1111-4111-8111-111111111111'); + const afterDelete = await getJobById(pool, '11111111-1111-4111-8111-111111111111'); + expect(afterDelete).toBeNull(); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/jobs/jobRepository.test.js` +Expected: FAIL — `src/jobs/jobRepository.js` does not exist yet. + +- [ ] **Step 3: Implement `src/jobs/jobRepository.js`** + +```js +function toCamelJob(row) { + if (!row) return null; + return { + id: row.id, + status: row.status, + family: row.family, + sourceFormat: row.source_format, + targetFormat: row.target_format, + originalFilename: row.original_filename, + inputPath: row.input_path, + outputPath: row.output_path, + inputMimeType: row.input_mime_type, + outputMimeType: row.output_mime_type, + errorMessage: row.error_message, + createdAt: row.created_at, + updatedAt: row.updated_at, + expiresAt: row.expires_at, + }; +} + +export async function createJob(pool, job) { + await pool.query( + `INSERT INTO conversion_jobs + (id, status, family, source_format, target_format, original_filename, input_path, input_mime_type, expires_at) + VALUES (?, 'pending', ?, ?, ?, ?, ?, ?, ?)`, + [ + job.id, + job.family, + job.sourceFormat, + job.targetFormat, + job.originalFilename, + job.inputPath, + job.inputMimeType, + job.expiresAt, + ] + ); +} + +export async function getJobById(pool, id) { + const rows = await pool.query( + `SELECT id, status, family, source_format, target_format, original_filename, + input_path, output_path, input_mime_type, output_mime_type, + error_message, created_at, updated_at, expires_at + FROM conversion_jobs WHERE id = ?`, + [id] + ); + return toCamelJob(rows[0]); +} + +export async function getJobErrorLog(pool, id) { + const rows = await pool.query('SELECT error_log FROM conversion_jobs WHERE id = ?', [id]); + return rows[0]?.error_log ?? null; +} + +export async function markProcessing(pool, id) { + await pool.query("UPDATE conversion_jobs SET status = 'processing' WHERE id = ?", [id]); +} + +export async function markDone(pool, id, { outputPath, outputMimeType }) { + await pool.query( + "UPDATE conversion_jobs SET status = 'done', output_path = ?, output_mime_type = ? WHERE id = ?", + [outputPath, outputMimeType, id] + ); +} + +export async function markFailed(pool, id, { errorMessage, errorLog }) { + await pool.query( + "UPDATE conversion_jobs SET status = 'failed', error_message = ?, error_log = ? WHERE id = ?", + [errorMessage, errorLog, id] + ); +} + +export async function findPendingJobs(pool, limit) { + const rows = await pool.query( + `SELECT id, status, family, source_format, target_format, original_filename, + input_path, output_path, input_mime_type, output_mime_type, + error_message, created_at, updated_at, expires_at + FROM conversion_jobs WHERE status = 'pending' ORDER BY created_at ASC LIMIT ?`, + [limit] + ); + return rows.map(toCamelJob); +} + +export async function findExpiredJobs(pool) { + const rows = await pool.query( + `SELECT id, status, family, source_format, target_format, original_filename, + input_path, output_path, input_mime_type, output_mime_type, + error_message, created_at, updated_at, expires_at + FROM conversion_jobs WHERE expires_at < NOW()` + ); + return rows.map(toCamelJob); +} + +export async function deleteJob(pool, id) { + await pool.query('DELETE FROM conversion_jobs WHERE id = ?', [id]); +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/jobs/jobRepository.test.js` +Expected: PASS (6 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/jobs/jobRepository.js test/jobs/jobRepository.test.js +git commit -m "feat: add conversion_jobs repository" +``` + +--- + +## Task 6: Converter registry + +**Files:** +- Create: `src/converters/registry.js` +- Test: `test/converters/registry.test.js` + +**Interfaces:** +- Produces: + - `export function register({ family, sourceFormat, targetFormat, convert })` — adds an entry. `convert` has signature `async (inputPath, outputPath) => void`. + - `export function resolve(sourceFormat, targetFormat)` — returns the registered `{ family, convert }` or `null` if no converter exists for that pair. + - `export function listTargetFormats(sourceFormat)` — returns an array of every `targetFormat` registered for that `sourceFormat` (empty array if none). + - `export function _resetForTests()` — clears the registry (test-only helper). + +This module holds no conversion logic itself — Tasks 7-10 call `register` at import time to populate it. + +- [ ] **Step 1: Write the failing tests** + +Create `test/converters/registry.test.js`: + +```js +import { describe, it, expect, beforeEach } from 'vitest'; +import { register, resolve, listTargetFormats, _resetForTests } from '../../src/converters/registry.js'; + +beforeEach(() => { + _resetForTests(); +}); + +describe('converter registry', () => { + it('resolves a registered source/target pair', () => { + const convert = async () => {}; + register({ family: 'image', sourceFormat: 'png', targetFormat: 'webp', convert }); + + const entry = resolve('png', 'webp'); + + expect(entry.family).toBe('image'); + expect(entry.convert).toBe(convert); + }); + + it('returns null for an unregistered pair', () => { + expect(resolve('png', 'made-up')).toBeNull(); + }); + + it('lists all target formats registered for a source format', () => { + register({ family: 'image', sourceFormat: 'png', targetFormat: 'webp', convert: async () => {} }); + register({ family: 'image', sourceFormat: 'png', targetFormat: 'jpg', convert: async () => {} }); + register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: async () => {} }); + + expect(listTargetFormats('png').sort()).toEqual(['jpg', 'webp']); + expect(listTargetFormats('docx')).toEqual(['pdf']); + expect(listTargetFormats('unknown-format')).toEqual([]); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx vitest run test/converters/registry.test.js` +Expected: FAIL — `src/converters/registry.js` does not exist yet. + +- [ ] **Step 3: Implement `src/converters/registry.js`** + +```js +let entries = new Map(); + +function key(sourceFormat, targetFormat) { + return `${sourceFormat}->${targetFormat}`; +} + +export function register({ family, sourceFormat, targetFormat, convert }) { + entries.set(key(sourceFormat, targetFormat), { family, convert }); +} + +export function resolve(sourceFormat, targetFormat) { + return entries.get(key(sourceFormat, targetFormat)) ?? null; +} + +export function listTargetFormats(sourceFormat) { + const prefix = `${sourceFormat}->`; + return [...entries.keys()] + .filter((k) => k.startsWith(prefix)) + .map((k) => k.slice(prefix.length)); +} + +export function _resetForTests() { + entries = new Map(); +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx vitest run test/converters/registry.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/converters/registry.js test/converters/registry.test.js +git commit -m "feat: add converter registry" +``` + +--- + +## Task 7: Image converter family (sharp) + +**Files:** +- Create: `src/converters/image.js` +- Test: `test/converters/image.test.js` + +**Interfaces:** +- Consumes: `register` from Task 6's `src/converters/registry.js`. +- Produces: `export function registerImageConverters()` — call once at process startup (from `app.js` and `worker.js` in later tasks) to populate the registry with every `{sourceFormat, targetFormat}` pair among `jpg, jpeg, png, webp, gif, tiff, avif` (source ≠ target), family `'image'`. Each entry's `convert(inputPath, outputPath)` runs `sharp(inputPath).toFormat().toFile(outputPath)`. + +Note: `bmp` is intentionally excluded from this task — `sharp` does not support BMP as an output format (confirmed against the official docs: output formats are JPEG, PNG, WebP, GIF, AVIF, TIFF). BMP support is out of scope for v1 per the design spec. + +- [ ] **Step 1: Write the failing test** + +Create `test/converters/image.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { registerImageConverters } from '../../src/converters/image.js'; +import { resolve, listTargetFormats } from '../../src/converters/registry.js'; +import { detectInputMime } from '../../src/mime.js'; + +let tmpDir; + +beforeAll(async () => { + registerImageConverters(); + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-image-')); +}); + +afterAll(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +describe('image converters', () => { + it('registers every pair among the supported formats', () => { + const targets = listTargetFormats('png').sort(); + expect(targets).toEqual(['avif', 'gif', 'jpeg', 'jpg', 'tiff', 'webp']); + }); + + it('converts a PNG fixture to WebP', async () => { + const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + const outputPath = path.join(tmpDir, 'output.webp'); + const entry = resolve('png', 'webp'); + + await entry.convert(inputPath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('image/webp'); + }); + + it('converts a PNG fixture to JPG', async () => { + const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + const outputPath = path.join(tmpDir, 'output.jpg'); + const entry = resolve('png', 'jpg'); + + await entry.convert(inputPath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('image/jpeg'); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run test/converters/image.test.js` +Expected: FAIL — `src/converters/image.js` does not exist yet. + +- [ ] **Step 3: Implement `src/converters/image.js`** + +```js +import sharp from 'sharp'; +import { register } from './registry.js'; + +const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif']; + +function sharpFormatName(format) { + return format === 'jpg' ? 'jpeg' : format; +} + +export function registerImageConverters() { + for (const sourceFormat of IMAGE_FORMATS) { + for (const targetFormat of IMAGE_FORMATS) { + if (sourceFormat === targetFormat) continue; + + register({ + family: 'image', + sourceFormat, + targetFormat, + convert: async (inputPath, outputPath) => { + await sharp(inputPath).toFormat(sharpFormatName(targetFormat)).toFile(outputPath); + }, + }); + } + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run test/converters/image.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/converters/image.js test/converters/image.test.js +git commit -m "feat: add image converter family (sharp)" +``` + +--- + +## Task 8: Image → PDF converter + +**Files:** +- Create: `src/converters/imageToPdf.js` +- Test: `test/converters/imageToPdf.test.js` + +**Interfaces:** +- Consumes: `register` from Task 6, `sharp` (already a dependency from Task 7). +- Produces: `export function registerImageToPdfConverter()` — registers `{sourceFormat, targetFormat: 'pdf'}` for every format in `jpg, jpeg, png, webp, gif, tiff, avif`, family `'image'`. `convert(inputPath, outputPath)` normalizes the input to a PNG buffer via `sharp(inputPath).png().toBuffer()`, then creates a one-page PDF sized to the image via `pdf-lib`, embeds the PNG, and writes it with `fs.writeFile`. + +- [ ] **Step 1: Write the failing test** + +Create `test/converters/imageToPdf.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { registerImageToPdfConverter } from '../../src/converters/imageToPdf.js'; +import { resolve, listTargetFormats } from '../../src/converters/registry.js'; +import { detectInputMime } from '../../src/mime.js'; + +let tmpDir; + +beforeAll(async () => { + registerImageToPdfConverter(); + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-image-to-pdf-')); +}); + +afterAll(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +describe('image to PDF converter', () => { + it('registers pdf as a target for every image format', () => { + expect(listTargetFormats('png')).toContain('pdf'); + expect(listTargetFormats('jpg')).toContain('pdf'); + }); + + it('converts a PNG fixture into a valid PDF', async () => { + const inputPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + const outputPath = path.join(tmpDir, 'output.pdf'); + const entry = resolve('png', 'pdf'); + + await entry.convert(inputPath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('application/pdf'); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run test/converters/imageToPdf.test.js` +Expected: FAIL — `src/converters/imageToPdf.js` does not exist yet. + +- [ ] **Step 3: Implement `src/converters/imageToPdf.js`** + +```js +import fs from 'node:fs/promises'; +import sharp from 'sharp'; +import { PDFDocument } from 'pdf-lib'; +import { register } from './registry.js'; + +const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif']; + +async function convert(inputPath, outputPath) { + const pngBuffer = await sharp(inputPath).png().toBuffer(); + const metadata = await sharp(pngBuffer).metadata(); + + const pdfDoc = await PDFDocument.create(); + const page = pdfDoc.addPage([metadata.width, metadata.height]); + const embeddedImage = await pdfDoc.embedPng(pngBuffer); + + page.drawImage(embeddedImage, { + x: 0, + y: 0, + width: metadata.width, + height: metadata.height, + }); + + const pdfBytes = await pdfDoc.save(); + await fs.writeFile(outputPath, pdfBytes); +} + +export function registerImageToPdfConverter() { + for (const sourceFormat of IMAGE_FORMATS) { + register({ family: 'image', sourceFormat, targetFormat: 'pdf', convert }); + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run test/converters/imageToPdf.test.js` +Expected: PASS (2 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/converters/imageToPdf.js test/converters/imageToPdf.test.js +git commit -m "feat: add image to PDF converter" +``` + +--- + +## Task 9: Document converters, part A — DOCX→HTML, TXT/HTML→PDF, DOCX→PDF + +**Files:** +- Create: `src/converters/document.js` +- Test: `test/converters/document.test.js` + +**Interfaces:** +- Consumes: `register` from Task 6. +- Produces (all added to `src/converters/document.js`, all called from one exported `export function registerDocumentConverters()`): + - `docx → html` via `mammoth.convertToHtml({ path: inputPath })`, writing `result.value` to `outputPath`. + - `txt → pdf` via reading the text file, wrapping it in a minimal HTML template (escaping `<`, `>`, `&`), and rendering with Puppeteer. + - `html → pdf` via reading the HTML file as-is and rendering with Puppeteer. + - `docx → pdf` by chaining: `mammoth.convertToHtml` then the same Puppeteer rendering used for `html → pdf`. + - A private helper `renderHtmlToPdf(html, outputPath)` shared by the three PDF-producing paths above, using `puppeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox'] })`, `page.setContent(html, { waitUntil: 'networkidle0' })`, `page.pdf({ path: outputPath, format: 'A4', printBackground: true })`, always closing the browser in a `finally` block. + +This task does not yet cover `pdf → text/html` or `pdf → docx` — those are Task 10. + +- [ ] **Step 1: Write the failing test** + +Create `test/converters/document.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { Document, Paragraph, TextRun, Packer } from 'docx'; +import { registerDocumentConverters } from '../../src/converters/document.js'; +import { resolve } from '../../src/converters/registry.js'; +import { detectInputMime } from '../../src/mime.js'; + +let tmpDir; +let docxFixturePath; + +beforeAll(async () => { + registerDocumentConverters(); + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-document-')); + + const doc = new Document({ + sections: [ + { + children: [new Paragraph({ children: [new TextRun('Hello from the fixture document')] })], + }, + ], + }); + const buffer = await Packer.toBuffer(doc); + docxFixturePath = path.join(tmpDir, 'fixture.docx'); + await fs.writeFile(docxFixturePath, buffer); +}); + +afterAll(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +describe('document converters', () => { + it('converts DOCX to HTML containing the source text', async () => { + const outputPath = path.join(tmpDir, 'output.html'); + const entry = resolve('docx', 'html'); + + await entry.convert(docxFixturePath, outputPath); + + const html = await fs.readFile(outputPath, 'utf8'); + expect(html).toContain('Hello from the fixture document'); + }, 20000); + + it('converts TXT to a valid PDF', async () => { + const inputPath = path.join(tmpDir, 'fixture.txt'); + await fs.writeFile(inputPath, 'Plain text content for the PDF'); + const outputPath = path.join(tmpDir, 'from-txt.pdf'); + const entry = resolve('txt', 'pdf'); + + await entry.convert(inputPath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('application/pdf'); + }, 20000); + + it('converts HTML to a valid PDF', async () => { + const inputPath = path.join(tmpDir, 'fixture.html'); + await fs.writeFile(inputPath, '

Hello HTML

'); + const outputPath = path.join(tmpDir, 'from-html.pdf'); + const entry = resolve('html', 'pdf'); + + await entry.convert(inputPath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('application/pdf'); + }, 20000); + + it('converts DOCX to a valid PDF', async () => { + const outputPath = path.join(tmpDir, 'from-docx.pdf'); + const entry = resolve('docx', 'pdf'); + + await entry.convert(docxFixturePath, outputPath); + + const detected = await detectInputMime(outputPath); + expect(detected.mime).toBe('application/pdf'); + }, 20000); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run test/converters/document.test.js` +Expected: FAIL — `src/converters/document.js` does not exist yet. + +- [ ] **Step 3: Implement the Puppeteer rendering helper and DOCX/TXT/HTML converters in `src/converters/document.js`** + +```js +import fs from 'node:fs/promises'; +import mammoth from 'mammoth'; +import puppeteer from 'puppeteer'; +import { register } from './registry.js'; + +async function renderHtmlToPdf(html, outputPath) { + const browser = await puppeteer.launch({ + headless: true, + args: ['--no-sandbox', '--disable-setuid-sandbox'], + }); + try { + const page = await browser.newPage(); + await page.setContent(html, { waitUntil: 'networkidle0' }); + await page.pdf({ path: outputPath, format: 'A4', printBackground: true }); + } finally { + await browser.close(); + } +} + +function escapeHtml(text) { + return text.replace(/&/g, '&').replace(//g, '>'); +} + +async function convertDocxToHtml(inputPath, outputPath) { + const result = await mammoth.convertToHtml({ path: inputPath }); + await fs.writeFile(outputPath, result.value); +} + +async function convertTxtToPdf(inputPath, outputPath) { + const text = await fs.readFile(inputPath, 'utf8'); + const html = `
${escapeHtml(text)}
`; + await renderHtmlToPdf(html, outputPath); +} + +async function convertHtmlToPdf(inputPath, outputPath) { + const html = await fs.readFile(inputPath, 'utf8'); + await renderHtmlToPdf(html, outputPath); +} + +async function convertDocxToPdf(inputPath, outputPath) { + const result = await mammoth.convertToHtml({ path: inputPath }); + await renderHtmlToPdf(result.value, outputPath); +} + +export function registerDocumentConverters() { + register({ family: 'document', sourceFormat: 'docx', targetFormat: 'html', convert: convertDocxToHtml }); + register({ family: 'document', sourceFormat: 'txt', targetFormat: 'pdf', convert: convertTxtToPdf }); + register({ family: 'document', sourceFormat: 'html', targetFormat: 'pdf', convert: convertHtmlToPdf }); + register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: convertDocxToPdf }); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run test/converters/document.test.js` +Expected: PASS (4 tests). Puppeteer launches a real headless Chromium per test — this is slower than the other unit tests (hence the 20000ms timeouts); this is expected. + +- [ ] **Step 5: Commit** + +```bash +git add src/converters/document.js test/converters/document.test.js +git commit -m "feat: add DOCX/TXT/HTML document converters" +``` + +--- + +## Task 10: Document converters, part B — PDF→text/HTML, PDF→DOCX (best-effort) + +**Files:** +- Modify: `src/converters/document.js` (add to the same file and to `registerDocumentConverters()`) +- Test: `test/converters/documentFromPdf.test.js` + +**Interfaces:** +- Consumes: `register` from Task 6, the fixture-generation pattern from Task 9 (generate inputs at test time instead of committing binary fixtures). +- Produces, added to `src/converters/document.js`: + - A private helper `async function extractPdfPageTexts(inputPath)` returning `string[]`, one entry per page, using `pdfjs-dist/legacy/build/pdf.mjs`'s `getDocument({ data }).promise` and `page.getTextContent()`. + - `pdf → txt`: joins page texts with `\n\n` and writes as plain text. + - `pdf → html`: wraps each page's text in a `

` tag (HTML-escaped) and writes a minimal HTML document. + - `pdf → docx`: builds one `docx` `Paragraph` per page text and writes via `Packer.toBuffer`. This is explicitly a **best-effort text reconstruction** — no attempt is made to preserve original layout, tables, or image positions, matching the limitation documented in the design spec. + +- [ ] **Step 1: Write the failing test** + +Create `test/converters/documentFromPdf.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { PDFDocument, StandardFonts } from 'pdf-lib'; +import { registerDocumentConverters } from '../../src/converters/document.js'; +import { resolve } from '../../src/converters/registry.js'; + +let tmpDir; +let pdfFixturePath; + +beforeAll(async () => { + registerDocumentConverters(); + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-document-from-pdf-')); + + const pdfDoc = await PDFDocument.create(); + const page = pdfDoc.addPage([600, 400]); + const font = await pdfDoc.embedFont(StandardFonts.Helvetica); + page.drawText('Extractable fixture text', { x: 50, y: 350, size: 24, font }); + const bytes = await pdfDoc.save(); + pdfFixturePath = path.join(tmpDir, 'fixture.pdf'); + await fs.writeFile(pdfFixturePath, bytes); +}); + +afterAll(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); +}); + +describe('PDF source document converters', () => { + it('extracts text from PDF to TXT', async () => { + const outputPath = path.join(tmpDir, 'output.txt'); + const entry = resolve('pdf', 'txt'); + + await entry.convert(pdfFixturePath, outputPath); + + const text = await fs.readFile(outputPath, 'utf8'); + expect(text).toContain('Extractable fixture text'); + }); + + it('extracts text from PDF to HTML', async () => { + const outputPath = path.join(tmpDir, 'output.html'); + const entry = resolve('pdf', 'html'); + + await entry.convert(pdfFixturePath, outputPath); + + const html = await fs.readFile(outputPath, 'utf8'); + expect(html).toContain('Extractable fixture text'); + expect(html).toContain('

'); + }); + + it('reconstructs PDF text into a DOCX (best-effort)', async () => { + const outputPath = path.join(tmpDir, 'output.docx'); + const entry = resolve('pdf', 'docx'); + + await entry.convert(pdfFixturePath, outputPath); + + const stat = await fs.stat(outputPath); + expect(stat.size).toBeGreaterThan(0); + + const mammoth = await import('mammoth'); + const result = await mammoth.default.convertToHtml({ path: outputPath }); + expect(result.value).toContain('Extractable fixture text'); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run test/converters/documentFromPdf.test.js` +Expected: FAIL — `pdf` is not yet a registered source format. + +- [ ] **Step 3: Add the PDF-source converters to `src/converters/document.js`** + +Add these imports at the top of `src/converters/document.js` (alongside the existing ones): + +```js +import * as pdfjsLib from 'pdfjs-dist/legacy/build/pdf.mjs'; +import { Document, Paragraph, TextRun, Packer } from 'docx'; +``` + +Add these functions to the same file: + +```js +async function extractPdfPageTexts(inputPath) { + const data = new Uint8Array(await fs.readFile(inputPath)); + const doc = await pdfjsLib.getDocument({ data }).promise; + + const pageTexts = []; + for (let pageNum = 1; pageNum <= doc.numPages; pageNum += 1) { + const page = await doc.getPage(pageNum); + const content = await page.getTextContent(); + pageTexts.push(content.items.map((item) => item.str).join(' ')); + } + return pageTexts; +} + +async function convertPdfToTxt(inputPath, outputPath) { + const pageTexts = await extractPdfPageTexts(inputPath); + await fs.writeFile(outputPath, pageTexts.join('\n\n')); +} + +async function convertPdfToHtml(inputPath, outputPath) { + const pageTexts = await extractPdfPageTexts(inputPath); + const body = pageTexts.map((text) => `

${escapeHtml(text)}

`).join('\n'); + await fs.writeFile(outputPath, `${body}`); +} + +async function convertPdfToDocx(inputPath, outputPath) { + const pageTexts = await extractPdfPageTexts(inputPath); + const doc = new Document({ + sections: [ + { + children: pageTexts.map((text) => new Paragraph({ children: [new TextRun(text)] })), + }, + ], + }); + const buffer = await Packer.toBuffer(doc); + await fs.writeFile(outputPath, buffer); +} +``` + +Update `registerDocumentConverters()` to also register these three: + +```js +export function registerDocumentConverters() { + register({ family: 'document', sourceFormat: 'docx', targetFormat: 'html', convert: convertDocxToHtml }); + register({ family: 'document', sourceFormat: 'txt', targetFormat: 'pdf', convert: convertTxtToPdf }); + register({ family: 'document', sourceFormat: 'html', targetFormat: 'pdf', convert: convertHtmlToPdf }); + register({ family: 'document', sourceFormat: 'docx', targetFormat: 'pdf', convert: convertDocxToPdf }); + register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'txt', convert: convertPdfToTxt }); + register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'html', convert: convertPdfToHtml }); + register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'docx', convert: convertPdfToDocx }); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run test/converters/documentFromPdf.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 5: Run the full document test suite to confirm no regression** + +Run: `npx vitest run test/converters/document.test.js test/converters/documentFromPdf.test.js` +Expected: PASS (7 tests total) + +- [ ] **Step 6: Commit** + +```bash +git add src/converters/document.js test/converters/documentFromPdf.test.js +git commit -m "feat: add best-effort PDF source converters (txt, html, docx)" +``` + +--- + +## Task 11: Input format validation helper (magic-byte vs. declared extension) + +**Files:** +- Modify: `src/mime.js` +- Test: `test/mime.test.js` (extend) + +**Interfaces:** +- Produces: `export async function resolveInputFormat(filePath, declaredFormat)` in `src/mime.js` — returns `{ mime: string, valid: boolean }`. + - Sniffs magic bytes via `detectInputMime`. + - If a signature is found, `valid` is `true` only when the detected extension matches `declaredFormat` (treating `jpg`/`jpeg` as equivalent — `file-type` returns `jpg` for JPEG, but both are valid declared extensions in this app). + - If no signature is found (expected for `txt` and `html`, which have no magic bytes), falls back to trusting `declaredFormat` **only** when it is `txt` or `html`, returning the matching static MIME type. Any other undetectable format is `valid: false`. + +- [ ] **Step 1: Write the failing tests (append to `test/mime.test.js`)** + +Add to `test/mime.test.js`: + +```js +import { resolveInputFormat } from '../src/mime.js'; +``` + +(add this import alongside the existing ones at the top of the file), then add: + +```js +describe('resolveInputFormat', () => { + it('accepts a PNG file declared as png', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + + const result = await resolveInputFormat(fixturePath, 'png'); + + expect(result).toEqual({ mime: 'image/png', valid: true }); + }); + + it('rejects a PNG file declared as a different format', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + + const result = await resolveInputFormat(fixturePath, 'pdf'); + + expect(result.valid).toBe(false); + }); + + it('treats jpg and jpeg as equivalent declared formats', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + const result = await resolveInputFormat(fixturePath, 'png'); + expect(result.valid).toBe(true); + }); + + it('trusts the declared format for undetectable txt files', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.txt'); + await fs.writeFile(fixturePath, 'plain text, no magic bytes'); + + const result = await resolveInputFormat(fixturePath, 'txt'); + + expect(result).toEqual({ mime: 'text/plain', valid: true }); + + await fs.unlink(fixturePath); + }); + + it('rejects an undetectable file declared as a binary format', async () => { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample-fake.png'); + await fs.writeFile(fixturePath, 'this is not really a PNG'); + + const result = await resolveInputFormat(fixturePath, 'png'); + + expect(result.valid).toBe(false); + + await fs.unlink(fixturePath); + }); +}); +``` + +Also add `import fs from 'node:fs/promises';` at the top of `test/mime.test.js` if not already present. + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npx vitest run test/mime.test.js` +Expected: FAIL — `resolveInputFormat` is not exported yet. + +- [ ] **Step 3: Implement `resolveInputFormat` in `src/mime.js`** + +Add to `src/mime.js`: + +```js +const UNDETECTABLE_TEXT_FORMATS = { + txt: 'text/plain', + html: 'text/html', +}; + +function normalizeFormat(format) { + return format === 'jpg' ? 'jpeg' : format; +} + +export async function resolveInputFormat(filePath, declaredFormat) { + const detected = await detectInputMime(filePath); + + if (!detected) { + const fallbackMime = UNDETECTABLE_TEXT_FORMATS[declaredFormat]; + if (fallbackMime) { + return { mime: fallbackMime, valid: true }; + } + return { mime: null, valid: false }; + } + + const valid = normalizeFormat(detected.ext) === normalizeFormat(declaredFormat); + return { mime: detected.mime, valid }; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npx vitest run test/mime.test.js` +Expected: PASS (8 tests total) + +- [ ] **Step 5: Commit** + +```bash +git add src/mime.js test/mime.test.js +git commit -m "feat: validate declared source format against sniffed magic bytes" +``` + +--- + +## Task 12: Express app — `POST /api/jobs` and `GET /api/formats` + +**Files:** +- Create: `src/app.js` +- Test: `test/api/jobs.test.js` + +**Interfaces:** +- Consumes: `resolveInputFormat` (Task 11), `resolve`/`listTargetFormats` (Task 6), `registerImageConverters`/`registerImageToPdfConverter`/`registerDocumentConverters` (Tasks 7-10), `createJob`/`getJobById` (Task 5), `ensureStorageDirs`/`deleteIfExists` (Task 3), `getPool` (Task 2). +- Produces: `export function createApp(config, pool)` — returns a configured Express `app` (does not call `.listen`). Registers all converters as a side effect the first time it's called (safe to call once per process). + +**Request contract for `POST /api/jobs`:** +- `multipart/form-data` with a `files` field (one or more files, field name repeated) and a `targetFormats` field: a JSON-encoded array of strings, same length and order as `files`. +- Response `201`: `{ "jobs": [ { "file": "photo.png", "id": "", "status": "pending" } | { "file": "photo.png", "error": "" }, ... ] }` — one entry per uploaded file, in the same order. A file-level problem never fails the whole request; only a malformed request (no files, `targetFormats` missing/wrong length) returns `400` with no jobs created. + +- [ ] **Step 1: Write the failing tests** + +Create `test/api/jobs.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { createApp } from '../../src/app.js'; +import { getPool, closePool } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { getJobById } from '../../src/jobs/jobRepository.js'; +import { ensureStorageDirs } from '../../src/storage.js'; + +let app; +let pool; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-api-')) }; + await ensureStorageDirs(config); + pool = getPool(config); + app = createApp(config, pool); +}); + +afterAll(async () => { + await closePool(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +describe('GET /api/formats', () => { + it('lists valid target formats for a known source format', async () => { + const response = await request(app).get('/api/formats').query({ source: 'png' }); + + expect(response.status).toBe(200); + expect(response.body.targets).toContain('webp'); + expect(response.body.targets).toContain('pdf'); + }); + + it('returns an empty list for an unknown source format', async () => { + const response = await request(app).get('/api/formats').query({ source: 'made-up' }); + + expect(response.status).toBe(200); + expect(response.body.targets).toEqual([]); + }); +}); + +describe('POST /api/jobs', () => { + it('creates a pending job for a valid image upload', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs).toHaveLength(1); + expect(response.body.jobs[0].status).toBe('pending'); + expect(response.body.jobs[0].file).toBe('photo.png'); + + const job = await getJobById(pool, response.body.jobs[0].id); + expect(job.status).toBe('pending'); + expect(job.sourceFormat).toBe('png'); + expect(job.targetFormat).toBe('webp'); + expect(job.originalFilename).toBe('photo.png'); + expect(job.inputMimeType).toBe('image/png'); + }); + + it('rejects a file whose content does not match its extension, without failing the whole batch', async () => { + const fakePath = path.join(config.storageDir, 'fake.png'); + await fs.writeFile(fakePath, 'not actually a png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fakePath, 'fake.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/does not match/); + + await fs.unlink(fakePath); + }); + + it('rejects an unsupported source/target pair', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['made-up-format'])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(201); + expect(response.body.jobs[0].error).toMatch(/Unsupported conversion/); + }); + + it('returns 400 when targetFormats length does not match the number of files', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify([])) + .attach('files', fixturePath, 'photo.png'); + + expect(response.status).toBe(400); + }); + + it('returns 400 when no files are uploaded', async () => { + const response = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify([])); + + expect(response.status).toBe(400); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobs.test.js` +Expected: FAIL — `src/app.js` does not exist yet. + +- [ ] **Step 3: Implement `src/app.js`** + +```js +import path from 'node:path'; +import express from 'express'; +import multer from 'multer'; +import { rateLimit } from 'express-rate-limit'; +import { v4 as uuidv4 } from 'uuid'; +import { resolve as resolveConverter, listTargetFormats } from './converters/registry.js'; +import { registerImageConverters } from './converters/image.js'; +import { registerImageToPdfConverter } from './converters/imageToPdf.js'; +import { registerDocumentConverters } from './converters/document.js'; +import { resolveInputFormat } from './mime.js'; +import { deleteIfExists } from './storage.js'; +import { createJob, getJobById } from './jobs/jobRepository.js'; + +let convertersRegistered = false; + +function registerAllConverters() { + if (convertersRegistered) return; + registerImageConverters(); + registerImageToPdfConverter(); + registerDocumentConverters(); + convertersRegistered = true; +} + +export function createApp(config, pool) { + registerAllConverters(); + + const app = express(); + + const storage = multer.diskStorage({ + destination: (req, file, cb) => cb(null, path.join(config.storageDir, 'uploads')), + filename: (req, file, cb) => { + const ext = path.extname(file.originalname).slice(1).toLowerCase(); + cb(null, `${uuidv4()}.${ext}`); + }, + }); + + const upload = multer({ + storage, + limits: { + fileSize: config.maxFileSizeMb * 1024 * 1024, + files: 10, + }, + }); + + const jobsRateLimiter = rateLimit({ + windowMs: config.rateLimitWindowMinutes * 60 * 1000, + limit: config.rateLimitMaxJobs, + standardHeaders: 'draft-8', + legacyHeaders: false, + }); + + app.get('/api/formats', (req, res) => { + const source = String(req.query.source ?? '').toLowerCase(); + res.json({ targets: listTargetFormats(source) }); + }); + + app.post('/api/jobs', jobsRateLimiter, upload.array('files', 10), async (req, res) => { + if (!req.files || req.files.length === 0) { + return res.status(400).json({ error: 'No files uploaded' }); + } + + let targetFormats; + try { + targetFormats = JSON.parse(req.body.targetFormats ?? '[]'); + } catch { + return res.status(400).json({ error: 'targetFormats must be a JSON array' }); + } + + if (!Array.isArray(targetFormats) || targetFormats.length !== req.files.length) { + return res.status(400).json({ error: 'targetFormats must have one entry per uploaded file' }); + } + + const results = []; + for (let i = 0; i < req.files.length; i += 1) { + const file = req.files[i]; + const targetFormat = targetFormats[i]; + const id = path.basename(file.filename, path.extname(file.filename)); + const sourceFormat = path.extname(file.filename).slice(1).toLowerCase(); + + const { mime, valid } = await resolveInputFormat(file.path, sourceFormat); + if (!valid) { + await deleteIfExists(file.path); + results.push({ file: file.originalname, error: 'File content does not match its extension' }); + continue; + } + + const registryEntry = resolveConverter(sourceFormat, targetFormat); + if (!registryEntry) { + await deleteIfExists(file.path); + results.push({ + file: file.originalname, + error: `Unsupported conversion: ${sourceFormat} to ${targetFormat}`, + }); + continue; + } + + const expiresAt = new Date(Date.now() + config.retentionHours * 3600 * 1000); + await createJob(pool, { + id, + family: registryEntry.family, + sourceFormat, + targetFormat, + originalFilename: file.originalname, + inputPath: file.filename, + inputMimeType: mime, + expiresAt, + }); + + results.push({ file: file.originalname, id, status: 'pending' }); + } + + res.status(201).json({ jobs: results }); + }); + + return app; +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobs.test.js` +Expected: PASS (7 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/app.js test/api/jobs.test.js +git commit -m "feat: add POST /api/jobs and GET /api/formats endpoints" +``` + +--- + +## Task 13: `GET /api/jobs/:id` and `GET /api/jobs/:id/download` + +**Files:** +- Modify: `src/app.js` (add two routes) +- Test: `test/api/jobStatus.test.js` + +**Interfaces:** +- Consumes: `getJobById` (Task 5), `outputPath` (Task 3). +- `GET /api/jobs/:id` → `200` with `{ id, status, originalFilename, sourceFormat, targetFormat, errorMessage }` (never `errorLog`), or `404` if the id doesn't exist. +- `GET /api/jobs/:id/download` → streams the file with `Content-Type` set from `output_mime_type` and `Content-Disposition: attachment` carrying `original_filename` (RFC 5987-encoded to safely support non-ASCII names and prevent header injection). `404` if the id doesn't exist, `409` if the job exists but isn't `done` yet. + +- [ ] **Step 1: Write the failing tests** + +Create `test/api/jobStatus.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { createApp } from '../../src/app.js'; +import { getPool, closePool } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { createJob, markDone } from '../../src/jobs/jobRepository.js'; +import { ensureStorageDirs, outputPath } from '../../src/storage.js'; + +let app; +let pool; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-status-')) }; + await ensureStorageDirs(config); + pool = getPool(config); + app = createApp(config, pool); +}); + +afterAll(async () => { + await closePool(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +function baseJob(id) { + return { + id, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'holiday photo.png', + inputPath: `${id}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() + 3600 * 1000), + }; +} + +describe('GET /api/jobs/:id', () => { + it('returns job status without the error log field', async () => { + const id = '66666666-6666-4666-8666-666666666666'; + await createJob(pool, baseJob(id)); + + const response = await request(app).get(`/api/jobs/${id}`); + + expect(response.status).toBe(200); + expect(response.body.status).toBe('pending'); + expect(response.body.originalFilename).toBe('holiday photo.png'); + expect(response.body.errorLog).toBeUndefined(); + }); + + it('returns 404 for an unknown id', async () => { + const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000'); + expect(response.status).toBe(404); + }); +}); + +describe('GET /api/jobs/:id/download', () => { + it('streams the converted file with correct headers once done', async () => { + const id = '77777777-7777-4777-8777-777777777777'; + await createJob(pool, baseJob(id)); + const filePath = outputPath(config, id, 'webp'); + await fs.writeFile(filePath, Buffer.from('fake webp bytes')); + await markDone(pool, id, { outputPath: `${id}.webp`, outputMimeType: 'image/webp' }); + + const response = await request(app).get(`/api/jobs/${id}/download`); + + expect(response.status).toBe(200); + expect(response.headers['content-type']).toBe('image/webp'); + expect(response.headers['content-disposition']).toContain('holiday photo.png'); + expect(response.body ?? response.text).toBeDefined(); + }); + + it('returns 409 when the job is not done yet', async () => { + const id = '88888888-8888-4888-8888-888888888888'; + await createJob(pool, baseJob(id)); + + const response = await request(app).get(`/api/jobs/${id}/download`); + + expect(response.status).toBe(409); + }); + + it('returns 404 for an unknown id', async () => { + const response = await request(app).get('/api/jobs/00000000-0000-4000-8000-000000000000/download'); + expect(response.status).toBe(404); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobStatus.test.js` +Expected: FAIL — routes don't exist yet. + +- [ ] **Step 3: Add the two routes to `src/app.js`** + +Add this import at the top of `src/app.js`: + +```js +import fs from 'node:fs'; +import { outputPath } from './storage.js'; +``` + +Add this helper function and the two routes inside `createApp`, before `return app;`: + +```js +function contentDispositionHeader(filename) { + const asciiFallback = filename.replace(/[^\x20-\x7E]/g, '_').replace(/"/g, "'"); + const encoded = encodeURIComponent(filename); + return `attachment; filename="${asciiFallback}"; filename*=UTF-8''${encoded}`; +} +``` + +```js + app.get('/api/jobs/:id', async (req, res) => { + const job = await getJobById(pool, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } + + res.json({ + id: job.id, + status: job.status, + originalFilename: job.originalFilename, + sourceFormat: job.sourceFormat, + targetFormat: job.targetFormat, + errorMessage: job.errorMessage, + }); + }); + + app.get('/api/jobs/:id/download', async (req, res) => { + const job = await getJobById(pool, req.params.id); + if (!job) { + return res.status(404).json({ error: 'Job not found' }); + } + if (job.status !== 'done') { + return res.status(409).json({ error: `Job is not ready yet (status: ${job.status})` }); + } + + const filePath = outputPath(config, job.id, job.targetFormat); + res.set('Content-Type', job.outputMimeType); + res.set('Content-Disposition', contentDispositionHeader(job.originalFilename)); + fs.createReadStream(filePath).pipe(res); + }); +``` + +**Note on error handling:** `npm install express` (Task 1) installs the current major version, Express 5, where an async route handler's rejected promise is automatically forwarded to error-handling middleware (no manual `try/catch`-to-`next` plumbing needed) — confirmed against the official Express 5 migration guide. Add the error-handling middleware itself now, as the last thing registered in `createApp`, so it can catch errors from every route defined so far (Task 17 will insert its static-file serving *before* this middleware, never after — an error handler registered before a route can't catch that route's errors): + +```js + app.use((err, req, res, next) => { + console.error('Unhandled API error:', err); + res.status(500).json({ error: 'Internal server error' }); + }); +``` + +Add this immediately before `return app;` at the end of `createApp`. + +Add this test to `test/api/jobStatus.test.js`, in a new `describe` block: + +```js +describe('unexpected server errors', () => { + it('returns a generic 500 without leaking internal error details', async () => { + const originalQuery = pool.query.bind(pool); + pool.query = () => Promise.reject(new Error('connection reset by peer')); + + const response = await request(app).get('/api/jobs/99999999-9999-4999-8999-999999999999'); + + pool.query = originalQuery; + + expect(response.status).toBe(500); + expect(response.body.error).toBe('Internal server error'); + expect(JSON.stringify(response.body)).not.toContain('connection reset by peer'); + }); +}); +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/jobStatus.test.js` +Expected: PASS (6 tests) + +- [ ] **Step 5: Run the full API test suite to confirm no regression** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api` +Expected: PASS (13 tests total) + +- [ ] **Step 6: Commit** + +```bash +git add src/app.js test/api/jobStatus.test.js +git commit -m "feat: add job status and download endpoints" +``` + +--- + +## Task 14: Server entry point + +**Files:** +- Create: `src/server.js` +- Modify: `package.json` (fix the `start` script from Task 1, which pointed at a file that was never created) + +**Interfaces:** +- Consumes: `loadConfig` (Task 1), `ensureStorageDirs` (Task 3), `getPool` (Task 2), `createApp` (Task 12). +- Produces: a runnable entry point with no exports of its own — this is the process Passenger starts. + +- [ ] **Step 1: Create `src/server.js`** + +```js +import { loadConfig } from './config.js'; +import { getPool } from './db.js'; +import { ensureStorageDirs } from './storage.js'; +import { createApp } from './app.js'; + +async function main() { + const config = loadConfig(); + await ensureStorageDirs(config); + const pool = getPool(config); + const app = createApp(config, pool); + + app.listen(config.port, () => { + console.log(`File converter API listening on port ${config.port}`); + }); +} + +main().catch((error) => { + console.error('Failed to start server:', error); + process.exit(1); +}); +``` + +- [ ] **Step 2: Update `package.json`'s `start` script** + +Change: + +```json +"start": "node src/app.js", +``` + +to: + +```json +"start": "node src/server.js", +``` + +- [ ] **Step 3: Verify the server starts** + +Run (with a reachable MariaDB and matching `.env`, copy `.env.example` to `.env` and adjust values first): + +```bash +npm start +``` + +Expected: console prints `File converter API listening on port 3000` (or your configured port), process stays running. Stop with Ctrl+C. + +- [ ] **Step 4: Commit** + +```bash +git add src/server.js package.json +git commit -m "feat: add server entry point" +``` + +--- + +## Task 15: Worker process + +**Files:** +- Create: `src/worker.js` +- Test: `test/worker.test.js` + +**Interfaces:** +- Consumes: `findPendingJobs`/`markProcessing`/`markDone`/`markFailed` (Task 5), `resolve` (Task 6), `uploadPath`/`outputPath`/`deleteIfExists` (Task 3), `outputMimeType` (Task 4), the three `register*Converters` functions (Tasks 7-10). +- Produces: + - `export async function processPendingJobs(pool, config)` — fetches up to `config.workerConcurrency` pending jobs and processes them concurrently; returns the number of jobs it picked up. **This is the function the test suite calls directly** — it does one pass and returns, making it usable both by the real polling loop and by tests/Task 18's end-to-end test, without needing to wait on a timer. + - `export function startWorker(pool, config)` — sets up a `setInterval` calling `processPendingJobs` every `config.workerPollIntervalMs`, returns a stop function. + - A `main()` that registers all converters and calls `startWorker`, run only when this file is executed directly (not on import). + - Each job conversion is bounded by a 60-second timeout; a timeout or thrown error marks the job `failed` with a generic `errorMessage` and the full error in `errorLog`, and any partial output file is deleted. + +- [ ] **Step 1: Write the failing tests** + +Create `test/worker.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { getPool, closePool } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; +import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; +import { createJob, getJobById, getJobErrorLog } from '../src/jobs/jobRepository.js'; +import { registerImageConverters } from '../src/converters/image.js'; +import { processPendingJobs } from '../src/worker.js'; + +let pool; +let config; + +beforeAll(async () => { + registerImageConverters(); + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-worker-')) }; + await ensureStorageDirs(config); + pool = getPool(config); +}); + +afterAll(async () => { + await closePool(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +async function createPendingImageJob(id, sourceFormat, targetFormat) { + const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.png'); + const inputFilePath = uploadPath(config, id, sourceFormat); + await fs.copyFile(fixturePath, inputFilePath); + + await createJob(pool, { + id, + family: 'image', + sourceFormat, + targetFormat, + originalFilename: `photo.${sourceFormat}`, + inputPath: `${id}.${sourceFormat}`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() + 3600 * 1000), + }); +} + +describe('processPendingJobs', () => { + it('converts a pending image job to done', async () => { + const id = '99999999-9999-4999-8999-999999999999'; + await createPendingImageJob(id, 'png', 'webp'); + + const processedCount = await processPendingJobs(pool, config); + + expect(processedCount).toBe(1); + const job = await getJobById(pool, id); + expect(job.status).toBe('done'); + expect(job.outputPath).toBe(`${id}.webp`); + expect(job.outputMimeType).toBe('image/webp'); + + const stat = await fs.stat(outputPath(config, id, 'webp')); + expect(stat.size).toBeGreaterThan(0); + }); + + it('marks a job failed with a safe message and a detailed log when the converter throws', async () => { + const id = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa'; + await createJob(pool, { + id, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'missing.png', + inputPath: `${id}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() + 3600 * 1000), + }); + // Note: input file is intentionally never written, so sharp will throw ENOENT. + + await processPendingJobs(pool, config); + + const job = await getJobById(pool, id); + expect(job.status).toBe('failed'); + expect(job.errorMessage).toBe('Conversion failed, please try again.'); + + const errorLog = await getJobErrorLog(pool, id); + expect(errorLog).toMatch(/ENOENT|no such file/i); + }); + + it('only picks up as many jobs as workerConcurrency allows', async () => { + await createPendingImageJob('bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', 'png', 'webp'); + await createPendingImageJob('cccccccc-cccc-4ccc-8ccc-cccccccccccc', 'png', 'webp'); + await createPendingImageJob('dddddddd-dddd-4ddd-8ddd-dddddddddddd', 'png', 'webp'); + + const limitedConfig = { ...config, workerConcurrency: 2 }; + const processedCount = await processPendingJobs(pool, limitedConfig); + + expect(processedCount).toBe(2); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/worker.test.js` +Expected: FAIL — `src/worker.js` does not exist yet. + +- [ ] **Step 3: Implement `src/worker.js`** + +```js +import { loadConfig } from './config.js'; +import { getPool } from './db.js'; +import { ensureStorageDirs, uploadPath, outputPath, deleteIfExists } from './storage.js'; +import { outputMimeType } from './mime.js'; +import { resolve as resolveConverter } from './converters/registry.js'; +import { registerImageConverters } from './converters/image.js'; +import { registerImageToPdfConverter } from './converters/imageToPdf.js'; +import { registerDocumentConverters } from './converters/document.js'; +import { findPendingJobs, markProcessing, markDone, markFailed } from './jobs/jobRepository.js'; + +const JOB_TIMEOUT_MS = 60000; + +function withTimeout(promise, ms) { + let timeoutId; + const timeout = new Promise((_, reject) => { + timeoutId = setTimeout(() => reject(new Error(`Conversion timed out after ${ms}ms`)), ms); + }); + return Promise.race([promise, timeout]).finally(() => clearTimeout(timeoutId)); +} + +async function processJob(pool, config, job) { + await markProcessing(pool, job.id); + + const inputFilePath = uploadPath(config, job.id, job.sourceFormat); + const outputFilePath = outputPath(config, job.id, job.targetFormat); + + try { + const entry = resolveConverter(job.sourceFormat, job.targetFormat); + if (!entry) { + throw new Error(`No converter registered for ${job.sourceFormat} -> ${job.targetFormat}`); + } + + await withTimeout(entry.convert(inputFilePath, outputFilePath), JOB_TIMEOUT_MS); + + await markDone(pool, job.id, { + outputPath: `${job.id}.${job.targetFormat}`, + outputMimeType: outputMimeType(job.targetFormat), + }); + } catch (error) { + await deleteIfExists(outputFilePath); + await markFailed(pool, job.id, { + errorMessage: 'Conversion failed, please try again.', + errorLog: error.stack ?? String(error), + }); + } +} + +export async function processPendingJobs(pool, config) { + const jobs = await findPendingJobs(pool, config.workerConcurrency); + await Promise.all(jobs.map((job) => processJob(pool, config, job))); + return jobs.length; +} + +export function startWorker(pool, config) { + const interval = setInterval(() => { + processPendingJobs(pool, config).catch((error) => { + console.error('Error while processing pending jobs:', error); + }); + }, config.workerPollIntervalMs); + return () => clearInterval(interval); +} + +async function main() { + const config = loadConfig(); + await ensureStorageDirs(config); + const pool = getPool(config); + + registerImageConverters(); + registerImageToPdfConverter(); + registerDocumentConverters(); + + startWorker(pool, config); + console.log(`Worker started, polling every ${config.workerPollIntervalMs}ms`); +} + +if (import.meta.url === `file://${process.argv[1]}`) { + main(); +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/worker.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/worker.js test/worker.test.js +git commit -m "feat: add worker process with bounded concurrency and per-job timeout" +``` + +--- + +## Task 16: Cleanup script and deployment files + +**Files:** +- Create: `src/cleanup.js` +- Create: `ecosystem.config.cjs` +- Create: `README.md` (deployment section) +- Test: `test/cleanup.test.js` + +**Interfaces:** +- Consumes: `findExpiredJobs`/`deleteJob` (Task 5), `uploadPath`/`outputPath`/`deleteIfExists` (Task 3). +- Produces: `export async function runCleanup(pool, config)` — deletes every expired job's row and its upload/output files (tolerating a missing output file for jobs that never finished), returns the number of jobs removed. A `main()` runs it once and exits, for use from cron. + +- [ ] **Step 1: Write the failing tests** + +Create `test/cleanup.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { getPool, closePool } from '../src/db.js'; +import { loadConfig } from '../src/config.js'; +import { ensureStorageDirs, uploadPath, outputPath } from '../src/storage.js'; +import { createJob, markDone, getJobById } from '../src/jobs/jobRepository.js'; +import { runCleanup } from '../src/cleanup.js'; + +let pool; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-cleanup-')) }; + await ensureStorageDirs(config); + pool = getPool(config); +}); + +afterAll(async () => { + await closePool(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +describe('runCleanup', () => { + it('deletes an expired done job, its input file, and its output file', async () => { + const id = 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee'; + await fs.writeFile(uploadPath(config, id, 'png'), 'input bytes'); + await fs.writeFile(outputPath(config, id, 'webp'), 'output bytes'); + await createJob(pool, { + id, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${id}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() - 1000), + }); + await markDone(pool, id, { outputPath: `${id}.webp`, outputMimeType: 'image/webp' }); + + const deletedCount = await runCleanup(pool, config); + + expect(deletedCount).toBe(1); + expect(await getJobById(pool, id)).toBeNull(); + await expect(fs.stat(uploadPath(config, id, 'png'))).rejects.toThrow(); + await expect(fs.stat(outputPath(config, id, 'webp'))).rejects.toThrow(); + }); + + it('deletes an expired pending job (with no output file) without throwing', async () => { + const id = 'ffffffff-ffff-4fff-8fff-ffffffffffff'; + await fs.writeFile(uploadPath(config, id, 'png'), 'input bytes'); + await createJob(pool, { + id, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${id}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() - 1000), + }); + + const deletedCount = await runCleanup(pool, config); + + expect(deletedCount).toBe(1); + expect(await getJobById(pool, id)).toBeNull(); + }); + + it('leaves non-expired jobs untouched', async () => { + const id = '12121212-1212-4212-8212-121212121212'; + await createJob(pool, { + id, + family: 'image', + sourceFormat: 'png', + targetFormat: 'webp', + originalFilename: 'photo.png', + inputPath: `${id}.png`, + inputMimeType: 'image/png', + expiresAt: new Date(Date.now() + 3600 * 1000), + }); + + const deletedCount = await runCleanup(pool, config); + + expect(deletedCount).toBe(0); + expect(await getJobById(pool, id)).not.toBeNull(); + }); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/cleanup.test.js` +Expected: FAIL — `src/cleanup.js` does not exist yet. + +- [ ] **Step 3: Implement `src/cleanup.js`** + +```js +import { loadConfig } from './config.js'; +import { getPool, closePool } from './db.js'; +import { uploadPath, outputPath, deleteIfExists } from './storage.js'; +import { findExpiredJobs, deleteJob } from './jobs/jobRepository.js'; + +export async function runCleanup(pool, config) { + const expiredJobs = await findExpiredJobs(pool); + + for (const job of expiredJobs) { + await deleteIfExists(uploadPath(config, job.id, job.sourceFormat)); + await deleteIfExists(outputPath(config, job.id, job.targetFormat)); + await deleteJob(pool, job.id); + } + + return expiredJobs.length; +} + +async function main() { + const config = loadConfig(); + const pool = getPool(config); + const deletedCount = await runCleanup(pool, config); + console.log(`Cleanup: removed ${deletedCount} expired job(s).`); + await closePool(); +} + +if (import.meta.url === `file://${process.argv[1]}`) { + main(); +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/cleanup.test.js` +Expected: PASS (3 tests) + +- [ ] **Step 5: Add the `cleanup` script check to `package.json`** + +Confirm `package.json` already has (added in Task 1): + +```json +"cleanup": "node src/cleanup.js" +``` + +- [ ] **Step 6: Create the pm2 ecosystem file for the worker** + +Create `ecosystem.config.cjs`: + +```js +module.exports = { + apps: [ + { + name: 'convert-worker', + script: 'src/worker.js', + interpreter: 'node', + env: { + NODE_ENV: 'production', + }, + }, + ], +}; +``` + +- [ ] **Step 7: Document deployment steps in `README.md`** + +Create `README.md`: + +```markdown +# File Converter + +## Local development + +1. Copy `.env.example` to `.env` and fill in your local MariaDB credentials. +2. Apply the schema: `mysql -h -u -p < db/schema.sql` +3. Install dependencies: `npm install` +4. Run the API: `npm start` +5. Run the worker (separate terminal): `npm run worker` +6. Run tests: `npm test` (requires the same MariaDB reachable via your `.env` vars, exported into the shell) + +## Deployment on o2switch + +1. Upload the project (excluding `node_modules/`) via SSH/Git. +2. Create/adjust `.env` on the server with production values (`STORAGE_DIR` pointing to a writable path under your account, MariaDB credentials from cPanel). +3. Apply `db/schema.sql` to the MariaDB database created in cPanel. +4. `npm install` (never with `--ignore-scripts` — Puppeteer needs its postinstall step to download Chromium). +5. Configure the app in cPanel "Setup Node.js App", pointing its entry point at `src/server.js`. Passenger manages this process (start/stop/restart). +6. Start the worker independently of Passenger, over SSH: `pm2 start ecosystem.config.cjs`, then `pm2 save`. Try `pm2 startup` to survive a server reboot; if that's not permitted without root on this account, fall back to a cPanel cron job every 5 minutes that runs `pm2 resurrect` (or checks `pm2 list` and restarts the app if absent) — validate which option this hosting plan actually allows once connected over SSH. +7. Add a cPanel cron job to run the cleanup script periodically, e.g. every 15 minutes: + `*/15 * * * * cd /home// && /usr/bin/node src/cleanup.js >> logs/cleanup.log 2>&1` + (adjust the path and node binary location to match your actual account — check with `which node` over SSH). +8. On every subsequent deployment: pull changes, `npm install`, `npm run build` (frontend, once Task 17 exists), then restart the Passenger app from cPanel and `pm2 restart convert-worker`. +``` + +- [ ] **Step 8: Commit** + +```bash +git add src/cleanup.js test/cleanup.test.js ecosystem.config.cjs README.md +git commit -m "feat: add cleanup script and o2switch deployment docs" +``` + +--- + +## Task 17: Frontend (React SPA) + +**Files:** +- Create: `frontend/` (scaffolded by Vite, then edited) +- Modify: `frontend/vite.config.js` +- Create: `frontend/src/api.js` +- Create: `frontend/src/FileCard.jsx` +- Modify: `frontend/src/App.jsx` +- Modify: `frontend/src/App.css` +- Modify: `src/app.js` (serve the built frontend) +- Modify: `package.json` (root `build` script) + +This task is implementation-focused rather than TDD — the design spec's test strategy scopes automated tests to converters/registry/API (Tasks 1-16), not UI components. Verification here is manual: run the dev server and the built app in a browser. + +- [ ] **Step 1: Scaffold the Vite React app** + +From the project root: + +```bash +npm create vite@latest frontend -- --template react +cd frontend && npm install && cd .. +``` + +- [ ] **Step 2: Configure the dev proxy so the Vite dev server forwards `/api` to Express** + +Replace the contents of `frontend/vite.config.js`: + +```js +import { defineConfig } from 'vite'; +import react from '@vitejs/plugin-react'; + +export default defineConfig({ + plugins: [react()], + server: { + proxy: { + '/api': 'http://localhost:3000', + }, + }, +}); +``` + +- [ ] **Step 3: Create `frontend/src/api.js`** + +```js +export async function fetchFormats(source) { + const response = await fetch(`/api/formats?source=${encodeURIComponent(source)}`); + const data = await response.json(); + return data.targets; +} + +export async function uploadFiles(items) { + const formData = new FormData(); + const targetFormats = []; + for (const item of items) { + formData.append('files', item.file); + targetFormats.push(item.targetFormat); + } + formData.append('targetFormats', JSON.stringify(targetFormats)); + + const response = await fetch('/api/jobs', { method: 'POST', body: formData }); + const data = await response.json(); + return data.jobs; +} + +export async function fetchJobStatus(id) { + const response = await fetch(`/api/jobs/${id}`); + return response.json(); +} + +export function downloadUrl(id) { + return `/api/jobs/${id}/download`; +} +``` + +- [ ] **Step 4: Create `frontend/src/FileCard.jsx`** + +```jsx +import { useEffect, useState } from 'react'; +import { fetchJobStatus, downloadUrl } from './api.js'; + +export function FileCard({ fileName, jobId, initialError }) { + const [status, setStatus] = useState(initialError ? 'failed' : 'pending'); + const [errorMessage, setErrorMessage] = useState(initialError ?? null); + + useEffect(() => { + if (!jobId || initialError) return undefined; + + let cancelled = false; + const interval = setInterval(async () => { + const job = await fetchJobStatus(jobId); + if (cancelled) return; + setStatus(job.status); + if (job.status === 'failed') setErrorMessage(job.errorMessage); + if (job.status === 'done' || job.status === 'failed') clearInterval(interval); + }, 1500); + + return () => { + cancelled = true; + clearInterval(interval); + }; + }, [jobId, initialError]); + + return ( +
  • + {fileName} + {(status === 'pending' || status === 'processing') && Conversion en cours...} + {status === 'done' && Télécharger} + {status === 'failed' && {errorMessage}} +
    • + ); +} +``` + +- [ ] **Step 5: Replace `frontend/src/App.jsx`** + +```jsx +import { useState } from 'react'; +import { fetchFormats, uploadFiles } from './api.js'; +import { FileCard } from './FileCard.jsx'; +import './App.css'; + +function extensionOf(fileName) { + return fileName.split('.').pop().toLowerCase(); +} + +export default function App() { + const [pendingFiles, setPendingFiles] = useState([]); + const [submittedJobs, setSubmittedJobs] = useState([]); + + async function handleFilesSelected(fileList) { + const files = Array.from(fileList); + const withTargets = await Promise.all( + files.map(async (file) => { + const targets = await fetchFormats(extensionOf(file.name)); + return { file, targets, targetFormat: targets[0] ?? null }; + }) + ); + setPendingFiles(withTargets); + } + + function updateTargetFormat(index, targetFormat) { + setPendingFiles((current) => current.map((item, i) => (i === index ? { ...item, targetFormat } : item))); + } + + async function handleConvert() { + const validItems = pendingFiles.filter((item) => item.targetFormat); + const jobs = await uploadFiles(validItems); + setSubmittedJobs((current) => [...current, ...jobs]); + setPendingFiles([]); + } + + return ( +
    +

    Convertisseur de fichiers

    + + handleFilesSelected(event.target.files)} /> + + {pendingFiles.length > 0 && ( +
    +
      + {pendingFiles.map((item, index) => ( +
    • + {item.file.name} + {item.targets.length > 0 ? ( + + ) : ( + Format non supporté + )} +
      • + ))} +
    + +
    + )} + +
      + {submittedJobs.map((job, index) => + job.id ? ( + + ) : ( + + ) + )} +
    +
    + ); +} +``` + +- [ ] **Step 6: Replace `frontend/src/App.css` with minimal layout styles** + +```css +main { + max-width: 640px; + margin: 2rem auto; + font-family: system-ui, sans-serif; +} + +ul { + list-style: none; + padding: 0; +} + +li { + display: flex; + align-items: center; + gap: 0.75rem; + padding: 0.5rem 0; +} + +.error { + color: #b00020; +} +``` + +- [ ] **Step 7: Serve the built frontend from Express** + +Add to `src/app.js`, near the top (with the other imports): + +```js +import path from 'node:path'; +``` + +(skip if already imported by Task 13). Add this after the API routes but **before** the error-handling middleware Task 13 added at the end of `createApp` (an error handler registered before a route never sees that route's errors, so the static/catch-all route must come first): + +```js + const frontendDist = path.join(import.meta.dirname, '..', 'frontend', 'dist'); + app.use(express.static(frontendDist)); + app.get(/^\/(?!api\/).*/, (req, res) => { + res.sendFile(path.join(frontendDist, 'index.html')); + }); +``` + +- [ ] **Step 8: Add a root-level `build` script** + +Add to the root `package.json` `scripts`: + +```json +"build": "npm install --prefix frontend && npm run build --prefix frontend" +``` + +- [ ] **Step 9: Manual verification — dev mode** + +Terminal 1: `npm start` (Express API on port 3000) +Terminal 2: `cd frontend && npm run dev` (Vite dev server, typically port 5173) + +Open the Vite dev server URL in a browser. Drop a PNG and a DOCX file simultaneously, pick target formats, click "Convertir", confirm both file cards poll independently and eventually show a working "Télécharger" link (requires the worker from Task 15 running too: `npm run worker` in a third terminal). + +- [ ] **Step 10: Manual verification — production build served by Express** + +```bash +npm run build +npm start +``` + +Open `http://localhost:3000` directly (no Vite dev server this time) and repeat the same upload/convert/download flow. + +- [ ] **Step 11: Commit** + +```bash +git add frontend package.json src/app.js +git commit -m "feat: add React frontend and serve it from Express" +``` + +--- + +## Task 18: End-to-end pipeline test (one case per family) + +**Files:** +- Create: `test/api/endToEnd.test.js` + +**Interfaces:** +- Consumes: `createApp` (Task 12), `processPendingJobs` (Task 15) — called directly instead of `startWorker`'s timer, so the test drives the worker deterministically instead of racing a `setInterval`. + +This closes the loop the spec's test strategy asked for: upload → status → download, for one image case and one document case, through the real HTTP API and the real worker function (no mocks). + +- [ ] **Step 1: Write the test** + +Create `test/api/endToEnd.test.js`: + +```js +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import request from 'supertest'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import os from 'node:os'; +import { Document, Paragraph, TextRun, Packer } from 'docx'; +import { createApp } from '../../src/app.js'; +import { getPool, closePool } from '../../src/db.js'; +import { loadConfig } from '../../src/config.js'; +import { ensureStorageDirs } from '../../src/storage.js'; +import { processPendingJobs } from '../../src/worker.js'; + +let app; +let pool; +let config; + +beforeAll(async () => { + config = { ...loadConfig(), storageDir: await fs.mkdtemp(path.join(os.tmpdir(), 'converter-e2e-')) }; + await ensureStorageDirs(config); + pool = getPool(config); + app = createApp(config, pool); +}); + +afterAll(async () => { + await closePool(); + await fs.rm(config.storageDir, { recursive: true, force: true }); +}); + +beforeEach(async () => { + await pool.query('DELETE FROM conversion_jobs'); +}); + +async function waitForDone(id, maxAttempts = 20) { + for (let attempt = 0; attempt < maxAttempts; attempt += 1) { + await processPendingJobs(pool, config); + const response = await request(app).get(`/api/jobs/${id}`); + if (response.body.status === 'done' || response.body.status === 'failed') { + return response.body; + } + } + throw new Error(`Job ${id} did not finish after ${maxAttempts} worker passes`); +} + +describe('end-to-end: image family', () => { + it('uploads a PNG, converts it to WebP, and downloads the result', async () => { + const fixturePath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.png'); + + const uploadResponse = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['webp'])) + .attach('files', fixturePath, 'photo.png'); + + const { id } = uploadResponse.body.jobs[0]; + const finalStatus = await waitForDone(id); + expect(finalStatus.status).toBe('done'); + + const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); + expect(downloadResponse.status).toBe(200); + expect(downloadResponse.headers['content-type']).toBe('image/webp'); + expect(downloadResponse.headers['content-disposition']).toContain('photo.png'); + }, 20000); +}); + +describe('end-to-end: document family', () => { + it('uploads a DOCX, converts it to PDF, and downloads the result', async () => { + const doc = new Document({ + sections: [{ children: [new Paragraph({ children: [new TextRun('End to end fixture text')] })] }], + }); + const buffer = await Packer.toBuffer(doc); + const docxPath = path.join(config.storageDir, 'e2e-fixture.docx'); + await fs.writeFile(docxPath, buffer); + + const uploadResponse = await request(app) + .post('/api/jobs') + .field('targetFormats', JSON.stringify(['pdf'])) + .attach('files', docxPath, 'report.docx'); + + const { id } = uploadResponse.body.jobs[0]; + const finalStatus = await waitForDone(id); + expect(finalStatus.status).toBe('done'); + + const downloadResponse = await request(app).get(`/api/jobs/${id}/download`); + expect(downloadResponse.status).toBe(200); + expect(downloadResponse.headers['content-type']).toBe('application/pdf'); + expect(downloadResponse.headers['content-disposition']).toContain('report.docx'); + + await fs.unlink(docxPath); + }, 20000); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails initially if the worker isn't wired correctly** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run test/api/endToEnd.test.js` +Expected at this point: PASS, since Tasks 1-17 already implemented everything this test exercises — this step is a regression check, not new implementation. If it fails, the bug is in the interaction between tasks (most likely a mismatch between how `app.js` names uploaded files and how `worker.js` reconstructs paths), not in a single file — investigate before continuing. + +- [ ] **Step 3: Run the entire test suite** + +Run: `STORAGE_DIR=./storage DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter npx vitest run` +Expected: PASS, all tests across all previous tasks plus this one. + +- [ ] **Step 4: Commit** + +```bash +git add test/api/endToEnd.test.js +git commit -m "test: add end-to-end upload-convert-download coverage for image and document families" +``` + +--- + +## Out of scope for this plan + +Audio, Video, Presentations, Fonts, Ebook, Archives — to be added later as new converter family modules registered the same way (`register*Converters()` functions consumed by `app.js` and `worker.js`), following the pattern established in Tasks 7-10. BMP output support (sharp does not support it) is also deferred pending confirmation it's actually needed.