1328 lines
48 KiB
Markdown
1328 lines
48 KiB
Markdown
# Archive Conversion Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Add a new `archive` converter family supporting `zip`, `tar`, `tar.gz`, `tar.bz2`, `7z`, `tar.7z` (all pairs, including same-format recompression) plus `rar` as an extraction-only source, all with a 0–9 compression-level control reusing the existing `quality` job field.
|
||
|
||
**Architecture:** One generic extract-then-rebuild pipeline in `src/converters/archive.js`: each conversion extracts the source archive's contents into a temp directory, runs a shared path-escape guard, then rebuilds that directory into the target format. Per-format extractor/creator functions are looked up from two maps and registered as the full cross product via the existing `registry.js`.
|
||
|
||
**Tech Stack:** `adm-zip` (zip extract), `archiver` (zip create, for compression-level control `adm-zip` lacks), `tar` (tar/tar.gz both directions, pure JS, wraps Node's built-in zlib), `7zip-min` (7z/tar.7z/tar.bz2 — wraps a precompiled `7za` binary via `child_process.spawn`, same category of dependency as `sharp`/`puppeteer`), `node-unrar-js` (rar extraction, WASM, license-compliant).
|
||
|
||
## Global Constraints
|
||
|
||
- Every new dependency must ship a prebuilt binary or be pure JS — no dependency in this feature requires a compiler toolchain (o2switch has none).
|
||
- `rar` is never a target format — only ever a source. No code should register or expose it as a target.
|
||
- Same-format pairs (`zip -> zip`, `7z -> 7z`, etc.) ARE allowed for the archive family — unlike every other family, which rejects `sourceFormat === targetFormat`.
|
||
- Compression level uses a uniform 0–9 scale across all controllable archive targets (`zip`, `tar.gz`, `tar.bz2`, `7z`, `tar.7z`), stored in the existing `ConversionJob.quality` column. `tar` has no compression control (same rule as `gif`/`ico` today).
|
||
- No Prisma schema changes anywhere in this feature.
|
||
- Spec reference: `docs/superpowers/specs/2026-08-01-archive-conversion-design.md`.
|
||
|
||
---
|
||
|
||
### Task 1: Double-extension helper (`src/archiveExtensions.js`)
|
||
|
||
**Files:**
|
||
- Create: `src/archiveExtensions.js`
|
||
- Test: `test/archiveExtensions.test.js`
|
||
|
||
**Interfaces:**
|
||
- Produces: `extractExtension(filename: string): string` — returns `'tar.gz'`, `'tar.bz2'`, or `'tar.7z'` when the filename ends with one of those (case-insensitive), otherwise falls back to the last dot segment lowercased (same behavior `path.extname(...).slice(1).toLowerCase()` had before). `stripExtension(filename: string): string` — returns the filename with whatever `extractExtension` would return removed from the end (plus the separating dot).
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
```js
|
||
// test/archiveExtensions.test.js
|
||
import { describe, it, expect } from 'vitest';
|
||
import { extractExtension, stripExtension } from '../src/archiveExtensions.js';
|
||
|
||
describe('extractExtension', () => {
|
||
it('returns the simple extension for a normal filename', () => {
|
||
expect(extractExtension('photo.PNG')).toBe('png');
|
||
});
|
||
|
||
it('recognizes tar.gz as a single compound extension', () => {
|
||
expect(extractExtension('backup.TAR.GZ')).toBe('tar.gz');
|
||
});
|
||
|
||
it('recognizes tar.bz2 as a single compound extension', () => {
|
||
expect(extractExtension('backup.tar.bz2')).toBe('tar.bz2');
|
||
});
|
||
|
||
it('recognizes tar.7z as a single compound extension', () => {
|
||
expect(extractExtension('backup.tar.7z')).toBe('tar.7z');
|
||
});
|
||
|
||
it('does not treat an unrelated double extension as compound', () => {
|
||
expect(extractExtension('archive.zip.bak')).toBe('bak');
|
||
});
|
||
});
|
||
|
||
describe('stripExtension', () => {
|
||
it('strips a simple extension', () => {
|
||
expect(stripExtension('photo.png')).toBe('photo');
|
||
});
|
||
|
||
it('strips a compound tar.gz extension, keeping the rest of the name intact', () => {
|
||
expect(stripExtension('my.backup.tar.gz')).toBe('my.backup');
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/archiveExtensions.test.js`
|
||
Expected: FAIL with a module-not-found error for `../src/archiveExtensions.js`.
|
||
|
||
- [ ] **Step 3: Write minimal implementation**
|
||
|
||
```js
|
||
// src/archiveExtensions.js
|
||
import path from 'node:path';
|
||
|
||
const DOUBLE_EXTENSIONS = ['tar.gz', 'tar.bz2', 'tar.7z'];
|
||
|
||
export function extractExtension(filename) {
|
||
const lower = filename.toLowerCase();
|
||
const match = DOUBLE_EXTENSIONS.find((ext) => lower.endsWith(`.${ext}`));
|
||
return match ?? path.extname(filename).slice(1).toLowerCase();
|
||
}
|
||
|
||
export function stripExtension(filename) {
|
||
const ext = extractExtension(filename);
|
||
return filename.slice(0, filename.length - ext.length - 1);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/archiveExtensions.test.js`
|
||
Expected: PASS (7 tests)
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add src/archiveExtensions.js test/archiveExtensions.test.js
|
||
git commit -m "feat(archive): add double-extension-aware filename parsing helper"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: Archive converter skeleton + zip (`src/converters/archive.js`)
|
||
|
||
**Files:**
|
||
- Create: `src/converters/archive.js`
|
||
- Modify: `package.json` (add `adm-zip`, `archiver`)
|
||
- Test: `test/converters/archive.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `register` from `./registry.js` (existing, `register({ family, sourceFormat, targetFormat, convert })`).
|
||
- Produces: `registerArchiveConverters(): void` — registers every pair currently in `EXTRACTORS` × `CREATORS` (only `zip` in both after this task; more formats added in later tasks). `assertNoPathEscape(extractDir: string): Promise<void>` — exported for direct testing, throws if any entry under `extractDir` resolves outside it.
|
||
|
||
- [ ] **Step 1: Add dependencies**
|
||
|
||
```bash
|
||
npm install adm-zip archiver
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
```js
|
||
// test/converters/archive.test.js
|
||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||
import fs from 'node:fs/promises';
|
||
import path from 'node:path';
|
||
import os from 'node:os';
|
||
import AdmZip from 'adm-zip';
|
||
import { registerArchiveConverters, assertNoPathEscape } from '../../src/converters/archive.js';
|
||
import { resolve, listTargetFormats } from '../../src/converters/registry.js';
|
||
|
||
let tmpDir;
|
||
let srcFixtureDir;
|
||
|
||
beforeAll(async () => {
|
||
registerArchiveConverters();
|
||
tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'converter-archive-'));
|
||
|
||
srcFixtureDir = path.join(tmpDir, 'src-fixture');
|
||
await fs.mkdir(path.join(srcFixtureDir, 'nested'), { recursive: true });
|
||
await fs.writeFile(path.join(srcFixtureDir, 'hello.txt'), 'hello world');
|
||
await fs.writeFile(path.join(srcFixtureDir, 'nested', 'inner.txt'), 'nested content');
|
||
});
|
||
|
||
afterAll(async () => {
|
||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||
});
|
||
|
||
function buildZipFixture(destPath) {
|
||
const zip = new AdmZip();
|
||
zip.addFile('hello.txt', Buffer.from('hello world'));
|
||
zip.addFile('nested/inner.txt', Buffer.from('nested content'));
|
||
zip.writeZip(destPath);
|
||
}
|
||
|
||
describe('archive converters — zip', () => {
|
||
it('registers zip -> zip (same-format pairs are allowed for this family)', () => {
|
||
expect(listTargetFormats('zip')).toContain('zip');
|
||
});
|
||
|
||
it('round-trips a zip fixture through zip -> zip', async () => {
|
||
const inputPath = path.join(tmpDir, 'fixture.zip');
|
||
buildZipFixture(inputPath);
|
||
const outputPath = path.join(tmpDir, 'output.zip');
|
||
const entry = resolve('zip', 'zip');
|
||
|
||
await entry.convert(inputPath, outputPath, { quality: 9 });
|
||
|
||
const outZip = new AdmZip(outputPath);
|
||
const names = outZip.getEntries().map((e) => e.entryName).sort();
|
||
expect(names).toEqual(['hello.txt', 'nested/inner.txt']);
|
||
expect(outZip.readAsText('hello.txt')).toBe('hello world');
|
||
expect(outZip.readAsText('nested/inner.txt')).toBe('nested content');
|
||
});
|
||
|
||
it('produces a smaller-or-equal zip at a higher compression level', async () => {
|
||
const inputPath = path.join(tmpDir, 'fixture-for-level.zip');
|
||
buildZipFixture(inputPath);
|
||
const lowPath = path.join(tmpDir, 'low.zip');
|
||
const highPath = path.join(tmpDir, 'high.zip');
|
||
const entry = resolve('zip', 'zip');
|
||
|
||
await entry.convert(inputPath, lowPath, { quality: 0 });
|
||
await entry.convert(inputPath, highPath, { quality: 9 });
|
||
|
||
const [lowStat, highStat] = await Promise.all([fs.stat(lowPath), fs.stat(highPath)]);
|
||
expect(highStat.size).toBeLessThanOrEqual(lowStat.size);
|
||
});
|
||
});
|
||
|
||
describe('assertNoPathEscape', () => {
|
||
it('does not throw for a directory with only well-behaved entries', async () => {
|
||
await expect(assertNoPathEscape(srcFixtureDir)).resolves.toBeUndefined();
|
||
});
|
||
});
|
||
|
||
describe('archive converters — zip-slip protection', () => {
|
||
it('rejects a zip whose entry path escapes the extraction directory', async () => {
|
||
const maliciousPath = path.join(tmpDir, 'evil.zip');
|
||
const zip = new AdmZip();
|
||
zip.addFile('../evil.txt', Buffer.from('pwned'));
|
||
zip.writeZip(maliciousPath);
|
||
|
||
const outputPath = path.join(tmpDir, 'should-not-exist.zip');
|
||
const entry = resolve('zip', 'zip');
|
||
|
||
await expect(entry.convert(maliciousPath, outputPath)).rejects.toThrow(/escapes/);
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: FAIL with a module-not-found error for `../../src/converters/archive.js`.
|
||
|
||
- [ ] **Step 4: Write minimal implementation**
|
||
|
||
```js
|
||
// src/converters/archive.js
|
||
import fs from 'node:fs/promises';
|
||
import { createWriteStream } from 'node:fs';
|
||
import path from 'node:path';
|
||
import os from 'node:os';
|
||
import AdmZip from 'adm-zip';
|
||
import archiver from 'archiver';
|
||
import { register } from './registry.js';
|
||
|
||
const MAX_EXTRACTED_BYTES = 2 * 1024 * 1024 * 1024;
|
||
|
||
export async function assertNoPathEscape(extractDir) {
|
||
const resolvedRoot = await fs.realpath(extractDir);
|
||
const entries = await fs.readdir(extractDir, { recursive: true, withFileTypes: true });
|
||
for (const entry of entries) {
|
||
const fullPath = path.join(entry.parentPath, entry.name);
|
||
const real = entry.isSymbolicLink() ? await fs.realpath(fullPath) : fullPath;
|
||
if (real !== resolvedRoot && !real.startsWith(resolvedRoot + path.sep)) {
|
||
throw new Error('Archive entry escapes extraction directory');
|
||
}
|
||
}
|
||
}
|
||
|
||
async function extractZip(inputPath, destDir) {
|
||
const zip = new AdmZip(inputPath);
|
||
let totalBytes = 0;
|
||
for (const entry of zip.getEntries()) {
|
||
const targetPath = path.join(destDir, entry.entryName);
|
||
if (targetPath !== destDir && !targetPath.startsWith(destDir + path.sep)) {
|
||
throw new Error('Archive entry escapes extraction directory');
|
||
}
|
||
if (entry.isDirectory) {
|
||
await fs.mkdir(targetPath, { recursive: true });
|
||
continue;
|
||
}
|
||
totalBytes += entry.header.size;
|
||
if (totalBytes > MAX_EXTRACTED_BYTES) {
|
||
throw new Error('Archive exceeds maximum extracted size');
|
||
}
|
||
await fs.mkdir(path.dirname(targetPath), { recursive: true });
|
||
await fs.writeFile(targetPath, entry.getData());
|
||
}
|
||
}
|
||
|
||
function createZip(srcDir, outputPath, quality) {
|
||
return new Promise((resolvePromise, reject) => {
|
||
const output = createWriteStream(outputPath);
|
||
const archive = archiver('zip', { zlib: { level: quality ?? 6 } });
|
||
output.on('close', resolvePromise);
|
||
archive.on('error', reject);
|
||
archive.pipe(output);
|
||
archive.directory(srcDir, false);
|
||
archive.finalize();
|
||
});
|
||
}
|
||
|
||
const EXTRACTORS = { zip: extractZip };
|
||
const CREATORS = { zip: createZip };
|
||
|
||
async function convert(inputPath, outputPath, options, sourceFormat, targetFormat) {
|
||
const { quality } = options ?? {};
|
||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'archive-convert-'));
|
||
try {
|
||
const extractDir = path.join(tmpDir, 'extracted');
|
||
await fs.mkdir(extractDir, { recursive: true });
|
||
await EXTRACTORS[sourceFormat](inputPath, extractDir);
|
||
await assertNoPathEscape(extractDir);
|
||
await CREATORS[targetFormat](extractDir, outputPath, quality);
|
||
} finally {
|
||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
|
||
export function registerArchiveConverters() {
|
||
for (const sourceFormat of Object.keys(EXTRACTORS)) {
|
||
for (const targetFormat of Object.keys(CREATORS)) {
|
||
register({
|
||
family: 'archive',
|
||
sourceFormat,
|
||
targetFormat,
|
||
convert: (inputPath, outputPath, options) =>
|
||
convert(inputPath, outputPath, options, sourceFormat, targetFormat),
|
||
});
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: PASS (6 tests). If the zip-slip fixture step itself throws while being *built* (some `adm-zip` versions sanitize `entryName` on `addFile`), replace the malicious-fixture construction with a raw zip central-directory byte patch instead — do not skip or weaken the assertion; the goal is a real entry whose resolved path escapes `destDir`.
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add src/converters/archive.js test/converters/archive.test.js package.json package-lock.json
|
||
git commit -m "feat(archive): add zip converter with compression level and zip-slip protection"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: tar and tar.gz
|
||
|
||
**Files:**
|
||
- Modify: `src/converters/archive.js`
|
||
- Modify: `package.json` (add `tar`)
|
||
- Modify: `test/converters/archive.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `EXTRACTORS`, `CREATORS`, `srcFixtureDir`, `tmpDir` (all already present in the file from Task 2).
|
||
- Produces: `EXTRACTORS.tar`, `EXTRACTORS['tar.gz']`, `CREATORS.tar`, `CREATORS['tar.gz']`.
|
||
|
||
- [ ] **Step 1: Add dependency**
|
||
|
||
```bash
|
||
npm install tar
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Append to `test/converters/archive.test.js`:
|
||
|
||
```js
|
||
import * as tar from 'tar';
|
||
|
||
describe('archive converters — tar / tar.gz', () => {
|
||
it('registers tar and tar.gz as targets for each other', () => {
|
||
expect(listTargetFormats('tar')).toContain('tar.gz');
|
||
expect(listTargetFormats('tar.gz')).toContain('tar');
|
||
});
|
||
|
||
it('converts zip -> tar, preserving nested paths and content', async () => {
|
||
const inputPath = path.join(tmpDir, 'for-tar.zip');
|
||
buildZipFixture(inputPath);
|
||
const outputPath = path.join(tmpDir, 'output.tar');
|
||
const entry = resolve('zip', 'tar');
|
||
|
||
await entry.convert(inputPath, outputPath);
|
||
|
||
const listDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tar-check-'));
|
||
await tar.extract({ file: outputPath, cwd: listDir });
|
||
expect(await fs.readFile(path.join(listDir, 'hello.txt'), 'utf8')).toBe('hello world');
|
||
expect(await fs.readFile(path.join(listDir, 'nested', 'inner.txt'), 'utf8')).toBe('nested content');
|
||
await fs.rm(listDir, { recursive: true, force: true });
|
||
});
|
||
|
||
it('converts tar -> tar.gz and back to tar, preserving content', async () => {
|
||
const tarPath = path.join(tmpDir, 'roundtrip.tar');
|
||
await tar.create({ file: tarPath, cwd: srcFixtureDir }, ['hello.txt', 'nested']);
|
||
|
||
const gzPath = path.join(tmpDir, 'roundtrip.tar.gz');
|
||
await resolve('tar', 'tar.gz').convert(tarPath, gzPath, { quality: 9 });
|
||
|
||
const backToTarPath = path.join(tmpDir, 'roundtrip-back.tar');
|
||
await resolve('tar.gz', 'tar').convert(gzPath, backToTarPath);
|
||
|
||
const listDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tar-gz-check-'));
|
||
await tar.extract({ file: backToTarPath, cwd: listDir });
|
||
expect(await fs.readFile(path.join(listDir, 'hello.txt'), 'utf8')).toBe('hello world');
|
||
await fs.rm(listDir, { recursive: true, force: true });
|
||
});
|
||
|
||
it('rejects a quality value for a tar target (no compression to control)', () => {
|
||
expect(CREATORS.tar).toBeDefined();
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: FAIL — `tar`/`tar.gz` not in `EXTRACTORS`/`CREATORS`, `resolve('zip', 'tar')` returns `null`.
|
||
|
||
- [ ] **Step 4: Write minimal implementation**
|
||
|
||
In `src/converters/archive.js`, add the import and functions, then extend the maps:
|
||
|
||
```js
|
||
import * as tar from 'tar';
|
||
|
||
async function extractTarLike(inputPath, destDir) {
|
||
// tar's extract auto-detects gzip compression from the file's magic bytes,
|
||
// so the same function handles both plain .tar and .tar.gz input.
|
||
await tar.extract({ file: inputPath, cwd: destDir });
|
||
}
|
||
|
||
async function createTar(srcDir, outputPath) {
|
||
const entries = await fs.readdir(srcDir);
|
||
await tar.create({ file: outputPath, cwd: srcDir }, entries);
|
||
}
|
||
|
||
async function createTarGz(srcDir, outputPath, quality) {
|
||
const entries = await fs.readdir(srcDir);
|
||
await tar.create({ file: outputPath, cwd: srcDir, gzip: { level: quality ?? 6 } }, entries);
|
||
}
|
||
```
|
||
|
||
Update the maps:
|
||
|
||
```js
|
||
const EXTRACTORS = { zip: extractZip, tar: extractTarLike, 'tar.gz': extractTarLike };
|
||
const CREATORS = { zip: createZip, tar: createTar, 'tar.gz': createTarGz };
|
||
```
|
||
|
||
- [ ] **Step 5: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: PASS (all tests so far)
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add src/converters/archive.js test/converters/archive.test.js package.json package-lock.json
|
||
git commit -m "feat(archive): add tar and tar.gz converters"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: tar.bz2
|
||
|
||
**Files:**
|
||
- Modify: `src/converters/archive.js`
|
||
- Modify: `package.json` (add `7zip-min`)
|
||
- Modify: `test/converters/archive.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `tar` (from Task 3), `EXTRACTORS`/`CREATORS` maps.
|
||
- Produces: `EXTRACTORS['tar.bz2']`, `CREATORS['tar.bz2']`, internal `bzip2Level(quality)`.
|
||
|
||
- [ ] **Step 1: Add dependency**
|
||
|
||
```bash
|
||
npm install 7zip-min
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Append to `test/converters/archive.test.js`:
|
||
|
||
```js
|
||
describe('archive converters — tar.bz2', () => {
|
||
it('registers tar.bz2 as a target for tar and vice versa', () => {
|
||
expect(listTargetFormats('tar')).toContain('tar.bz2');
|
||
expect(listTargetFormats('tar.bz2')).toContain('tar');
|
||
});
|
||
|
||
it('converts zip -> tar.bz2 -> tar, preserving content', async () => {
|
||
const zipPath = path.join(tmpDir, 'for-bz2.zip');
|
||
buildZipFixture(zipPath);
|
||
|
||
const bz2Path = path.join(tmpDir, 'output.tar.bz2');
|
||
await resolve('zip', 'tar.bz2').convert(zipPath, bz2Path, { quality: 9 });
|
||
|
||
const backToTarPath = path.join(tmpDir, 'from-bz2.tar');
|
||
await resolve('tar.bz2', 'tar').convert(bz2Path, backToTarPath);
|
||
|
||
const listDir = await fs.mkdtemp(path.join(os.tmpdir(), 'bz2-check-'));
|
||
await tar.extract({ file: backToTarPath, cwd: listDir });
|
||
expect(await fs.readFile(path.join(listDir, 'hello.txt'), 'utf8')).toBe('hello world');
|
||
expect(await fs.readFile(path.join(listDir, 'nested', 'inner.txt'), 'utf8')).toBe('nested content');
|
||
await fs.rm(listDir, { recursive: true, force: true });
|
||
});
|
||
}, 20000);
|
||
```
|
||
|
||
(The 20s per-suite timeout override accounts for `7za` subprocess startup, which is slower than the in-process libraries used elsewhere in this file.)
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: FAIL — `resolve('zip', 'tar.bz2')` returns `null`.
|
||
|
||
- [ ] **Step 4: Write minimal implementation**
|
||
|
||
```js
|
||
import _7z from '7zip-min';
|
||
|
||
function bzip2Level(quality) {
|
||
return Math.max(1, quality ?? 9);
|
||
}
|
||
|
||
async function extractTarBz2(inputPath, destDir) {
|
||
const decompressDir = await fs.mkdtemp(path.join(os.tmpdir(), 'archive-bz2-'));
|
||
try {
|
||
await _7z.unpack(inputPath, decompressDir);
|
||
const [tarName] = await fs.readdir(decompressDir);
|
||
await tar.extract({ file: path.join(decompressDir, tarName), cwd: destDir });
|
||
} finally {
|
||
await fs.rm(decompressDir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
|
||
async function createTarBz2(srcDir, outputPath, quality) {
|
||
const buildDir = await fs.mkdtemp(path.join(os.tmpdir(), 'archive-bz2-'));
|
||
try {
|
||
const tarPath = path.join(buildDir, 'archive.tar');
|
||
const entries = await fs.readdir(srcDir);
|
||
await tar.create({ file: tarPath, cwd: srcDir }, entries);
|
||
await _7z.cmd(['a', '-tbzip2', `-mx=${bzip2Level(quality)}`, outputPath, tarPath]);
|
||
} finally {
|
||
await fs.rm(buildDir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
```
|
||
|
||
Update the maps:
|
||
|
||
```js
|
||
const EXTRACTORS = { zip: extractZip, tar: extractTarLike, 'tar.gz': extractTarLike, 'tar.bz2': extractTarBz2 };
|
||
const CREATORS = { zip: createZip, tar: createTar, 'tar.gz': createTarGz, 'tar.bz2': createTarBz2 };
|
||
```
|
||
|
||
- [ ] **Step 5: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: PASS. If `_7z.unpack` produces an intermediate filename this code doesn't anticipate (the `const [tarName] = await fs.readdir(decompressDir)` assumes exactly one file lands there), inspect `decompressDir`'s actual contents via a temporary `console.log` and adjust — `7za`'s exact output naming for a bare `.bz2` input is the one detail in this task not independently confirmed before writing this plan.
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add src/converters/archive.js test/converters/archive.test.js package.json package-lock.json
|
||
git commit -m "feat(archive): add tar.bz2 converter via 7za"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: 7z and tar.7z
|
||
|
||
**Files:**
|
||
- Modify: `src/converters/archive.js`
|
||
- Modify: `test/converters/archive.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `_7z` (from Task 4), `tar` (from Task 3).
|
||
- Produces: `EXTRACTORS['7z']`, `EXTRACTORS['tar.7z']`, `CREATORS['7z']`, `CREATORS['tar.7z']`, internal `sevenZipLevel(quality)`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Append to `test/converters/archive.test.js`:
|
||
|
||
```js
|
||
describe('archive converters — 7z and tar.7z', () => {
|
||
it('registers 7z and tar.7z as targets for each other and for zip', () => {
|
||
expect(listTargetFormats('zip')).toContain('7z');
|
||
expect(listTargetFormats('zip')).toContain('tar.7z');
|
||
expect(listTargetFormats('7z')).toContain('tar.7z');
|
||
});
|
||
|
||
it('converts zip -> 7z, preserving nested paths and content', async () => {
|
||
const zipPath = path.join(tmpDir, 'for-7z.zip');
|
||
buildZipFixture(zipPath);
|
||
const sevenZPath = path.join(tmpDir, 'output.7z');
|
||
|
||
await resolve('zip', '7z').convert(zipPath, sevenZPath, { quality: 9 });
|
||
|
||
const listDir = await fs.mkdtemp(path.join(os.tmpdir(), '7z-check-'));
|
||
await resolve('7z', 'zip').convert(sevenZPath, path.join(listDir, 'roundtrip.zip'));
|
||
const rtZip = new AdmZip(path.join(listDir, 'roundtrip.zip'));
|
||
expect(rtZip.readAsText('hello.txt')).toBe('hello world');
|
||
expect(rtZip.readAsText('nested/inner.txt')).toBe('nested content');
|
||
await fs.rm(listDir, { recursive: true, force: true });
|
||
});
|
||
|
||
it('converts zip -> tar.7z -> tar, preserving content', async () => {
|
||
const zipPath = path.join(tmpDir, 'for-tar7z.zip');
|
||
buildZipFixture(zipPath);
|
||
const tar7zPath = path.join(tmpDir, 'output.tar.7z');
|
||
|
||
await resolve('zip', 'tar.7z').convert(zipPath, tar7zPath, { quality: 5 });
|
||
|
||
const backToTarPath = path.join(tmpDir, 'from-tar7z.tar');
|
||
await resolve('tar.7z', 'tar').convert(tar7zPath, backToTarPath);
|
||
|
||
const listDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tar7z-check-'));
|
||
await tar.extract({ file: backToTarPath, cwd: listDir });
|
||
expect(await fs.readFile(path.join(listDir, 'hello.txt'), 'utf8')).toBe('hello world');
|
||
await fs.rm(listDir, { recursive: true, force: true });
|
||
});
|
||
|
||
it('produces a smaller-or-equal 7z at a higher compression level', async () => {
|
||
const zipPath = path.join(tmpDir, 'for-7z-level.zip');
|
||
buildZipFixture(zipPath);
|
||
const lowPath = path.join(tmpDir, 'low.7z');
|
||
const highPath = path.join(tmpDir, 'high.7z');
|
||
|
||
await resolve('zip', '7z').convert(zipPath, lowPath, { quality: 0 });
|
||
await resolve('zip', '7z').convert(zipPath, highPath, { quality: 9 });
|
||
|
||
const [lowStat, highStat] = await Promise.all([fs.stat(lowPath), fs.stat(highPath)]);
|
||
expect(highStat.size).toBeLessThanOrEqual(lowStat.size);
|
||
});
|
||
}, 20000);
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: FAIL — `resolve('zip', '7z')` returns `null`.
|
||
|
||
- [ ] **Step 3: Write minimal implementation**
|
||
|
||
```js
|
||
function sevenZipLevel(quality) {
|
||
return quality ?? 5;
|
||
}
|
||
|
||
async function extract7z(inputPath, destDir) {
|
||
await _7z.unpack(inputPath, destDir);
|
||
}
|
||
|
||
async function create7z(srcDir, outputPath, quality) {
|
||
await _7z.cmd(['a', '-r', `-mx=${sevenZipLevel(quality)}`, outputPath, path.join(srcDir, '*')]);
|
||
}
|
||
|
||
async function extractTar7z(inputPath, destDir) {
|
||
const decompressDir = await fs.mkdtemp(path.join(os.tmpdir(), 'archive-7z-'));
|
||
try {
|
||
await _7z.unpack(inputPath, decompressDir);
|
||
const [tarName] = await fs.readdir(decompressDir);
|
||
await tar.extract({ file: path.join(decompressDir, tarName), cwd: destDir });
|
||
} finally {
|
||
await fs.rm(decompressDir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
|
||
async function createTar7z(srcDir, outputPath, quality) {
|
||
const buildDir = await fs.mkdtemp(path.join(os.tmpdir(), 'archive-7z-'));
|
||
try {
|
||
const tarPath = path.join(buildDir, 'archive.tar');
|
||
const entries = await fs.readdir(srcDir);
|
||
await tar.create({ file: tarPath, cwd: srcDir }, entries);
|
||
await _7z.cmd(['a', `-mx=${sevenZipLevel(quality)}`, outputPath, tarPath]);
|
||
} finally {
|
||
await fs.rm(buildDir, { recursive: true, force: true });
|
||
}
|
||
}
|
||
```
|
||
|
||
Update the maps:
|
||
|
||
```js
|
||
const EXTRACTORS = {
|
||
zip: extractZip,
|
||
tar: extractTarLike,
|
||
'tar.gz': extractTarLike,
|
||
'tar.bz2': extractTarBz2,
|
||
'7z': extract7z,
|
||
'tar.7z': extractTar7z,
|
||
};
|
||
const CREATORS = {
|
||
zip: createZip,
|
||
tar: createTar,
|
||
'tar.gz': createTarGz,
|
||
'tar.bz2': createTarBz2,
|
||
'7z': create7z,
|
||
'tar.7z': createTar7z,
|
||
};
|
||
```
|
||
|
||
- [ ] **Step 4: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: PASS
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add src/converters/archive.js test/converters/archive.test.js
|
||
git commit -m "feat(archive): add 7z and tar.7z converters"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: rar extraction + full registration coverage
|
||
|
||
**Files:**
|
||
- Modify: `src/converters/archive.js`
|
||
- Modify: `package.json` (add `node-unrar-js`)
|
||
- Create: `test/fixtures/sample.rar` (downloaded, see Step 1)
|
||
- Create: `test/fixtures/RAR_ATTRIBUTION.md`
|
||
- Modify: `test/converters/archive.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing new from earlier tasks besides the maps.
|
||
- Produces: `EXTRACTORS.rar`. `CREATORS` gains no `rar` key — confirmed nowhere in this codebase ever registers `rar` as a target.
|
||
|
||
- [ ] **Step 1: Add dependency and download the RAR fixture**
|
||
|
||
```bash
|
||
npm install node-unrar-js
|
||
```
|
||
|
||
Download a small, permissively-licensed sample RAR (MIT-licensed `node-unrar.js` project's own test fixture, containing multiple files in a folder — used by that project for its own tests):
|
||
|
||
```bash
|
||
curl -sL -o test/fixtures/sample.rar https://raw.githubusercontent.com/YuJianrong/node-unrar.js/master/testFiles/FolderTest.rar
|
||
```
|
||
|
||
Create the attribution file:
|
||
|
||
```markdown
|
||
# RAR fixture attribution
|
||
|
||
`sample.rar` is a byte-identical copy of `testFiles/FolderTest.rar` from the
|
||
[YuJianrong/node-unrar.js](https://github.com/YuJianrong/node-unrar.js)
|
||
repository (MIT-licensed), used there for that project's own extraction
|
||
tests. Reused here because there is no way to create a `.rar` fixture from
|
||
this codebase — the `unrar` library's license bars any tool (including this
|
||
project's own converters) from writing the RAR format, so RAR support is
|
||
extraction-only everywhere, including in test fixtures.
|
||
```
|
||
|
||
Write that to `test/fixtures/RAR_ATTRIBUTION.md`.
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Append to `test/converters/archive.test.js`:
|
||
|
||
```js
|
||
describe('archive converters — rar (extraction only)', () => {
|
||
it('lists rar as a source with the other 6 archive formats as targets, and never lists rar as a target for anything', () => {
|
||
const targets = listTargetFormats('rar');
|
||
expect(targets.sort()).toEqual(['7z', 'tar', 'tar.7z', 'tar.bz2', 'tar.gz', 'zip'].sort());
|
||
expect(listTargetFormats('zip')).not.toContain('rar');
|
||
});
|
||
|
||
it('converts the sample.rar fixture to zip', async () => {
|
||
const rarPath = path.join(import.meta.dirname, '..', 'fixtures', 'sample.rar');
|
||
const outputPath = path.join(tmpDir, 'from-rar.zip');
|
||
const entry = resolve('rar', 'zip');
|
||
|
||
await entry.convert(rarPath, outputPath);
|
||
|
||
const outZip = new AdmZip(outputPath);
|
||
expect(outZip.getEntries().length).toBeGreaterThan(0);
|
||
});
|
||
});
|
||
|
||
describe('archive converters — full registration matrix', () => {
|
||
it('registers all 42 source/target pairs (7 sources x 6 targets)', () => {
|
||
const sources = ['zip', 'tar', 'tar.gz', 'tar.bz2', '7z', 'tar.7z', 'rar'];
|
||
const targets = ['zip', 'tar', 'tar.gz', 'tar.bz2', '7z', 'tar.7z'];
|
||
for (const source of sources) {
|
||
expect(listTargetFormats(source).sort()).toEqual(targets.sort());
|
||
}
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: FAIL — `resolve('rar', 'zip')` returns `null`; `listTargetFormats('rar')` is empty.
|
||
|
||
- [ ] **Step 4: Write minimal implementation**
|
||
|
||
```js
|
||
import { createExtractorFromFile } from 'node-unrar-js';
|
||
|
||
async function extractRar(inputPath, destDir) {
|
||
const extractor = await createExtractorFromFile({ filepath: inputPath, targetPath: destDir });
|
||
const { files } = extractor.extract();
|
||
for (const _file of files) {
|
||
// Iterating fully is required: node-unrar-js writes each entry to disk
|
||
// lazily as this generator is advanced.
|
||
}
|
||
}
|
||
```
|
||
|
||
Update `EXTRACTORS` only (rar is never in `CREATORS`):
|
||
|
||
```js
|
||
const EXTRACTORS = {
|
||
zip: extractZip,
|
||
tar: extractTarLike,
|
||
'tar.gz': extractTarLike,
|
||
'tar.bz2': extractTarBz2,
|
||
'7z': extract7z,
|
||
'tar.7z': extractTar7z,
|
||
rar: extractRar,
|
||
};
|
||
```
|
||
|
||
- [ ] **Step 5: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/converters/archive.test.js`
|
||
Expected: PASS (full suite)
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add src/converters/archive.js test/converters/archive.test.js test/fixtures/sample.rar test/fixtures/RAR_ATTRIBUTION.md package.json package-lock.json
|
||
git commit -m "feat(archive): add rar extraction support"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: MIME types and content-sniffing aliases
|
||
|
||
**Files:**
|
||
- Modify: `src/mime.js`
|
||
- Modify: `test/mime.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `OUTPUT_MIME_TYPES`, `normalizeFormat` (both already in `src/mime.js`).
|
||
- Produces: no new exports — extends existing ones.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Append to `test/mime.test.js`:
|
||
|
||
```js
|
||
describe('outputMimeType — archives', () => {
|
||
it('returns the correct MIME type for each archive target format', () => {
|
||
expect(outputMimeType('zip')).toBe('application/zip');
|
||
expect(outputMimeType('tar')).toBe('application/x-tar');
|
||
expect(outputMimeType('tar.gz')).toBe('application/gzip');
|
||
expect(outputMimeType('tar.bz2')).toBe('application/x-bzip2');
|
||
expect(outputMimeType('7z')).toBe('application/x-7z-compressed');
|
||
expect(outputMimeType('tar.7z')).toBe('application/x-7z-compressed');
|
||
});
|
||
});
|
||
|
||
describe('resolveInputFormat — compound archive extensions', () => {
|
||
it('accepts a real gzip file declared as tar.gz (file-type sniffs the outer gzip layer only)', async () => {
|
||
const zlib = await import('node:zlib');
|
||
const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.tar.gz');
|
||
await fs.writeFile(fixturePath, zlib.gzipSync(Buffer.from('irrelevant payload for this check')));
|
||
|
||
const result = await resolveInputFormat(fixturePath, 'tar.gz');
|
||
expect(result.valid).toBe(true);
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
|
||
it('accepts a real 7z file declared as tar.7z (file-type sniffs the outer 7z layer only)', async () => {
|
||
const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.tar.7z');
|
||
// Minimal valid 7z signature header (6-byte magic + 2-byte version), enough for file-type to sniff `ext: '7z'`.
|
||
await fs.writeFile(fixturePath, Buffer.from([0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c, 0x00, 0x04]));
|
||
|
||
const result = await resolveInputFormat(fixturePath, 'tar.7z');
|
||
expect(result.valid).toBe(true);
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/mime.test.js`
|
||
Expected: FAIL — `outputMimeType('zip')` throws (unknown format), and the `tar.gz`/`tar.7z` checks report `valid: false`.
|
||
|
||
- [ ] **Step 3: Write minimal implementation**
|
||
|
||
In `src/mime.js`, extend `OUTPUT_MIME_TYPES`:
|
||
|
||
```js
|
||
zip: 'application/zip',
|
||
tar: 'application/x-tar',
|
||
'tar.gz': 'application/gzip',
|
||
'tar.bz2': 'application/x-bzip2',
|
||
'7z': 'application/x-7z-compressed',
|
||
'tar.7z': 'application/x-7z-compressed',
|
||
```
|
||
|
||
(add these lines inside the existing `OUTPUT_MIME_TYPES` object, anywhere among the existing entries.)
|
||
|
||
Extend `normalizeFormat`:
|
||
|
||
```js
|
||
function normalizeFormat(format) {
|
||
if (format === 'jpg') return 'jpeg';
|
||
if (format === 'heif') return 'heic';
|
||
if (format === 'azw3') return 'mobi';
|
||
if (format === 'fb2') return 'xml';
|
||
if (format === 'tar.gz') return 'gz';
|
||
if (format === 'tar.bz2') return 'bz2';
|
||
if (format === 'tar.7z') return '7z';
|
||
return format;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/mime.test.js`
|
||
Expected: PASS
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add src/mime.js test/mime.test.js
|
||
git commit -m "feat(archive): add MIME types and compound-extension aliases for archive formats"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: Wire into `app.js` and `worker.js`
|
||
|
||
**Files:**
|
||
- Modify: `src/app.js`
|
||
- Modify: `src/worker.js`
|
||
- Modify: `test/api/jobs.test.js`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `registerArchiveConverters` (Task 6), `extractExtension`/`stripExtension` (Task 1).
|
||
- Produces: no new exports — wires existing pieces into the two entry points.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Append to `test/api/jobs.test.js`:
|
||
|
||
```js
|
||
import AdmZip from 'adm-zip';
|
||
|
||
function buildZipFixture(destPath) {
|
||
const zip = new AdmZip();
|
||
zip.addFile('hello.txt', Buffer.from('hello world'));
|
||
zip.writeZip(destPath);
|
||
}
|
||
|
||
describe('POST /api/jobs — archives', () => {
|
||
it('creates a pending job converting a zip upload to tar.gz', async () => {
|
||
const fixturePath = path.join(config.storageDir, 'archive.zip');
|
||
buildZipFixture(fixturePath);
|
||
|
||
const response = await request(app)
|
||
.post('/api/jobs')
|
||
.field('targetFormats', JSON.stringify(['tar.gz']))
|
||
.attach('files', fixturePath, 'archive.zip');
|
||
|
||
expect(response.status).toBe(201);
|
||
expect(response.body.jobs[0].status).toBe('pending');
|
||
|
||
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
|
||
expect(job.sourceFormat).toBe('zip');
|
||
expect(job.targetFormat).toBe('tar.gz');
|
||
expect(job.family).toBe('archive');
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
|
||
it('creates a pending job with a compression level for a same-format zip -> zip conversion', async () => {
|
||
const fixturePath = path.join(config.storageDir, 'recompress.zip');
|
||
buildZipFixture(fixturePath);
|
||
|
||
const response = await request(app)
|
||
.post('/api/jobs')
|
||
.field('targetFormats', JSON.stringify(['zip']))
|
||
.field('qualities', JSON.stringify([9]))
|
||
.attach('files', fixturePath, 'recompress.zip');
|
||
|
||
expect(response.status).toBe(201);
|
||
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
|
||
expect(job.targetFormat).toBe('zip');
|
||
expect(job.quality).toBe(9);
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
|
||
it('rejects an out-of-range compression level for a zip target', async () => {
|
||
const fixturePath = path.join(config.storageDir, 'bad-level.zip');
|
||
buildZipFixture(fixturePath);
|
||
|
||
const response = await request(app)
|
||
.post('/api/jobs')
|
||
.field('targetFormats', JSON.stringify(['zip']))
|
||
.field('qualities', JSON.stringify([15]))
|
||
.attach('files', fixturePath, 'bad-level.zip');
|
||
|
||
expect(response.status).toBe(201);
|
||
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
|
||
it('rejects a compression level for a tar target (no compression to control)', async () => {
|
||
const fixturePath = path.join(config.storageDir, 'for-tar.zip');
|
||
buildZipFixture(fixturePath);
|
||
|
||
const response = await request(app)
|
||
.post('/api/jobs')
|
||
.field('targetFormats', JSON.stringify(['tar']))
|
||
.field('qualities', JSON.stringify([5]))
|
||
.attach('files', fixturePath, 'for-tar.zip');
|
||
|
||
expect(response.status).toBe(201);
|
||
expect(response.body.jobs[0].error).toMatch(/Invalid quality/);
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
|
||
it('parses a compound .tar.gz original filename as a single source format, not "gz"', async () => {
|
||
const zlib = await import('node:zlib');
|
||
const fixturePath = path.join(config.storageDir, 'backup.tar.gz');
|
||
await fs.writeFile(fixturePath, zlib.gzipSync(Buffer.from('irrelevant payload for this check')));
|
||
|
||
const response = await request(app)
|
||
.post('/api/jobs')
|
||
.field('targetFormats', JSON.stringify(['zip']))
|
||
.attach('files', fixturePath, 'backup.tar.gz');
|
||
|
||
expect(response.status).toBe(201);
|
||
const job = await getJobByUuid(prisma, response.body.jobs[0].id);
|
||
expect(job.sourceFormat).toBe('tar.gz');
|
||
|
||
await fs.unlink(fixturePath);
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/api/jobs.test.js`
|
||
Expected: FAIL — every archive conversion is reported as `Unsupported conversion`, and the `.tar.gz` case reports `sourceFormat: 'gz'`.
|
||
|
||
- [ ] **Step 3: Write minimal implementation**
|
||
|
||
In `src/app.js`, add imports near the other converter imports:
|
||
|
||
```js
|
||
import { registerArchiveConverters } from './converters/archive.js';
|
||
import { extractExtension, stripExtension } from './archiveExtensions.js';
|
||
```
|
||
|
||
Update `registerAllConverters`:
|
||
|
||
```js
|
||
function registerAllConverters() {
|
||
if (convertersRegistered) return;
|
||
registerImageConverters();
|
||
registerImageToPdfConverter();
|
||
registerDocumentConverters();
|
||
registerIcoConverter();
|
||
registerHeicConverter();
|
||
registerFontConverter();
|
||
registerDfontConverter();
|
||
registerEbookConverter();
|
||
registerArchiveConverters();
|
||
convertersRegistered = true;
|
||
}
|
||
```
|
||
|
||
Update `isValidQuality`:
|
||
|
||
```js
|
||
function isValidQuality(targetFormat, quality) {
|
||
if (quality === null || quality === undefined) return true;
|
||
if (!Number.isInteger(quality)) return false;
|
||
if (targetFormat === 'gif' || targetFormat === 'ico' || targetFormat === 'tar') return false;
|
||
if (targetFormat === 'png') return quality >= 0 && quality <= 9;
|
||
if (['zip', 'tar.gz', 'tar.bz2', '7z', 'tar.7z'].includes(targetFormat)) return quality >= 0 && quality <= 9;
|
||
return quality >= 1 && quality <= 100;
|
||
}
|
||
```
|
||
|
||
Replace the multer `filename` callback's extension line:
|
||
|
||
```js
|
||
filename: (req, file, cb) => {
|
||
const ext = extractExtension(file.originalname);
|
||
cb(null, `${uuidv4()}.${ext}`);
|
||
},
|
||
```
|
||
|
||
Replace the `sourceFormat` line in the `POST /api/jobs` handler:
|
||
|
||
```js
|
||
const sourceFormat = extractExtension(file.filename);
|
||
```
|
||
|
||
Replace the `downloadFilename` line in the download route:
|
||
|
||
```js
|
||
const downloadFilename = `${stripExtension(job.originalFilename)}.${job.targetFormat}`;
|
||
```
|
||
|
||
In `src/worker.js`, add the import near the other converter imports and the call in `main()`:
|
||
|
||
```js
|
||
import { registerArchiveConverters } from './converters/archive.js';
|
||
```
|
||
|
||
```js
|
||
registerImageConverters();
|
||
registerImageToPdfConverter();
|
||
registerDocumentConverters();
|
||
registerIcoConverter();
|
||
registerHeicConverter();
|
||
registerFontConverter();
|
||
registerDfontConverter();
|
||
registerEbookConverter();
|
||
registerArchiveConverters();
|
||
```
|
||
|
||
- [ ] **Step 4: Run test to verify it passes**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run test/api/jobs.test.js`
|
||
Expected: PASS
|
||
|
||
- [ ] **Step 5: Run the full backend test suite**
|
||
|
||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run`
|
||
Expected: PASS, except the two pre-existing unrelated failures documented in `CLAUDE.md` (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, both failing on `main` already).
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add src/app.js src/worker.js test/api/jobs.test.js
|
||
git commit -m "feat(archive): wire archive converters, validation, and extension parsing into app.js/worker.js"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 9: Frontend — archive family, compression control, extension parsing
|
||
|
||
**Files:**
|
||
- Create: `frontend/src/utils/archiveExtensions.js`
|
||
- Modify: `frontend/src/data/formats.js`
|
||
- Modify: `frontend/src/utils/fileFamily.js`
|
||
- Modify: `frontend/src/components/FormatsGrid.jsx`
|
||
- Modify: `frontend/src/components/FileConfigCard.jsx`
|
||
- Modify: `frontend/src/pages/HomePage.jsx`
|
||
- Modify: `frontend/src/locales/en.json`
|
||
- Modify: `frontend/src/locales/fr.json`
|
||
|
||
**Interfaces:**
|
||
- Produces: `extensionOf(filename: string): string` (frontend equivalent of the backend's `extractExtension`, same compound-extension list).
|
||
|
||
There is no automated frontend test runner in this project (`frontend/package.json` has no `test` script) — every prior converter family's frontend work was verified by manual browser check, and this task follows the same convention.
|
||
|
||
- [ ] **Step 1: Create the shared frontend extension helper**
|
||
|
||
```js
|
||
// frontend/src/utils/archiveExtensions.js
|
||
const DOUBLE_EXTENSIONS = ['tar.gz', 'tar.bz2', 'tar.7z'];
|
||
|
||
export function extensionOf(filename) {
|
||
const lower = filename.toLowerCase();
|
||
const match = DOUBLE_EXTENSIONS.find((ext) => lower.endsWith(`.${ext}`));
|
||
return match ?? filename.split('.').pop().toLowerCase();
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Replace both existing buggy `extensionOf`/`extensionOf` duplicates**
|
||
|
||
In `frontend/src/pages/HomePage.jsx`, remove the local function:
|
||
|
||
```js
|
||
function extensionOf(fileName) {
|
||
return fileName.split('.').pop().toLowerCase();
|
||
}
|
||
```
|
||
|
||
and add the import instead, near the other imports:
|
||
|
||
```js
|
||
import { extensionOf } from '../utils/archiveExtensions.js';
|
||
```
|
||
|
||
In `frontend/src/components/FileConfigCard.jsx`, remove:
|
||
|
||
```js
|
||
function extensionOf(fileName) {
|
||
return fileName.split('.').pop();
|
||
}
|
||
```
|
||
|
||
and add:
|
||
|
||
```js
|
||
import { extensionOf } from '../utils/archiveExtensions.js';
|
||
```
|
||
|
||
(both files already call `extensionOf(...)` with the same single argument shape, so no call-site changes are needed beyond the import swap.)
|
||
|
||
- [ ] **Step 3: Add default compression levels for archive targets**
|
||
|
||
In `frontend/src/pages/HomePage.jsx`, extend `DEFAULT_QUALITY`:
|
||
|
||
```js
|
||
const DEFAULT_QUALITY = {
|
||
jpg: 80,
|
||
jpeg: 80,
|
||
webp: 80,
|
||
avif: 50,
|
||
tiff: 80,
|
||
png: 6,
|
||
zip: 6,
|
||
'tar.gz': 6,
|
||
'tar.bz2': 9,
|
||
'7z': 5,
|
||
'tar.7z': 5,
|
||
};
|
||
```
|
||
|
||
- [ ] **Step 4: Add the `archives` format family**
|
||
|
||
In `frontend/src/data/formats.js`, add a new entry to `FORMAT_FAMILIES`:
|
||
|
||
```js
|
||
{
|
||
key: 'archives',
|
||
formats: ['zip', 'tar', 'tar.gz', 'tar.bz2', '7z', 'tar.7z', 'rar'],
|
||
},
|
||
```
|
||
|
||
- [ ] **Step 5: Add the archive family icon in both places it's defined**
|
||
|
||
In `frontend/src/utils/fileFamily.js`, add the import and map entry:
|
||
|
||
```js
|
||
import { Image, FileText, TextAa, BookOpen, Archive, File as FileIcon } from '@phosphor-icons/react';
|
||
```
|
||
|
||
```js
|
||
const FAMILY_ICONS = {
|
||
images: Image,
|
||
documents: FileText,
|
||
fonts: TextAa,
|
||
ebooks: BookOpen,
|
||
archives: Archive,
|
||
};
|
||
```
|
||
|
||
In `frontend/src/components/FormatsGrid.jsx` (a separate, duplicated `FAMILY_ICONS` map — confirmed by reading the file):
|
||
|
||
```js
|
||
import { Image, FileText, TextAa, BookOpen, Archive } from '@phosphor-icons/react';
|
||
|
||
const FAMILY_ICONS = {
|
||
images: Image,
|
||
documents: FileText,
|
||
fonts: TextAa,
|
||
ebooks: BookOpen,
|
||
archives: Archive,
|
||
};
|
||
```
|
||
|
||
- [ ] **Step 6: Add the compression-level control to `FileConfigCard.jsx`**
|
||
|
||
Add the constant near `QUALITY_FORMATS`:
|
||
|
||
```js
|
||
const ARCHIVE_COMPRESSION_FORMATS = ['zip', 'tar.gz', 'tar.bz2', '7z', 'tar.7z'];
|
||
```
|
||
|
||
Add a new conditional block alongside the existing PNG block:
|
||
|
||
```jsx
|
||
{ARCHIVE_COMPRESSION_FORMATS.includes(item.targetFormat) && (
|
||
<RangeField
|
||
label={t('quality.compression')}
|
||
value={item.quality}
|
||
min={0}
|
||
max={9}
|
||
onChange={(value) => onQualityChange(index, value)}
|
||
/>
|
||
)}
|
||
```
|
||
|
||
- [ ] **Step 7: Add the `formats.archives` translation key**
|
||
|
||
In `frontend/src/locales/en.json`, add to the `formats` object:
|
||
|
||
```json
|
||
"archives": "Archives"
|
||
```
|
||
|
||
In `frontend/src/locales/fr.json`, add to the `formats` object:
|
||
|
||
```json
|
||
"archives": "Archives"
|
||
```
|
||
|
||
- [ ] **Step 8: Manual verification**
|
||
|
||
Follow the project's own guidance on checking for an already-running dev server before starting a new one (`CLAUDE.md`'s manual end-to-end testing section). If none is running:
|
||
|
||
```bash
|
||
npm run dev --prefix frontend
|
||
```
|
||
|
||
In a browser: upload a small `.zip`, confirm the target chips include `TAR`, `TAR.GZ`, `TAR.BZ2`, `7Z`, `TAR.7Z`, `ZIP` (not `RAR`); confirm the compression slider (0–9) appears for `zip`/`tar.gz`/`tar.bz2`/`7z`/`tar.7z` targets and disappears for `tar`; confirm the "Archives" tile appears on the homepage formats grid with the archive icon. Then run a real conversion end-to-end (requires the backend server and worker running with Task 8's changes — restart them if they were already running before this task, per `CLAUDE.md`'s note that a running worker won't pick up new converter code).
|
||
|
||
- [ ] **Step 9: Commit**
|
||
|
||
```bash
|
||
git add frontend/src/utils/archiveExtensions.js frontend/src/data/formats.js frontend/src/utils/fileFamily.js frontend/src/components/FormatsGrid.jsx frontend/src/components/FileConfigCard.jsx frontend/src/pages/HomePage.jsx frontend/src/locales/en.json frontend/src/locales/fr.json
|
||
git commit -m "feat(frontend): add archive format family, compression control, and fixed double-extension parsing"
|
||
```
|
||
|
||
---
|
||
|
||
## Self-Review Notes
|
||
|
||
- **Spec coverage:** every section of `docs/superpowers/specs/2026-08-01-archive-conversion-design.md` maps to a task — architecture/libraries (Tasks 2–6), compression level (Tasks 2–5, 8), double-extension gap (Tasks 1, 8, 9), MIME/sniffing (Task 7), security (Task 2), registration wiring (Task 8), frontend (Task 9), testing (all tasks include their own).
|
||
- **Dependency correction from the spec:** the spec's draft table originally proposed `unbzip2-stream` for `tar.bz2` extraction; verifying `7zip-min`'s actual API during planning showed `7za` already reads bzip2 natively, so Task 4 uses `7zip-min` for both directions and that dependency was dropped from both the spec and this plan.
|
||
- **Type/name consistency checked:** `EXTRACTORS`/`CREATORS` keys, `extractExtension`/`stripExtension` (backend) and `extensionOf` (frontend), `assertNoPathEscape`, `registerArchiveConverters` are spelled identically everywhere they're referenced across tasks.
|
||
- **No placeholders:** every step has literal code, not a description of code; the two open technical uncertainties honestly flagged (Task 4's exact `7za` intermediate filename, Task 2's zip-slip fixture construction) each carry a concrete fallback action, not a TBD.
|