Markdown conversion doc + CLAUDE.md improve
This commit is contained in:
@@ -0,0 +1,575 @@
|
||||
# Markdown (.md) Document Conversion Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add Markdown (`.md`) as a document format convertible in both directions against `html`, `txt`, `docx`, and `pdf`.
|
||||
|
||||
**Architecture:** All new conversion functions live in the existing `src/converters/document.js` and are registered through the existing `registerDocumentConverters()` function via the existing `register()` call in `src/converters/registry.js` — no registry/API/frontend changes are needed because `GET /api/formats` and `POST /api/jobs` are already format-agnostic (verified by reading `src/app.js`: they call `listTargetFormats`/`resolveConverter` generically). `md->html` and `html->md` are direct library calls; `md->pdf` and `pdf->md` reuse the existing `renderHtmlToPdf` / `extractPdfPageTexts` helpers; `docx->md` reuses the existing `mammoth.convertToHtml` call piped through the new HTML→MD library; `md->docx` is the only genuinely new algorithm (a `markdown-it` token-stream walk into `docx` `Paragraph`/`TextRun` objects); `md<->txt` is a raw file copy.
|
||||
|
||||
**Tech Stack:** Node.js, `markdown-it` (Markdown → HTML, v15.0.0), `turndown` (HTML → Markdown, v7.2.4), existing `docx`/`mammoth`/`puppeteer`/`pdfjs-dist` already in the project.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- MIME type for `.md` is `text/markdown` (per the spec's MIME plumbing section).
|
||||
- `.md` files are undetectable by magic bytes — `resolveInputFormat` must trust the declared extension, exactly like the existing `txt`/`html` entries in `UNDETECTABLE_TEXT_FORMATS`.
|
||||
- No changes to `src/app.js`, `src/jobs/jobRepository.js`, the database schema, or any frontend file — confirmed unnecessary in the spec's Scope section.
|
||||
- `md -> docx` must never throw on unrecognized Markdown constructs (tables, blockquotes, images) — it degrades to a plain-text paragraph for that block rather than failing the job, per the spec's fallback rule.
|
||||
- Every new registered conversion pair must have a passing test in the style already used in `test/converters/document.test.js` / `test/converters/documentFromPdf.test.js` (fixture in a `beforeAll` tmp dir, `resolve(source, target)` from the registry, assert on output content/validity).
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Dependencies and MIME plumbing for `.md`
|
||||
|
||||
**Files:**
|
||||
- Modify: `package.json` (add `markdown-it` and `turndown` to `dependencies`)
|
||||
- Modify: `src/mime.js`
|
||||
- Test: `test/mime.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `OUTPUT_MIME_TYPES.md === 'text/markdown'`, `UNDETECTABLE_TEXT_FORMATS.md === 'text/markdown'` in `src/mime.js`, consumed by later tasks' tests and by `src/mime.js`'s existing exported functions (`outputMimeType`, `resolveInputFormat`) unchanged in signature.
|
||||
|
||||
- [ ] **Step 1: Install the new dependencies**
|
||||
|
||||
Run: `npm install markdown-it turndown`
|
||||
Expected: `package.json` gains `"markdown-it": "^15.0.0"` and `"turndown": "^7.2.4"` under `"dependencies"`; `package-lock.json` updates accordingly. (If already installed from prior research in this session, this is a no-op — verify with `npm ls markdown-it turndown`.)
|
||||
|
||||
- [ ] **Step 2: Write the failing tests**
|
||||
|
||||
In `test/mime.test.js`, add inside the existing `describe('outputMimeType', ...)` block, as a new assertion appended to the existing "returns the MIME type for a known target format" test body:
|
||||
|
||||
```js
|
||||
expect(outputMimeType('md')).toBe('text/markdown');
|
||||
```
|
||||
|
||||
Add a new test inside the existing `describe('resolveInputFormat', ...)` block, modeled directly on the existing "trusts the declared format for undetectable txt files" test right above it:
|
||||
|
||||
```js
|
||||
it('trusts the declared format for undetectable md files', async () => {
|
||||
const fixturePath = path.join(import.meta.dirname, 'fixtures', 'sample.md');
|
||||
await fs.writeFile(fixturePath, '# plain markdown, no magic bytes');
|
||||
|
||||
const result = await resolveInputFormat(fixturePath, 'md');
|
||||
|
||||
expect(result).toEqual({ mime: 'text/markdown', valid: true });
|
||||
|
||||
await fs.unlink(fixturePath);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run test/mime.test.js`
|
||||
Expected: FAIL — `outputMimeType('md')` throws `No known MIME type for target format "md"`, and the new `resolveInputFormat` test gets `{ mime: null, valid: false }` instead of the expected object.
|
||||
|
||||
- [ ] **Step 4: Implement the MIME plumbing**
|
||||
|
||||
In `src/mime.js`, add `md: 'text/markdown'` to the `OUTPUT_MIME_TYPES` object (after the `docx` entry):
|
||||
|
||||
```js
|
||||
const OUTPUT_MIME_TYPES = {
|
||||
jpg: 'image/jpeg',
|
||||
jpeg: 'image/jpeg',
|
||||
png: 'image/png',
|
||||
webp: 'image/webp',
|
||||
gif: 'image/gif',
|
||||
tiff: 'image/tiff',
|
||||
avif: 'image/avif',
|
||||
bmp: 'image/bmp',
|
||||
pdf: 'application/pdf',
|
||||
html: 'text/html',
|
||||
txt: 'text/plain',
|
||||
docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
md: 'text/markdown',
|
||||
};
|
||||
```
|
||||
|
||||
And add `md: 'text/markdown'` to `UNDETECTABLE_TEXT_FORMATS`:
|
||||
|
||||
```js
|
||||
const UNDETECTABLE_TEXT_FORMATS = {
|
||||
txt: 'text/plain',
|
||||
html: 'text/html',
|
||||
md: 'text/markdown',
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run test/mime.test.js`
|
||||
Expected: PASS (all tests in the file, including the two new assertions).
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add package.json package-lock.json src/mime.js test/mime.test.js
|
||||
git commit -m "feat: add MIME plumbing for markdown (.md) format"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: `md -> html` and `html -> md` converters
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/converters/document.js`
|
||||
- Test: `test/converters/document.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `register` from `./registry.js` (existing).
|
||||
- Produces: registry entries `resolve('md', 'html')` and `resolve('html', 'md')`, each `{ family: 'document', convert: async (inputPath, outputPath) => void }`, consumed by Task 3's `md -> pdf` implementation (which reuses the same `MarkdownIt` render call) and by the test suite.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
In `test/converters/document.test.js`, add two new `it` blocks inside `describe('document converters', ...)`, after the existing "converts DOCX to a valid PDF" test:
|
||||
|
||||
```js
|
||||
it('converts MD to HTML', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture.md');
|
||||
await fs.writeFile(inputPath, '# Heading\n\nSome **bold** text.');
|
||||
const outputPath = path.join(tmpDir, 'from-md.html');
|
||||
const entry = resolve('md', 'html');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const html = await fs.readFile(outputPath, 'utf8');
|
||||
expect(html).toContain('<h1>Heading</h1>');
|
||||
expect(html).toContain('<strong>bold</strong>');
|
||||
});
|
||||
|
||||
it('converts HTML to MD', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture-for-md.html');
|
||||
await fs.writeFile(inputPath, '<h1>Heading</h1><p>Some <strong>bold</strong> text.</p>');
|
||||
const outputPath = path.join(tmpDir, 'from-html.md');
|
||||
const entry = resolve('html', 'md');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const markdown = await fs.readFile(outputPath, 'utf8');
|
||||
expect(markdown).toContain('# Heading');
|
||||
expect(markdown).toContain('**bold**');
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: FAIL with `TypeError: Cannot read properties of null (reading 'convert')` for both new tests (since `resolve('md', 'html')` and `resolve('html', 'md')` return `null`).
|
||||
|
||||
- [ ] **Step 3: Implement the converters**
|
||||
|
||||
In `src/converters/document.js`, add the two new imports at the top, alongside the existing ones:
|
||||
|
||||
```js
|
||||
import MarkdownIt from 'markdown-it';
|
||||
import TurndownService from 'turndown';
|
||||
```
|
||||
|
||||
Add the two new functions after `convertDocxToPdf` (and before `extractPdfPageTexts`):
|
||||
|
||||
```js
|
||||
async function convertMdToHtml(inputPath, outputPath) {
|
||||
const markdown = await fs.readFile(inputPath, 'utf8');
|
||||
const html = new MarkdownIt().render(markdown);
|
||||
await fs.writeFile(outputPath, html);
|
||||
}
|
||||
|
||||
async function convertHtmlToMd(inputPath, outputPath) {
|
||||
const html = await fs.readFile(inputPath, 'utf8');
|
||||
const markdown = new TurndownService({ headingStyle: 'atx' }).turndown(html);
|
||||
await fs.writeFile(outputPath, markdown);
|
||||
}
|
||||
```
|
||||
|
||||
Register both in `registerDocumentConverters()`, after the existing `register({ ..., sourceFormat: 'html', targetFormat: 'pdf', ... })` line:
|
||||
|
||||
```js
|
||||
register({ family: 'document', sourceFormat: 'md', targetFormat: 'html', convert: convertMdToHtml });
|
||||
register({ family: 'document', sourceFormat: 'html', targetFormat: 'md', convert: convertHtmlToMd });
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: PASS (all tests in the file).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/converters/document.js test/converters/document.test.js
|
||||
git commit -m "feat: add md<->html document converters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: `md -> pdf` and `pdf -> md` converters
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/converters/document.js`
|
||||
- Test: `test/converters/document.test.js`, `test/converters/documentFromPdf.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `renderHtmlToPdf(html, outputPath)` (existing, defined at the top of `document.js`), `extractPdfPageTexts(inputPath)` (existing), `convertPdfToTxt` (existing, defined via `extractPdfPageTexts`).
|
||||
- Produces: registry entries `resolve('md', 'pdf')` and `resolve('pdf', 'md')`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
In `test/converters/document.test.js`, add after the `md -> html` / `html -> md` tests from Task 2:
|
||||
|
||||
```js
|
||||
it('converts MD to a valid PDF', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture-for-pdf.md');
|
||||
await fs.writeFile(inputPath, '# Heading\n\nSome text.');
|
||||
const outputPath = path.join(tmpDir, 'from-md.pdf');
|
||||
const entry = resolve('md', 'pdf');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const detected = await detectInputMime(outputPath);
|
||||
expect(detected.mime).toBe('application/pdf');
|
||||
}, 20000);
|
||||
```
|
||||
|
||||
In `test/converters/documentFromPdf.test.js`, add after the existing "extracts text from PDF to HTML" test:
|
||||
|
||||
```js
|
||||
it('extracts text from PDF to MD', async () => {
|
||||
const outputPath = path.join(tmpDir, 'output.md');
|
||||
const entry = resolve('pdf', 'md');
|
||||
|
||||
await entry.convert(pdfFixturePath, outputPath);
|
||||
|
||||
const markdown = await fs.readFile(outputPath, 'utf8');
|
||||
expect(markdown).toContain('Extractable fixture text');
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js test/converters/documentFromPdf.test.js`
|
||||
Expected: FAIL with `TypeError: Cannot read properties of null (reading 'convert')` for both new tests.
|
||||
|
||||
- [ ] **Step 3: Implement the converters**
|
||||
|
||||
In `src/converters/document.js`, add after `convertHtmlToMd` (from Task 2):
|
||||
|
||||
```js
|
||||
async function convertMdToPdf(inputPath, outputPath) {
|
||||
const markdown = await fs.readFile(inputPath, 'utf8');
|
||||
const html = new MarkdownIt().render(markdown);
|
||||
await renderHtmlToPdf(html, outputPath);
|
||||
}
|
||||
```
|
||||
|
||||
Register in `registerDocumentConverters()`, after the `md -> html` / `html -> md` lines added in Task 2:
|
||||
|
||||
```js
|
||||
register({ family: 'document', sourceFormat: 'md', targetFormat: 'pdf', convert: convertMdToPdf });
|
||||
register({ family: 'document', sourceFormat: 'pdf', targetFormat: 'md', convert: convertPdfToTxt });
|
||||
```
|
||||
|
||||
Note `pdf -> md` reuses the existing `convertPdfToTxt` function directly (no new function needed) — extracted PDF text is already valid Markdown, and this keeps the two outputs byte-identical by construction.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js test/converters/documentFromPdf.test.js`
|
||||
Expected: PASS (all tests in both files).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/converters/document.js test/converters/document.test.js test/converters/documentFromPdf.test.js
|
||||
git commit -m "feat: add md<->pdf document converters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: `docx -> md` and `md -> docx` converters
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/converters/document.js`
|
||||
- Test: `test/converters/document.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `mammoth.convertToHtml({ path })` (existing import), `TurndownService` (from Task 2's import), `Document`, `Paragraph`, `TextRun`, `Packer` from `docx` (existing import — `HeadingLevel` added in this task), `MarkdownIt` (from Task 2's import).
|
||||
- Produces: registry entries `resolve('docx', 'md')` and `resolve('md', 'docx')`; a new internal (unexported) helper function `markdownToDocxParagraphs(markdownText)` returning `Paragraph[]`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
In `test/converters/document.test.js`, add after the `md -> pdf` test from Task 3:
|
||||
|
||||
```js
|
||||
it('converts DOCX to MD containing the source text', async () => {
|
||||
const outputPath = path.join(tmpDir, 'from-docx.md');
|
||||
const entry = resolve('docx', 'md');
|
||||
|
||||
await entry.convert(docxFixturePath, outputPath);
|
||||
|
||||
const markdown = await fs.readFile(outputPath, 'utf8');
|
||||
expect(markdown).toContain('Hello from the fixture document');
|
||||
}, 20000);
|
||||
|
||||
it('converts MD to DOCX preserving headings, bold text, and bullet lists', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture-for-docx.md');
|
||||
await fs.writeFile(
|
||||
inputPath,
|
||||
'# Report Title\n\nSome **bold** finding.\n\n- first point\n- second point\n'
|
||||
);
|
||||
const outputPath = path.join(tmpDir, 'from-md.docx');
|
||||
const entry = resolve('md', 'docx');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const mammoth = await import('mammoth');
|
||||
const result = await mammoth.default.convertToHtml({ path: outputPath });
|
||||
expect(result.value).toContain('Report Title');
|
||||
expect(result.value).toContain('<strong>bold</strong>');
|
||||
expect(result.value).toContain('first point');
|
||||
expect(result.value).toContain('<li>');
|
||||
}, 20000);
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: FAIL with `TypeError: Cannot read properties of null (reading 'convert')` for both new tests.
|
||||
|
||||
- [ ] **Step 3: Implement the converters**
|
||||
|
||||
In `src/converters/document.js`, change the existing `docx` import line from:
|
||||
|
||||
```js
|
||||
import { Document, Paragraph, TextRun, Packer } from 'docx';
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```js
|
||||
import { Document, Paragraph, TextRun, HeadingLevel, Packer } from 'docx';
|
||||
```
|
||||
|
||||
Add, after `convertMdToPdf` (from Task 3):
|
||||
|
||||
```js
|
||||
const MARKDOWN_HEADING_LEVELS = {
|
||||
h1: HeadingLevel.HEADING_1,
|
||||
h2: HeadingLevel.HEADING_2,
|
||||
h3: HeadingLevel.HEADING_3,
|
||||
h4: HeadingLevel.HEADING_4,
|
||||
h5: HeadingLevel.HEADING_5,
|
||||
h6: HeadingLevel.HEADING_6,
|
||||
};
|
||||
|
||||
function markdownInlineTokensToRuns(children) {
|
||||
const runs = [];
|
||||
let bold = false;
|
||||
let italics = false;
|
||||
for (const token of children ?? []) {
|
||||
if (token.type === 'strong_open') bold = true;
|
||||
else if (token.type === 'strong_close') bold = false;
|
||||
else if (token.type === 'em_open') italics = true;
|
||||
else if (token.type === 'em_close') italics = false;
|
||||
else if (token.type === 'text' || token.type === 'code_inline') {
|
||||
if (token.content) runs.push(new TextRun({ text: token.content, bold, italics }));
|
||||
} else if (token.type === 'softbreak' || token.type === 'hardbreak') {
|
||||
runs.push(new TextRun({ text: ' ' }));
|
||||
}
|
||||
}
|
||||
return runs;
|
||||
}
|
||||
|
||||
function markdownToDocxParagraphs(markdownText) {
|
||||
const tokens = new MarkdownIt().parse(markdownText, {});
|
||||
const paragraphs = [];
|
||||
const listStack = [];
|
||||
|
||||
for (let i = 0; i < tokens.length; i += 1) {
|
||||
const token = tokens[i];
|
||||
|
||||
if (token.type === 'heading_open') {
|
||||
const inline = tokens[i + 1];
|
||||
paragraphs.push(
|
||||
new Paragraph({
|
||||
heading: MARKDOWN_HEADING_LEVELS[token.tag],
|
||||
children: markdownInlineTokensToRuns(inline.children),
|
||||
})
|
||||
);
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (token.type === 'bullet_list_open') {
|
||||
listStack.push({ type: 'bullet' });
|
||||
continue;
|
||||
}
|
||||
if (token.type === 'ordered_list_open') {
|
||||
listStack.push({ type: 'ordered', counter: 0 });
|
||||
continue;
|
||||
}
|
||||
if (token.type === 'bullet_list_close' || token.type === 'ordered_list_close') {
|
||||
listStack.pop();
|
||||
continue;
|
||||
}
|
||||
if (token.type === 'list_item_open') {
|
||||
const current = listStack[listStack.length - 1];
|
||||
if (current?.type === 'ordered') current.counter += 1;
|
||||
continue;
|
||||
}
|
||||
if (token.type === 'list_item_close') {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (token.type === 'paragraph_open') {
|
||||
const inline = tokens[i + 1];
|
||||
const runs = markdownInlineTokensToRuns(inline.children);
|
||||
const current = listStack[listStack.length - 1];
|
||||
if (current?.type === 'bullet') {
|
||||
paragraphs.push(new Paragraph({ children: runs, bullet: { level: listStack.length - 1 } }));
|
||||
} else if (current?.type === 'ordered') {
|
||||
paragraphs.push(
|
||||
new Paragraph({ children: [new TextRun({ text: `${current.counter}. ` }), ...runs] })
|
||||
);
|
||||
} else {
|
||||
paragraphs.push(new Paragraph({ children: runs }));
|
||||
}
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (token.type === 'fence' || token.type === 'code_block') {
|
||||
if (token.content.trim()) paragraphs.push(new Paragraph({ children: [new TextRun({ text: token.content })] }));
|
||||
continue;
|
||||
}
|
||||
|
||||
if (token.type === 'hr') {
|
||||
paragraphs.push(new Paragraph({ children: [new TextRun({ text: '---' })] }));
|
||||
}
|
||||
}
|
||||
|
||||
return paragraphs;
|
||||
}
|
||||
|
||||
async function convertDocxToMd(inputPath, outputPath) {
|
||||
const result = await mammoth.convertToHtml({ path: inputPath });
|
||||
const markdown = new TurndownService({ headingStyle: 'atx' }).turndown(result.value);
|
||||
await fs.writeFile(outputPath, markdown);
|
||||
}
|
||||
|
||||
async function convertMdToDocx(inputPath, outputPath) {
|
||||
const markdown = await fs.readFile(inputPath, 'utf8');
|
||||
const paragraphs = markdownToDocxParagraphs(markdown);
|
||||
const doc = new Document({ sections: [{ children: paragraphs }] });
|
||||
const buffer = await Packer.toBuffer(doc);
|
||||
await fs.writeFile(outputPath, buffer);
|
||||
}
|
||||
```
|
||||
|
||||
Register in `registerDocumentConverters()`, after the `md -> pdf` / `pdf -> md` lines added in Task 3:
|
||||
|
||||
```js
|
||||
register({ family: 'document', sourceFormat: 'docx', targetFormat: 'md', convert: convertDocxToMd });
|
||||
register({ family: 'document', sourceFormat: 'md', targetFormat: 'docx', convert: convertMdToDocx });
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: PASS (all tests in the file).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/converters/document.js test/converters/document.test.js
|
||||
git commit -m "feat: add md<->docx document converters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: `md -> txt` and `txt -> md` passthrough converters
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/converters/document.js`
|
||||
- Test: `test/converters/document.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: registry entries `resolve('md', 'txt')` and `resolve('txt', 'md')`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
In `test/converters/document.test.js`, add after the `md -> docx` test from Task 4:
|
||||
|
||||
```js
|
||||
it('copies MD content unchanged to TXT', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture-for-txt.md');
|
||||
const content = '# Heading\n\nSome **bold** text.\n';
|
||||
await fs.writeFile(inputPath, content);
|
||||
const outputPath = path.join(tmpDir, 'from-md.txt');
|
||||
const entry = resolve('md', 'txt');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const output = await fs.readFile(outputPath, 'utf8');
|
||||
expect(output).toBe(content);
|
||||
});
|
||||
|
||||
it('copies TXT content unchanged to MD', async () => {
|
||||
const inputPath = path.join(tmpDir, 'fixture-for-md-from-txt.txt');
|
||||
const content = 'Plain text content.\n';
|
||||
await fs.writeFile(inputPath, content);
|
||||
const outputPath = path.join(tmpDir, 'from-txt.md');
|
||||
const entry = resolve('txt', 'md');
|
||||
|
||||
await entry.convert(inputPath, outputPath);
|
||||
|
||||
const output = await fs.readFile(outputPath, 'utf8');
|
||||
expect(output).toBe(content);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: FAIL with `TypeError: Cannot read properties of null (reading 'convert')` for both new tests.
|
||||
|
||||
- [ ] **Step 3: Implement the converters**
|
||||
|
||||
In `src/converters/document.js`, add after `convertMdToDocx` (from Task 4):
|
||||
|
||||
```js
|
||||
async function copyFileContents(inputPath, outputPath) {
|
||||
await fs.copyFile(inputPath, outputPath);
|
||||
}
|
||||
```
|
||||
|
||||
Register in `registerDocumentConverters()`, after the `docx -> md` / `md -> docx` lines added in Task 4:
|
||||
|
||||
```js
|
||||
register({ family: 'document', sourceFormat: 'md', targetFormat: 'txt', convert: copyFileContents });
|
||||
register({ family: 'document', sourceFormat: 'txt', targetFormat: 'md', convert: copyFileContents });
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `npx vitest run test/converters/document.test.js`
|
||||
Expected: PASS (all tests in the file).
|
||||
|
||||
- [ ] **Step 5: Run the full test suite to confirm no regressions**
|
||||
|
||||
Run: `DB_HOST=127.0.0.1 DB_USER=convert_user DB_PASSWORD=change_me DB_NAME=file_converter STORAGE_DIR=./storage PORT=3000 npx vitest run`
|
||||
Expected: PASS for all files except the two pre-existing, unrelated failures documented in `CLAUDE.md` (`test/cleanup.test.js` and `test/jobs/jobRepository.test.js`, both about expired-job clock/timezone handling). If any other test fails, investigate before proceeding — do not assume it's pre-existing without checking against `main`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/converters/document.js test/converters/document.test.js
|
||||
git commit -m "feat: add md<->txt passthrough document converters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Final Verification
|
||||
|
||||
After Task 5, the full Markdown conversion matrix (`md<->html`, `md<->txt`, `md<->docx`, `md<->pdf`) is registered and tested. As a manual sanity check (optional but recommended before considering this done):
|
||||
|
||||
1. Run `npm run build` and start the app locally (`npm start` plus `npm run worker` in another terminal, per existing project scripts).
|
||||
2. Upload a `.md` file through the UI and confirm `html`, `txt`, `docx`, and `pdf` all appear as target format options (served by the now-generic `/api/formats` endpoint).
|
||||
3. Upload a `.docx`, `.html`, `.txt`, and `.pdf` file and confirm `md` appears as a target format option for each.
|
||||
4. Convert one file in each direction and download the result to confirm it opens correctly in a real viewer (e.g. Word for `.docx`, a browser for `.html`/`.pdf`).
|
||||
Reference in New Issue
Block a user