Files
convert/docs/superpowers/specs/2026-07-31-heic-heif-ico-image-formats-design.md
anthony bac93c853a Add design spec for HEIC/HEIF (source-only) and ICO image formats
Drops HDR from scope: the installed sharp/libvips build has it fully
disabled and no viable library exists for either direction.
2026-07-31 09:30:26 +02:00

215 lines
12 KiB
Markdown

# HEIC/HEIF/ICO image format support
Date: 2026-07-31
## Goal
Extend the image converter with three new formats: **ICO** (bidirectional, full support) and **HEIC**/**HEIF** (source-only — decode only). A fourth requested format, **HDR (Radiance .hdr)**, is explicitly dropped: the installed sharp/libvips build has it fully disabled (`sharp.format.rad` reports `input`/`output` all `false`), and the only npm package found (`parse-hdr`) is decode-only, unmaintained since 2016, and built for WebGL textures rather than general conversion.
## Why HEIC/HEIF are source-only
The installed sharp build's `heif` codec only supports the `.avif` alias (already supported today); encoding true `.heic`/`.heif` (HEVC) requires an x265 encoder that isn't in prebuilt sharp/libvips builds for licensing reasons. No pure-JS/WASM package fills this gap either — `heic-convert` (built on `heic-decode``libheif-js`, a WASM build of libheif) only decodes HEIC/HEIF to JPEG/PNG, with no encode path. So HEIC/HEIF can only be a **source** format: decode into a buffer, then feed that buffer through the existing sharp pipeline.
## Scope
- New source formats: `heic`, `heif` → every existing image target (`jpg, jpeg, png, webp, gif, tiff, avif`) plus the new `ico` target. Nothing registers `heic`/`heif` as a *target*`listTargetFormats` (and therefore the frontend's dropdown) reflects only what's registered, so no target-side special-casing is needed.
- New bidirectional format: `ico` ↔ every existing image format.
- Out of scope: HDR, document converters, any non-image conversion family.
## New dependencies
- `icojs` — pure JS, no native bindings. Both decodes (`decodeIco(buffer, 'image/png')` → array of embedded PNG buffers) and encodes (`encodeIco([{ buffer }, ...])` → ICO buffer from PNG buffers). Safe for o2switch (no compile step).
- `heic-convert` — pure JS wrapper around `libheif-js` (WASM build of libheif, no native compile step). `convert({ buffer, format: 'PNG' })` decodes the main image in a HEIC/HEIF file to a PNG buffer. Safe for o2switch for the same reason `sharp`/`puppeteer` already are — the binary is prebuilt/shipped, not compiled locally.
## ICO size selection
ICO encoding needs a target resolution; there's no natural single "quality" scalar the way jpeg/png/pdf have. The frontend offers a dropdown of `16, 32, 48, 256, 512` px (default `256`), single choice — not a multi-size bundle.
This value doesn't reuse the existing generic `quality` column (which stays strictly about quality/compression, matching its use for jpeg/webp/avif/tiff quality and png compression level). Instead, `ConversionJob` gets a new nullable column:
```prisma
iconSize Int? @map("icon_size") @db.UnsignedSmallInt
```
Per the project's documented shadow-db-free migration process (`CLAUDE.md`), generate this with:
```
DATABASE_URL=$(node scripts/printDatabaseUrl.js) npx prisma migrate diff \
--from-schema-datasource prisma/schema.prisma --to-schema-datamodel prisma/schema.prisma --script
```
against the schema with `iconSize` added, review the output SQL, then hand-create the migration folder and `prisma migrate resolve --applied`.
Allowed values: `16, 32, 48, 256, 512`. `null`/omitted defaults to `256` at conversion time. Any other value is a validation error, mirrored the same way an invalid `quality` is today.
## Backend flow
**`src/converters/ico.js`** (new):
```js
import fs from 'node:fs/promises';
import sharp from 'sharp';
import { decodeIco, encodeIco } from 'icojs';
import { register } from './registry.js';
import { sharpFormatName, buildFormatOptions } from './image.js';
const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif'];
export const DEFAULT_ICON_SIZE = 256;
export async function encodeIcoFromInput(input, size = DEFAULT_ICON_SIZE) {
const pngBuffer = await sharp(input).resize(size, size, { fit: 'contain' }).png().toBuffer();
return Buffer.from(await encodeIco([{ buffer: pngBuffer }]));
}
async function decodeIcoLargestImage(inputPath) {
const buffer = await fs.readFile(inputPath);
const images = await decodeIco(buffer, 'image/png');
const largest = images.reduce((a, b) => (a.width * a.height >= b.width * b.height ? a : b));
return Buffer.from(largest.buffer);
}
export function registerIcoConverter() {
for (const format of IMAGE_FORMATS) {
register({
family: 'image',
sourceFormat: 'ico',
targetFormat: format,
convert: async (inputPath, outputPath, { quality } = {}) => {
const pngBuffer = await decodeIcoLargestImage(inputPath);
await sharp(pngBuffer)
.toFormat(sharpFormatName(format), buildFormatOptions(format, quality))
.toFile(outputPath);
},
});
register({
family: 'image',
sourceFormat: format,
targetFormat: 'ico',
convert: async (inputPath, outputPath, { iconSize } = {}) => {
const icoBuffer = await encodeIcoFromInput(inputPath, iconSize ?? DEFAULT_ICON_SIZE);
await fs.writeFile(outputPath, icoBuffer);
},
});
}
}
```
(`sharpFormatName`/`buildFormatOptions` are exported from `image.js` rather than duplicated.)
**`src/converters/heic.js`** (new):
```js
import fs from 'node:fs/promises';
import sharp from 'sharp';
import convert from 'heic-convert';
import { register } from './registry.js';
import { sharpFormatName, buildFormatOptions } from './image.js';
import { encodeIcoFromInput, DEFAULT_ICON_SIZE } from './ico.js';
const IMAGE_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif', 'tiff', 'avif'];
const HEIC_SOURCE_FORMATS = ['heic', 'heif'];
async function decodeToPng(inputPath) {
const buffer = await fs.readFile(inputPath);
return convert({ buffer, format: 'PNG' });
}
export function registerHeicConverter() {
for (const sourceFormat of HEIC_SOURCE_FORMATS) {
for (const targetFormat of IMAGE_FORMATS) {
register({
family: 'image',
sourceFormat,
targetFormat,
convert: async (inputPath, outputPath, { quality } = {}) => {
const pngBuffer = await decodeToPng(inputPath);
await sharp(pngBuffer)
.toFormat(sharpFormatName(targetFormat), buildFormatOptions(targetFormat, quality))
.toFile(outputPath);
},
});
}
register({
family: 'image',
sourceFormat,
targetFormat: 'ico',
convert: async (inputPath, outputPath, { iconSize } = {}) => {
const pngBuffer = await decodeToPng(inputPath);
const icoBuffer = await encodeIcoFromInput(pngBuffer, iconSize ?? DEFAULT_ICON_SIZE);
await fs.writeFile(outputPath, icoBuffer);
},
});
}
}
```
**`src/converters/image.js`**: export `sharpFormatName` and `buildFormatOptions` (currently only `buildFormatOptions` is exported; `sharpFormatName` is private) so `ico.js`/`heic.js` can reuse them instead of redefining.
**`src/app.js` / `src/worker.js`**: both currently call `registerImageConverters()`, `registerImageToPdfConverter()`, `registerDocumentConverters()` independently at startup. Add `registerIcoConverter()` and `registerHeicConverter()` to both call sites.
**`src/mime.js`**:
- `OUTPUT_MIME_TYPES.ico = 'image/x-icon'` — matches what `file-type` itself reports for real ICO content (verified against the installed `file-type` source), keeping input-detection and output-mime symmetric with every other existing entry.
- `normalizeFormat`: add `heif → heic` alongside the existing `jpg → jpeg`. Verified against the installed `file-type` v22 source: its HEIF-family detector always returns `ext: 'heic'` for *any* brand in that family (`mif1`, `msf1`, `heic`, `heix`, `hevc`, `hevx`) — the `mime` field differs (`image/heif` vs `image/heic`) but `ext` never does. Without this alias, uploading a file named `photo.heif` would have `sourceFormat = 'heif'` (from the filename) but `detectInputMime` would report `ext: 'heic'`, failing the `resolveInputFormat` content-matches-extension check.
**`src/app.js` (`POST /api/jobs`)**: parse a new `iconSizes` array the same way `qualities` is parsed today (JSON array, one entry per file, empty/omitted array = all `null`). Add `isValidIconSize(targetFormat, iconSize)`:
```js
const VALID_ICON_SIZES = [16, 32, 48, 256, 512];
function isValidIconSize(targetFormat, iconSize) {
if (iconSize === null || iconSize === undefined) return true;
if (targetFormat !== 'ico') return false; // only meaningful for ico targets
return VALID_ICON_SIZES.includes(iconSize);
}
```
Also extend `isValidQuality` to reject `ico` (no quality knob), mirroring the existing `gif` case:
```js
if (targetFormat === 'gif' || targetFormat === 'ico') return false;
```
Both `quality` and `iconSize` get passed to `createJob`.
**`src/jobs/jobRepository.js`**: add `iconSize` to the `jobSelect` projection and to `createJob`'s `data` object (`iconSize: job.iconSize ?? null`).
**`src/worker.js`**: `entry.convert(inputFilePath, outputFilePath, { quality: job.quality, iconSize: job.iconSize })`.
## Frontend
**`frontend/src/App.jsx`**:
- Each `pendingFiles` item gains an `iconSize` field, defaulted to `256` when `targetFormat` becomes `'ico'` (via `updateTargetFormat`), reset to `null` for every other format.
- New conditional control, alongside the existing png/pdf ones:
```jsx
{item.targetFormat === 'ico' && (
<label>
Taille de l'icône
<select value={item.iconSize ?? 256} onChange={(event) => updateIconSize(index, Number(event.target.value))}>
{[16, 32, 48, 256, 512].map((size) => (
<option key={size} value={size}>{size}px</option>
))}
</select>
</label>
)}
```
- New `updateIconSize` setter, mirroring `updateQuality`.
**`frontend/src/api.js`**: `uploadFiles` builds and appends an `iconSizes` array (`item.iconSize ?? null` per item), JSON-stringified, alongside `targetFormats`/`qualities`.
## Validation & edge cases
- Backend independently re-validates `iconSize` against the resolved `targetFormat` — never trusts the frontend default.
- Omitting `iconSizes` entirely (old client) behaves like an array of `null`s; every `ico` conversion then falls back to the 256px default inside the converter itself.
- An `iconSize` provided for a non-`ico` target, or a value outside the fixed set, is a per-file error in the `POST /api/jobs` response (same pattern as invalid `quality`/unsupported conversion pairs today) — it does not fail the rest of the batch.
- `ico → X`: if the source `.ico` contains multiple embedded resolutions, the largest is used (matches "give me the best available image" as the least surprising default; there's no size-selection concern on the decode side, since the destination format has no notion of embedded multi-resolution).
## Testing
- `test/converters/ico.test.js`: round-trip test (`png → ico → png`, or similar) confirming pixel dimensions match the requested `iconSize`; a decode test using a multi-resolution ICO (produced by `icojs.encodeIco` in `beforeAll` from several sharp-resized PNGs) confirming the largest embedded image is selected.
- `test/converters/heic.test.js`: decode tests for both `.heic` and `.heif` fixtures → jpg/png/ico. Requires committing small real sample files under `test/fixtures/` (e.g. `test/fixtures/sample.heic`, `test/fixtures/sample.heif`) since nothing in the toolchain can encode HEIC/HEIF to generate these programmatically.
- `test/mime.test.js` (or wherever `normalizeFormat`/`resolveInputFormat` is currently tested): add a case asserting a file named `*.heif` with real HEIF content passes `resolveInputFormat`'s validity check.
- `test/api/jobs.test.js`: extend to cover `iconSizes` parsing (valid array, value on non-ico target, out-of-range value, omitted field).
- `test/jobs/jobRepository.test.js`: extend the `createJob`/`getJobByUuid` round-trip test to include `iconSize`.
- Frontend: manual browser check only (no frontend test framework exists today, consistent with how `quality`/`targetFormat` controls are verified).