close
Skip to content

Add JPEG XL support, loaded on demand#77584

Open
adamsilverstein wants to merge 10 commits into
trunkfrom
try/jxl-lazy-load
Open

Add JPEG XL support, loaded on demand#77584
adamsilverstein wants to merge 10 commits into
trunkfrom
try/jxl-lazy-load

Conversation

@adamsilverstein
Copy link
Copy Markdown
Member

@adamsilverstein adamsilverstein commented Apr 23, 2026

Summary

Third experimental approach to #76981: lazy-load the 3 MB vips-jxl.wasm module only when a user first processes a JXL image. No canonical plugin required, and no bundle-size hit on editor pages that never touch JXL.

Sibling of:

This PR keeps the WASM inside Gutenberg but splits it into its own script module chunk (@wordpress/vips/jxl-wasm) that the browser only fetches on first JXL use.

Fixes #76981.

How it works

  1. New tiny script module: packages/vips/src/jxl-wasm.ts exports the base64 data URL for vips-jxl.wasm. Added to wpScriptModuleExports in packages/vips/package.json so the build emits a standalone build/modules/vips/jxl-wasm.min.js (~3.0 MB).
  2. Main-thread helper: vipsEnsureJxlSupport() in packages/vips/src/vips-worker.ts dynamic-imports @wordpress/vips/jxl-wasm and RPC-passes the URL to the worker via setJxlWasmUrl(). The import and RPC are cached, so subsequent calls are no-ops.
  3. Worker-side reinit: The worker stores the URL and adds vips-jxl.wasm to dynamicLibraries on the next getVips() call. If vips was already initialized without JXL, the existing instance is discarded and recreated with JXL support. locateFile returns the URL when vips requests vips-jxl.wasm.
  4. Upload trigger: packages/upload-media calls vipsEnsureJxlSupport() from prepareItem whenever the input or output type is image/jxl. image/jxl is added to CLIENT_SIDE_SUPPORTED_MIME_TYPES, 'jxl' to VALID_IMAGE_FORMATS and ImageFormat, and the vipsConvertImageFormat wrapper MIME union. JXL encoding uses effort=3 (libvips default of 7 is too slow for interactive use).
  5. Dynamic dependency: The module asset file for @wordpress/vips/worker correctly declares a dynamic module_dependencies entry on @wordpress/vips/jxl-wasm, so WordPress's import map resolves it at runtime.

Screencast

jxl.to.jpeg.on.upload.mp4

Bundle size impact

Measured locally from npm run build. Raw is the on-disk minified size; transferred (gzip) is what the browser actually downloads and what the CI size bot reports:

Artifact Raw (minified) Transferred (gzip) When loaded
build/modules/vips/worker.min.js 13,752,457 B (~13.1 MB) ~4.56 MB Always — unchanged vs trunk (+262 B)
build/modules/vips/jxl-wasm.min.js 3,109,421 B (~3.0 MB) ~1.1 MB On-demand — first JXL use only

The CI size bot's headline +1.1 MB (+13.83%) is the gzipped JXL chunk, not the 3.0 MB raw figure — the actual network cost is smaller than the on-disk size. Editor sessions that never process a JXL image transfer no extra bytes (the worker grows by only 262 B). When a user uploads a JXL image, the browser fetches the separate chunk (~1.1 MB gzip) once and caches it.

The chunk is the vips-jxl.wasm dynamic library (2.22 MB raw / 0.77 MB gzip) inlined as a base64 data URL inside a JS module. Base64 inlining adds ~33% on disk and ~0.33 MB to the gzip transfer (1.1 MB vs the 0.77 MB the raw .wasm would gzip to) — see the discussion below on lighter-weight alternatives.

Comparison matrix

Aspect Trunk #77570 (bundled) #76990 (plugin) This PR (lazy chunk)
Editor worker size 13.1 MB 16.1 MB 13.1 MB 13.1 MB
JXL available out of the box No Yes After install Yes
Plugin install flow None Required for non-admins None
JXL WASM hosted by Gutenberg Plugin Gutenberg
WASM downloaded only on JXL use No Yes (per plugin) Yes
Independent versioning of JXL WASM No Yes No
Graceful fallback for users who can't install N/A Yes (server-side if supported) N/A

Test plan

  • npm run build produces build/modules/vips/worker.min.js at ~13.1 MB and a separate build/modules/vips/jxl-wasm.min.js at ~3.0 MB.
  • Verify the worker asset file declares a dynamic module_dependencies entry on @wordpress/vips/jxl-wasm.
  • Load the block editor without uploading JXL: DevTools Network tab should show no request for jxl-wasm.min.js.
  • Upload a .jxl file: the browser fetches jxl-wasm.min.js once, then processes the image client-side (resize, compress, thumbnails).
  • Upload a JXL, then a JPEG in the same session: JXL-initialized vips instance handles both.
  • Configure JXL as an output format (e.g., JPEG → JXL) and verify transcoding works.
  • Verify existing image formats (JPEG, PNG, WebP, AVIF, GIF, HEIC) still work unchanged.
  • Confirm size bot report aligns with locally measured numbers.

Refs #76981.

Add client-side JPEG XL support without growing the vips worker
bundle. The 3 MB vips-jxl.wasm module is split into its own
@wordpress/vips/jxl-wasm script module that is only fetched the
first time a JXL image is processed.

How it works:
- packages/vips/src/jxl-wasm.ts is a tiny new script module whose
  only job is to export the base64 data URL for vips-jxl.wasm.
- @wordpress/vips/jxl-wasm is registered as a new wpScriptModuleExports
  entry so the build emits build/modules/vips/jxl-wasm.min.js (~3 MB)
  as a separately loadable module.
- vips-worker.ts adds vipsEnsureJxlSupport(), which on first call
  dynamic-imports '@wordpress/vips/jxl-wasm' to get the data URL,
  then RPC-passes it to the worker via setJxlWasmUrl().
- The worker (packages/vips/src/index.ts) stores the URL, adds
  vips-jxl.wasm to dynamicLibraries on the next getVips() call, and
  returns the URL from locateFile(). If vips was already initialized
  without JXL, the existing instance is discarded so the reinit picks
  up JXL support.
- packages/upload-media calls vipsEnsureJxlSupport() from prepareItem
  whenever the input or output type is image/jxl. image/jxl is added
  to CLIENT_SIDE_SUPPORTED_MIME_TYPES and 'jxl' to VALID_IMAGE_FORMATS
  and the ImageFormat type. The vipsConvertImageFormat wrapper MIME
  union is widened accordingly.
- JXL encoding uses effort=3 (libvips default 7 is too slow for
  interactive use).

Size impact:
- worker.min.js: unchanged (~13.1 MB, same as trunk).
- jxl-wasm.min.js: new separate module (~3.0 MB), fetched only when
  a JXL image is encountered.

Alternative to #77570, which bundles vips-jxl.wasm directly into the
worker (+3 MB on every editor page load). Opened so the size bot can
compare.

Refs #76981.
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 23, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: BlackStar1991 <blackstar1991@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 23, 2026

Size Change: +1.1 MB (+13.43%) ⚠️

Total Size: 9.31 MB

📦 View Changed
Filename Size Change
build/modules/vips/jxl-wasm.min.js 1.1 MB +1.1 MB (new file) 🆕
build/modules/vips/worker.min.js 4.56 MB +259 B (+0.01%)
build/scripts/upload-media/index.min.js 12.3 kB +159 B (+1.31%)

compressed-size-action

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 23, 2026

Flaky tests detected in 96125ce.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/26313133000
📝 Reported issues:

Resolve conflict in packages/upload-media/src/store/private-actions.ts:
trunk moved image format transcoding from client-side prepareItem to
server-driven thumbnail generation. Adopt the new pipeline and keep
the JXL WASM lazy-load, but trigger it in two places: prepareItem for
JXL input files, and transcodeImageItem for server-requested JXL
output.
@adamsilverstein adamsilverstein self-assigned this Apr 28, 2026
@adamsilverstein adamsilverstein requested review from andrewserong and youknowriad and removed request for youknowriad April 28, 2026 17:43
@adamsilverstein adamsilverstein added [Status] In Progress Tracking issues with work in progress [Type] Feature New feature to highlight in changelogs. [Feature] Client Side Media Media processing in the browser with WASM labels May 18, 2026
@adamsilverstein
Copy link
Copy Markdown
Member Author

A few notes from a review pass:

Bundle size / lighter-weight JXL options (re #76981)

  • The size bot reports gzip, so the headline +1.1 MB is the gzipped chunk, not the 3 MB raw figure. Updated the description's table to show both.
  • vips-jxl.wasm is 2.22 MB raw / 0.77 MB gzip. Base64-inlining it into a JS module bumps the gzip transfer to ~1.1 MB (+~0.33 MB). Cheapest same-architecture trim would be serving the raw .wasm instead of base64 — though I know we inline deliberately to avoid host MIME/path issues, so that's a tradeoff.
  • mediabunny doesn't help here — it's a video/audio toolkit, no still-image/JXL path.
  • jSquash @jsquash/jxl (standalone libjxl) is smaller — enc 0.49 MB / dec 0.30 MB gzip — but it's a second WASM runtime. We'd lose the single libvips pipeline (decode → resize → thumbnails → metadata → encode) and have to shuttle raw pixels between two heaps. Not worth it unless JXL were the only format we processed. The 3 MB is inherent to libjxl regardless of wrapper; the real win is keeping it lazy, which this PR does.
  • No browser encodes JXL (and WebCodecs has none), so a WASM encoder is unavoidable for now — which favors lazy-load over bundling.

Minor code note

  • In getVips(), the JXL reinit path (vipsPromise = undefined) orphans the previously-initialized vips instance without an explicit teardown (cleanup is the Emscripten auto-delete delay fn, not a heap teardown). It's bounded — only fires when a non-JXL image is processed before a JXL one in the same session, and only once — but given the prior OOM history (Investigate and fix crashes and console errors during client-side image upload processing #76706) a brief comment acknowledging it would help.

@adamsilverstein adamsilverstein requested a review from ramonjd May 21, 2026 17:05
JXL is not broadly web-compatible: most browsers (including Chrome) cannot
display it and the server cannot read it (GD/Imagick have no JXL decoder, and
fileinfo reports it as image/x-jxl). Uploading JXL as-is was rejected outright
— by the editor's allowed-MIME check and by core's wp_check_filetype_and_ext()
— and even when allowed produced an undisplayable attachment with no
dimensions or sub-sizes.

Decode JXL to JPEG client-side with vips (the JXL WASM module is already
lazy-loaded on demand) and upload the JPEG, mirroring how HEIC is handled. The
original .jxl is preserved as a companion file in $metadata['original'] so no
data is lost. The editor and front end now use the portable JPEG, with real
dimensions and the full set of sub-sizes.

- Register image/jxl as an allowed upload MIME type, and restore the type
  during validation via a magic-byte-checked wp_check_filetype_and_ext filter
  so the sideloaded original passes despite the image/x-jxl finfo mismatch.
- Add original-jxl sideload handling to the REST controller, skipping the
  dimension read since JXL cannot be measured server-side.
- Generalize the HEIC companion delete hook to clean up JXL originals too.
- Add an e2e test and JXL asset covering the conversion and companion.
# Conflicts:
#	packages/upload-media/src/store/private-actions.ts
@adamsilverstein
Copy link
Copy Markdown
Member Author

I did some testing on this feature, made some small fixes and got it working, it now properly handles uploaded JXL files.

I created a JXL using Squoosh for testing:
Cliff-Palace.jxl.zip

Because JPEG XL (JXL) is not supported in Chromium where client-side media is active, and generally to provide a web-safe format for users, I decided to treat JXL uploads similar to HEIC uploads.

So we do the following:

  1. sideload the original JXL upload so it is preserved for the user
  2. output encode as JPEG for all display (sub) sizes, jpeg gets used on the front end/srcset and editor views.

The output format should also honor the image_output_formats filter, so JXL output will still be possible, just not the default.

See screencast:

jxl.to.jpeg.on.upload.mp4

@adamsilverstein
Copy link
Copy Markdown
Member Author

I will work on a core backport once the core client-side media feature is restored.

# Conflicts:
#	packages/vips/CHANGELOG.md
#	packages/vips/README.md
@adamsilverstein adamsilverstein changed the title Client-side media: Lazy-load JPEG XL (JXL) WASM on demand Add JPEG XL support, loaded on demand May 28, 2026
Guard the contract that high-bit-depth (>8-bit) and gain-map JXL uploads
flatten to 8-bit JPEG sub-sizes while the full-fidelity original is
preserved byte-for-byte as the .jxl companion file.

Add two 200x150 fixtures (a genuine 16-bit JXL and an 8-bit JXL carrying
an ISO 21496-1 jhgm gain-map box) plus a self-contained generator
script. Both decode in wasm-vips to a JPEG derivative; the tests assert
the JPEG main + sub-size and verify the stored original is identical to
the upload, proving the bit depth and gain map survive in the original.
# Conflicts:
#	packages/upload-media/CHANGELOG.md
#	packages/upload-media/src/store/types.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Feature] Client Side Media Media processing in the browser with WASM [Status] In Progress Tracking issues with work in progress [Type] Feature New feature to highlight in changelogs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add JPEG XL (JXL) support

1 participant