WebP Server Go 0.16.0 Released: More Reliable Image Format Negotiation and CDN Caching

Hi everyone, it has been a while!

We have spent much of the past few months iterating on and optimizing the internal infrastructure behind WebP Cloud and Public Services, so we have published very few posts. Some friends even wondered whether our products had quietly disappeared 😆. Fortunately, they have not—and we finally have time to share an update on WebP Server Go.

We believe that open-source software is the foundation of freedom. While continuing to develop our SaaS products, WebP Cloud and Public Services, we have also kept maintaining and improving the open-source version so that users always have more choices.

We recently released WebP Server Go 0.16.0. At first glance, this release may seem to contain only a few changes to browser detection and response headers, but it solves a long-standing problem: how to reliably serve the same image in a format the browser actually supports, and how CDNs should correctly cache those different responses.

What Is WebP Server Go?

WebP Server Go is an open-source image optimization program licensed under GPLv3, with its source code available at https://github.com/webp-sh/webp_server_go.

Without changing the original image URL, it can dynamically optimize JPG, PNG, and other images into WebP, AVIF, or JPEG XL on the fly, then serve the smallest version among the formats supported by the browser.

For example, an image URL on a web page can remain unchanged:

https://example.com/pics/photo.jpg

However, a browser that supports AVIF may receive image/avif, one that supports WebP may receive image/webp, and a client that supports none of these modern formats will fall back to the original image. This process requires neither batch-processing images in advance nor changing image URLs on your website.

Why We Redesigned Browser Capability Detection

WebP Server Go previously considered both the Accept request header and the User-Agent, attempting to infer WebP, AVIF, or JPEG XL support from the browser name, operating system, and version number.

This approach covered some early browsers that did not send a complete Accept header. However, as browser versions and image format support continued to change, User-Agent inference became increasingly prone to errors.

JPEG XL is a good example. Chrome once removed its experimental support, Safari later added official support, and more recently both Mozilla and Chromium began working toward enabling JPEG XL by default. Maintaining rules such as “this browser supports this format starting from this version” would mean constantly chasing browser release cycles. Worse, an incorrect guess could cause us to send an image that the client cannot decode.

HTTP already provides a mechanism designed for exactly this problem: the client uses Accept to explicitly tell the server what it can accept, and the server uses Content-Type to tell the client what it ultimately returned.

Starting with 0.16.0, we have therefore removed User-Agent-based image format inference and now use the Accept request header exclusively for content negotiation.

Parsing Accept More Strictly

Removing User-Agent detection does not mean that we can simply search the request header for image/webp, image/avif, or image/jxl, because Accept also contains quality values, or q values.

Consider the following request:

Accept: image/webp,image/jxl;q=0

Here, image/jxl;q=0 does not mean “JPEG XL is supported, but with the lowest priority.” It means “JPEG XL is not acceptable for this request.” According to RFC 9110, content with a quality value of 0 is considered unacceptable.

Previous versions might have interpreted the request as supporting JPEG XL merely because the string contained image/jxl—yes, we got fooled!

if strings.Contains(accept, "image/jxl") {
    supported["jxl"] = true
}

Version 0.16.0 parses the complete media type and quality value, and enables a modern image format only when that format is explicitly listed with q>0.

The new rules can be summarized as follows:

• image/webp: WebP is supported
• image/avif;q=0.8: AVIF is supported
• image/jxl;q=0: JPEG XL is not supported
• image/* or */*: Do not infer support for any modern image format
• A missing or invalid Accept header: Fall back to the original image

We deliberately do not treat wildcards as a signal that WebP, AVIF, or JPEG XL is supported. A wildcard may semantically accept many types of content, but it does not prove that the client can actually decode a particular format. When in doubt, returning the original image is safer than optimistically sending an image the client cannot display.

For example, suppose AVIF and WebP conversion are enabled on the server, and the converted AVIF file is the smallest:

curl -I \
  -H 'Accept: image/avif,image/webp,image/*;q=0.8' \
  http://127.0.0.1:3333/DSC05955.jpg

The response will look similar to this:

HTTP/1.1 200 OK
Content-Type: image/avif
Vary: Accept
X-Compression-Rate: 0.13

If we instead explicitly reject JPEG XL:

curl -I \
  -H 'Accept: image/jxl;q=0,image/webp' \
  http://127.0.0.1:3333/DSC05955.jpg

WebP Server Go will not return image/jxl.

Vary: Accept and CDN Caching

Content negotiation solves the question of which format the origin server should return, but adding a CDN in front of it introduces another problem.

From a CDN’s perspective, the following two requests access the same URL:

GET /pics/photo.jpg
Accept: image/avif,image/webp

GET /pics/photo.jpg
Accept: image/jpeg,image/png

If the CDN uses only the URL as its cache key, the AVIF image generated and cached for the first visitor could be sent directly to a later visitor whose browser does not support AVIF. Because WebP Server Go did not previously express this response variation completely, our documentation had always recommended disabling CDN caching for images. That was safe, but it also gave up one of the CDN’s most important performance benefits.

Starting with 0.16.0, we decided to solve this problem. Every response that goes through image format negotiation now includes:

Vary: Accept

This response header tells shared caches that responses for the same URL vary according to the request’s Accept header, so a cached response for one format must not be reused for a client with different capabilities.

However, an origin server returning Vary: Accept does not mean that every CDN will automatically split its cache accordingly:

• Cloudflare requires Vary support to be enabled in Cache Rules; we also recommend normalizing Accept into the capabilities actually used, such as image/avif, image/webp, and image/jxl
• Fastly handles Vary according to the HTTP specification, but normalizing Accept is recommended to prevent equivalent requests from creating too many cache entries
• Amazon CloudFront requires Accept to be added to the cache key in a Cache Policy
• For CDNs that do not support Vary or custom cache keys, caching should still be disabled for WebP Server Go image paths

We have rewritten the relevant documentation. For detailed configuration and verification instructions, see Using WebP Server Go with a CDN.

If you use image processing parameters such as width, height, max_width, or max_height, make sure those query parameters remain part of the CDN cache key. After enabling the new variant caching rules, remember to purge any previously generated cache entries.

What This Means for Existing Users

Modern browsers loading resources through <img>, CSS, or other image contexts usually declare their supported image formats explicitly in Accept, so no changes to web page code should be necessary after upgrading.

Behavior will mainly change in the following situations:

• A request was previously identified as supporting a modern format solely from its User-Agent, without explicitly declaring that format in Accept
• A script, crawler, or command-line client sends only Accept: */*
• A custom reverse proxy or CDN removes or rewrites the browser’s Accept request header while forwarding the request

In 0.16.0, these requests will take the more conservative original-image fallback path. If you use cURL or a programmatic client to test a particular format, explicitly send the corresponding Accept header, for example:

curl -I -H 'Accept: image/webp' https://example.com/pics/photo.jpg

This upgrade introduces no required configuration changes. Existing CONVERT_TYPES, quality settings, and image path settings will continue to work. Since we no longer parse the User-Agent, we have also removed the github.com/mileusna/useragent dependency, making format negotiation simpler and easier to test.

How to Upgrade

If you deploy with Docker Compose, run:

docker compose pull
docker compose up -d

After upgrading, test the same image with different Accept request headers, confirm that the responses include Vary: Accept, and configure the corresponding cache rules for your CDN.

Version 0.16.0 does not introduce a flashy new image processing feature, but it brings WebP Server Go back to clearer HTTP semantics: browsers explicitly declare their capabilities, the origin selects the response format, and caches can understand why responses differ. As formats such as JPEG XL become available in more browsers, this mechanism will also adapt to future changes better than continuously maintaining browser version tables.

We hope you enjoy this release. If you encounter any problems while upgrading or configuring your CDN, or if you have new ideas, please let us know by opening an issue at Issues · webp-sh/webp_server_go!