WebP Cloud Services Blog

How do browsers implement CSS Filters? And how to achieve the same filter effects using govips

· Nova Kwok

For those of you familiar with CSS, you’re likely aware of the various filter effects it offers. Applying these filters, along with gradients and blending modes, can help achieve effects similar to Instagram filters.

Using CSS filters is quite simple. You just need to add the filter property and pass in functions and parameters. For instance, if you want to adjust the contrast of an image, you can do so like this:

filter: contrast(200%);

You can even combine multiple filters:

filter: contrast(175%) brightness(103%) sepia(0.8);

Here’s a more complete example in HTML:

<!DOCTYPE html>
<html>
<head>
<style>
img {
  filter: contrast(175%) brightness(103%) sepia(0.8);
}
</style>
</head>
<body>

<img src="pineapple.jpg" alt="Pineapple" width="300" height="300">

</body>
</html>

In most cases, knowing how to use these filters suffices. However, our new project, WebP Cloud, recently added support for image filters. We aim to transform images with applied filters (rather than applying filters through frontend code).

When developing this feature, we need to understand not only how to use filters but also the underlying principles behind browser implementations of these effects. We then apply the same concepts to the libvips/govips library.

So, returning to the initial question, how do browsers implement a specific filter?

W3C

The World Wide Web Consortium (W3C) is a standardization organization that establishes a range of standards and encourages developers and content providers to adhere to these standards. This includes specifications for languages, development guidelines, and behavior of rendering engines. CSS is one such standard defined by them.

On the W3C website, you can find the CSS SPECIFICATIONS. The latest 2023 snapshot can be accessed here.

CSS Filter Reference Implementation

Let’s take the contrast filter as an example. In practice, you use it like this:

filter: contrast(200%);

The SVG equivalent expression for the contrast function, as provided by W3C, is given below. Here, you can see the parameters accepted by the contrast function:

<filter id="contrast">
  <feComponentTransfer>
      <feFuncR type="linear" slope="[amount]" intercept="-(0.5 * [amount]) + 0.5"/>
      <feFuncG type="linear" slope="[amount]" intercept="-(0.5 * [amount]) + 0.5"/>
      <feFuncB type="linear" slope="[amount]" intercept="-(0.5 * [amount]) + 0.5"/>
  </feComponentTransfer>
</filter>

Let’s break down the code:

  • feFuncR, feFuncG, feFuncB: These can be thought of as individual functions that operate on the corresponding pixel channels (R, G, B).
  • type indicates the function’s type. Since the type is linear, meaning a linear function, it includes slope and intercept attributes:
    • slope represents the slope (a in the mathematical formula).
    • intercept represents the intercept (b in the mathematical formula).

Linear Function

Each attribute in the contrast code above can be understood with simpler mathematical knowledge:

For linear, the function is defined by the following linear equation: C' = slope * C + intercept

In simpler terms, think of it like the linear equation you learned in school: y = ax + b, where ‘a’ is the slope and ‘b’ is the intercept. Here’s an illustration:

For example, increasing contrast actually involves performing this linear function calculation on each pixel.

Most color images are composed of RGB values, at least in the RGB color space.

In SVG, we have feFuncR, feFuncG, and feFuncB, corresponding to the three RGB values.

The slope represents the ‘a’ in the mathematical formula, so in the SVG code, it’s represented as amount.

The intercept represents the ‘b’ in the mathematical formula, so in the SVG code, it’s -(0.5 * [amount]) + 0.5.

So, you have your mathematical formula: y = amount * x + (-(0.5 * amount)) + 0.5 (For simplicity, we won’t simplify this expression right now.)

If the user inputs contrast(200%), then amount is 2, so our formula becomes:

y = 2x + (-(0.5 * 2)) + 0.5

For any given red value x, a new red value y can be calculated, and the same applies to green and blue values. After processing each pixel, when they are combined, the image transforms into a new appearance with the filter applied.

Implementation in govips

Now that we understand the principle behind the contrast filter, we can put it into action using govips! Fortunately, govips provides bindings for Linear operations, which greatly simplifies our task. The following code sets the contrast to 200%:

slope := []float64{2}
intercept := []float64{-(128 * 2) + 128}
_ = img.Linear(slope, intercept)

In CSS, we use 1 to represent the full intensity of a color. In Go, we use 255 as an equivalent replacement.

Both slope and intercept are arrays.

Huh? Why do we have only one element in our arrays? Wouldn’t this only adjust the red channel? The answer is no, this is sufficient because the RGB calculation formula is the same. According to libvips documentation:

If the arrays of constants have just one element, that constant is used for all image bands.

If there’s only one element in the array, the given value will be applied to all image channels (i.e., RGB).

Implementation in Chromium

In theory, this approach seems logical, but to be sure, it’s always good to take a look at the browser’s source code. Luckily, Chromium is open source, so let’s dig into the source code!

At line 41 of render_surface_filters.cc, there seems to be code related to contrast!

void GetContrastMatrix(float amount, float matrix[20]) {
  memset(matrix, 0, 20 * sizeof(float));
  matrix[0] = matrix[6] = matrix[12] = amount;
  matrix[4] = matrix[9] = matrix[14] = (-0.5f * amount + 0.5f);
  matrix[18] = 1.f;
}

Wait, how did they come up with a matrix? Why set the values at positions 0, 6, and 12 to amount? The assignments for 4, 9, and 14 seem to follow the formula -(0.5 * [amount]) + 0.5, which makes sense…

Matrices

Matrices are a topic usually covered in high school or university. But don’t worry, we won’t get into any matrix addition, subtraction, or multiplication here, nor will we touch on linear equations or transformations. After all, your elementary or middle school math knowledge will be sufficient!

We just need to understand two concepts:

  1. The meaning of a 3x2 matrix is 3 rows and 2 columns, represented in the form of a two-dimensional array:
a   b
c   d
e   f
  1. In color space, besides RGB, there’s another model called RGBA, where A stands for Alpha channel, representing the opacity of an image.

  • RGBA is a color space model composed of the RGB color space and an Alpha channel. RGBA represents Red, Green, Blue, and Alpha channels.

    The Alpha channel indicates the opacity of an image and can be expressed as a percentage, integer, or a real number from 0 to 1, similar to RGB parameters. For instance, if a pixel’s Alpha channel value is 0%, it is completely transparent and invisible. A value of 100% means the pixel is fully opaque, like in traditional digital images.

    For the common image formats like JPG and PNG, JPG doesn’t provide an Alpha channel, so there are no “transparent” parts in JPG images. However, PNG, due to its Alpha channel, can have transparent parts. Also, modern image formats like WebP and AVIF support Alpha channels.

For RGB, the color of each pixel can be represented using a 3x3 matrix:

R  G  B
R  G  B
R  G  B

In the RGBA color space, the color of each pixel can be represented using a 4x4 matrix:

R  G  B  A
R  G  B  A
R  G  B  A
R  G  B  A

In practice, we often add an additional column for offsets, making it more flexible. This is because:

  1. Using only a 3x3 or 4x4 matrix can represent linear transformations, meaning we can control how scaling and rotation are based on the input image’s channel values, but we can’t add an offset to it.
  2. Adding an extra column turns it into a 3x4 or 4x5 matrix, allowing for offset calculations. The offset is the intercept we’ve been mentioning.

For an RGBA 4x5 matrix, it looks like this, using zero-based numbers for clarity and adding RGBA “coordinates” for better understanding. The rows represent the desired result, while the columns can be understood as the colors we’re manipulating:

     R    G    B   A    Offset
R    0    1    2   3    4
G    5    6    7   8    9
B    10   11   12  13   14
A    15   16   17  18   19

Given a slope of amount (denoted as ‘a’), for the RGB matrix, the diagonal values are R, G, and B themselves. Thus, it becomes:

     R    G    B   A    Offset
R    a    1    2   3    4
G    5    a    7   8    9
B    10   11   a   13   14
A    15   16   17  18   19

The term -0.5 * [amount] + 0.5 in the algorithm represents the intercept. It’s the same formula for all three RGB colors, and the intercept (offset) is denoted as ‘o’. Thus, the matrix becomes:

     R    G    B   A    Offset
R    a    1    2   3    o
G    5    a    7   8    o
B    10   11   a   13   o
A    15   16   17  18   19

In programming terms, if our flattened matrix is denoted as ’m’, then:

m[0]=m[6]=m[12]=amount

m[4]=m[9]=m[14]=-0.5*amount+0.5

Now, let’s proceed with the final row.

Since our matrix has an alpha channel, we don’t want to change the opacity; we only want to modify the color channels (RGB). As there’s no mention of altering the alpha channel in the CSS standard for increasing brightness, we set m[18]=1, which is also the diagonal for A, meaning no change.

Looking back at the Chromium code, does it look identical?

Congratulations, you’ve learned the basics of a certain matrix operation! 🎉

Implementing the sepia Filter

Now that we’ve learned about matrix operations, let’s try implementing a vintage filter!

SVG

In the W3C specification, the sepia filter is defined as follows:

<filter id="sepia">
  <feColorMatrix type="matrix"
             values="
    (0.393 + 0.607 * [1 - amount]) (0.769 - 0.769 * [1 - amount]) (0.189 - 0.189 * [1 - amount]) 0 0
    (0.349 - 0.349 * [1 - amount]) (0.686 + 0.314 * [1 - amount]) (0.168 - 0.168 * [1 - amount]) 0 0
    (0.272 - 0.272 * [1 - amount]) (0.534 - 0.534 * [1 - amount]) (0.131 + 0.869 * [1 - amount]) 0 0
    0 0 0 1 0"/>
</filter>

It’s quite understandable; it’s still a matrix, a 4x5 matrix. The calculations for the three RGB colors are no longer linear, so the offset at the end of each row is 0. We don’t need to alter the image’s opacity, so the fourth element of the fourth row remains 1.

Implementation in Chromium

The above SVG code is directly used in Chromium, with the caller place having the 1 - amount logic applied:

void GetSepiaMatrix(float amount, float matrix[20]) {
  matrix[0] = 0.393f + 0.607f * amount;
  matrix[1] = 0.769f - 0.769f * amount;
  matrix[2] = 0.189f - 0.189f * amount;
  matrix[3] = matrix[4] = 0.f;

  matrix[5] = 0.349f - 0.349f * amount;
  matrix[6] = 0.686f + 0.314f * amount;
  matrix[7] = 0.168f - 0.168f * amount;
  matrix[8] = matrix[9] = 0.f;

  matrix[10] = 0.272f - 0.272f * amount;
  matrix[11] = 0.534f - 0.534f * amount;
  matrix[12] = 0.131f + 0.869f * amount;
  matrix[13] = matrix[14] = 0.f;

  matrix[15] = matrix[16] = matrix[17] = matrix[19] = 0.f;
  matrix[18] = 1.f;
}

govips

The goal here is to generate a similar matrix based on user input and then use a specific method to create an image. In this case, the method is Recomb, using a 3x3 RGB matrix as an example:

matrix := [][]float64{
    {0.393 + 0.607*(1-amount), 0.769 - 0.769*(1-amount), 0.189 - 0.189*(1-amount)},
    {0.349 - 0.349*(1-amount), 0.686 + 0.314*(1-amount), 0.168 - 0.168*(1-amount)},
    {0.272 - 0.272*(1-amount), 0.534 - 0.534*(1-amount), 0.131 + 0.869*(1-amount)},
}
err := img.Recomb(matrix)
fmt.Println(err)

Your editor and Go compiler:

Argh! govips doesn’t have the Recomb method.

Adding the recomb Method to govips

Since govips doesn’t provide a Recomb method, after looking through the https://github.com/libvips/libvips library, we found the vips_recomb function. We plan to add the calling functionality for this function to govips. To ensure our implementation is correct, we’ve referred to the source code and usage of Node.js sharp’s recomb function.

In Sharp’s code, they use it like this:

sharp(input)
  .recomb([
   [0.3588, 0.7044, 0.1368],
   [0.2990, 0.5870, 0.1140],
   [0.2392, 0.4696, 0.0912],
  ])
  .raw()
  .toBuffer(function(err, data, info) {
    // data contains the raw pixel data after applying the matrix
    // With this example input, a sepia filter has been applied
  });

Our approach is quite similar – passing a matrix to operate with. Now, let’s modify govips.

The vips_recomb documentation is available here: https://www.libvips.org/API/current/libvips-conversion.html#vips-recomb, and its signature is as follows:

int
vips_recomb (VipsImage *in,
             VipsImage **out,
             VipsImage *m,
             ...);

Since recomb falls under libvips’s conversion, we need to add the function prototype in conversion.c and conversion.h:

// vips/conversion.c

int recomb_image(VipsImage *in, VipsImage **out, VipsImage *m) {
  return vips_recomb(in, out, m, NULL);
}
// vips/conversion.h

int recomb_image(VipsImage *in, VipsImage **out, VipsImage *m);

In conversion.go, we use CGO to call the just-created recomb_image:

// vips/conversion.go

func vipsRecomb(in *C.VipsImage, m *C.VipsImage) (*C.VipsImage, error) {
	incOpCounter("recomb")
	var out *C.VipsImage

	if err := C.recomb_image(in, &out, m); err != 0 {
		return nil, handleImageError(out)
	}

	return out, nil
}

Next, in image.go, we add a method to img. This method flattens the Go 2D array, converts it into a C array, and then uses vips_image_new_from_memory to generate a new image.

// vips/image.go

func (r *ImageRef) Recomb(matrix [][]float64) error {
	numBands := r.Bands()
	// Ensure the provided matrix is 3x3
	if len(matrix) != 3 || len(matrix[0]) != 3 || len(matrix[1]) != 3 || len(matrix[2]) != 3 {
		return errors.New("Invalid recombination matrix")
	}
	// If the image is RGBA, expand the matrix to 4x4
	if numBands == 4 {
		matrix = append(matrix, []float64{0, 0, 0, 1})
		for i := 0; i < 3; i++ {
			matrix[i] = append(matrix[i], 0)
		}
	} else if numBands != 3 {
		return errors.New("unsupported number of bands")
	}

	// Flatten the matrix
	var matrixValues []float64
	for _, row := range matrix {
		for _, value := range row {
			matrixValues = append(matrixValues, value)
		}
	}

	// Convert the Go slice to a C array and get its size
	matrixPtr := unsafe.Pointer(&matrixValues[0])
	matrixSize := C.size_t(len(matrixValues) * 8) // 8 bytes for each float64

	// Create a VipsImage from the matrix in memory
	matrixImage := C.vips_image_new_from_memory(matrixPtr, matrixSize, C.int(numBands), C.int(numBands), 1, C.VIPS_FORMAT_DOUBLE)

	// Check for any Vips errors
	errMsg := C.GoString(C.vips_error_buffer())
	if errMsg != "" {
		C.vips_error_clear()
		return errors.New("Vips error: " + errMsg)
	}

	// Recombine the image using the matrix
	out, err := vipsRecomb(r.image, matrixImage)
	if err != nil {
		return err
	}

	r.setImage(out)
	return nil
}

Usage

Use Go mod replace to use the modified code:

module awesomeProject

go 1.21

require github.com/davidbyttow/govips/v2 v2.13.0

require (
	golang.org/x/image v0.5.0 // indirect
	golang.org/x/net v0.7.0 // indirect
	golang.org/x/text v0.7.0 // indirect
)

// You may need to run export GOPRIVATE=github.com/webp-sh/govips
replace github.com/davidbyttow/govips/v2 => github.com/webp-sh/govips/v2 v2.13.0

Then, you can call the functions normally, for example:

package main

import (
	"fmt"
	"os"

	"github.com/davidbyttow/govips/v2/vips"
)

func main() {
	vips.Startup(nil)
	defer vips.Shutdown()

	img, _ := vips.NewImageFromFile("test.png")
	matrix := [][]float64{
		{0.3588, 0.7044, 0.1368},
		{0.2990, 0.5870, 0.1140},
		{0.2392, 0.4696, 0.0912},
	}

	err := img.Recomb(matrix)
	if err != nil {
		fmt.Println(err)
	}
	image1bytes, _, _ := img.ExportPng(vips.NewPngExportParams())
	_ = os.WriteFile("experiment.png", image1bytes, 0644)
}

Effects Demonstration

Effects demonstrated on the online website:

Effects using govips, almost identical!

Pull Request

We’ve submitted a Pull Request to govips, hoping that they will merge this feature. After all, we don’t want to keep using the “replace” method, and we also hope that more people with similar needs can directly benefit from this convenient Recomb feature.

As of the time of writing, there hasn’t been any response yet. So, if you also want to use the Recomb method, you can “replace” it in your go.mod file to use our fork.

Perhaps in the near future, we will add more features to govips, such as incorporating common CSS filter presets. This will eliminate the need to refer to W3C documents to determine the corresponding matrices or manually construct matrices for Recomb. It’s a way of contributing to the entire community of https://github.com/libvips/libvips, which is the true essence of open source.

Application in WebP Cloud

We spent a lot of time studying the algorithms behind CSS filters, as we recently added watermark and filter features to WebP Cloud.

The watermark feature is particularly suitable for displaying images on websites where you don’t want your images to be easily stolen by others. We provide two ways to use it:

  • Create a watermark configuration file in the Dashboard and apply it to your proxy. This way, all images output by this proxy will have the watermark.
  • If you just want to give it a try or only have the need to add watermarks to some images, you can use the Params by adding them to the end of the image URL, like this:
https://559a238.webp.ee/images/create-proxy.png
?visual_effect=watermark,
text__5oiR55qE5LiW55WM5piv5LuA5LmI,
width__0.1,
height__0.1,
offset_x__0.23,
offset_y__0.34,
opacity__1,
color__001489,
font__WGlhb2xhaVND
Original ImageWatermarked Image

Regarding the filters, we currently support more than a dozen filters (1977, aden, brannan, brooklyn, clarendon, earlybird, gingham, hudson, etc.), and we’re continuously optimizing them. This exceeds the number of filters offered by Instagram. Just like with watermarks, you can directly apply filters to proxies in the Dashboard, allowing all images to have filter effects. Alternatively, you can add parameters directly to the URL, like this:

https://559a238.webp.ee/images/create-proxy.png
?visual_effect=filter,name__moon
Original ImageFiltered Image
https://559a238.webp.ee/images/create-proxy.pnghttps://559a238.webp.ee/images/create-proxy.png?visual_effect=filter,name__moon

The WebP Cloud Services team is a small team of three people from Shanghai and Helsingborg. As we are not seeking funding and have no profit pressure, we are committed to doing what we believe is right. We strive to do our best within the limits of our resources and capabilities. We also engage in various activities on our products without affecting the services we provide externally.

As with the new features we add, WebP Cloud firmly believes that all users should have equal rights of use. Therefore, whether you are a paying user or a free user, the watermark and filter features are available for unlimited use. You can choose to apply several watermarks or filters globally to a proxy, or you can experiment with them by adding URL parameters.

If you’re interested in trying it out or want to see what images look like with filters or watermarks applied, feel free to check out our documentation: https://docs.webp.se/webp-cloud/visual-effects/

References

The WebP Cloud Services team is a small team of three individuals from Shanghai and Helsingborg. Since we are not funded and have no profit pressure, we remain committed to doing what we believe is right. We strive to do our best within the scope of our resources and capabilities. We also engage in various activities without affecting the services we provide to the public, and we continuously explore novel ideas in our products.

If you find this service interesting, feel free to log in to the WebP Cloud Dashboard to experience it. If you’re curious about other magical features it offers, take a look at our WebP Cloud Services Docs. We hope everyone enjoys using it!


Discuss on Hacker News