WebP Cloud Services Blog

How does different VipsInteresting values in libvips determine the cropping position of an image?

· Nova Kwok

这篇文章有简体中文版本，在： libvips 中不同的 VipsInteresting 是怎么决定图片裁剪位置的

When using WebP Cloud or WebP Server Go, we come across an option that allows users to dynamically resize images by providing width and/or height. This is particularly useful for scenarios that require fast thumbnail generation. The effect on WebP Cloud is as follows:

WebP Cloud is SaaS (Software-as-a-Service) version of our open-source component called WebP Server Go. With WebP Cloud, users can simply log in using their GitHub account and provide the URL of the original image. In return, they receive a new URL that serves the image in WebP format, with built-in WebP conversion, CDN (Content Delivery Network), and caching capabilities. For example, an image with a size of 100KB at https://blog.webp.se/hetzner-arm64/c1-board.png can be transformed into a WebP version with only 60KB in size, available at https://p2k7zwb.webp.ee/hetzner-arm64/c1-board.png, while preserving the image quality with minimal degradation.

https://blog.webp.se/hetzner-arm64/c1-board.jpghttps://p2k7zwb.webp.ee/hetzner-arm64/c1-board.jpg?width=200
107.81 KB7.27 KB

The corresponding code in WebP Server Go would look something like this:

func resizeImage(img *vips.ImageRef, extraParams config.ExtraParams) error {
	imgHeightWidthRatio := float32(img.Metadata().Height) / float32(img.Metadata().Width)
	if extraParams.Width > 0 && extraParams.Height > 0 {
		err := img.Thumbnail(extraParams.Width, extraParams.Height, vips.InterestingAll)
		if err != nil {
			return err
		}
	} else if extraParams.Width > 0 && extraParams.Height == 0 {
		err := img.Thumbnail(extraParams.Width, int(float32(extraParams.Width)*imgHeightWidthRatio), vips.InterestingAll)
		if err != nil {
			return err
		}
	} else if extraParams.Height > 0 && extraParams.Width == 0 {
		err := img.Thumbnail(int(float32(extraParams.Height)/imgHeightWidthRatio), extraParams.Height, vips.InterestingAll)
		if err != nil {
			return err
		}
	}
	return nil
}

The logic here is as follows:

  • If the user only provides either Width or Height, the corresponding dimension of the image will be changed, while the other dimension will be adjusted proportionally to prevent distortion.
  • If the user provides both Width and Height simultaneously, we perform a Thumbnail operation on the image, using the provided Width and Height values. However, the image will not be distorted as a result. Why is that?

By observing…

err := img.Thumbnail(extraParams.Width, extraParams.Height, vips.InterestingAll)

We can see that there is a vips.InterestingAll at the end. According to the documentation of davidbyttow/govips, we can understand that there are the following options:

const (
	InterestingNone      Interesting = C.VIPS_INTERESTING_NONE
	InterestingCentre    Interesting = C.VIPS_INTERESTING_CENTRE
	InterestingEntropy   Interesting = C.VIPS_INTERESTING_ENTROPY
	InterestingAttention Interesting = C.VIPS_INTERESTING_ATTENTION
	InterestingLow       Interesting = C.VIPS_INTERESTING_LOW
	InterestingHigh      Interesting = C.VIPS_INTERESTING_HIGH
	InterestingAll       Interesting = C.VIPS_INTERESTING_ALL
	InterestingLast      Interesting = C.VIPS_INTERESTING_LAST
)

Since libvips is a Go wrapper for vips, we can refer to the libvips documentation to find the following table:

VIPS_INTERESTING_NONEdo nothing
VIPS_INTERESTING_CENTREjust take the centre
VIPS_INTERESTING_ENTROPYuse an entropy measure
VIPS_INTERESTING_ATTENTIONlook for features likely to draw human attention
VIPS_INTERESTING_LOWposition the crop towards the low coordinate
VIPS_INTERESTING_HIGHposition the crop towards the high coordinate
VIPS_INTERESTING_ALLeverything is interesting
VIPS_INTERESTING_LAST

Original Image

“origin.png (PNG Image, 384 × 512 pixels)”

It can be seen that the original image is a 621 × 692 pixels image.

VIPS Crop

We conducted the testing using a modified version of WebP Server Go, with the following request URLs:

http://localhost:3333/origin.png?width=400&height=400
http://localhost:3333/origin.png?width=400&height=800

InterestingNone

“origin.png (WEBP Image, 300 × 400 pixels)”“origin.png (WEBP Image, 400 × 533 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingCentre

“origin.png (WEBP Image, 400 × 400 pixels)”“origin.png (WEBP Image, 400 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingEntropy

“origin.png (WEBP Image, 400 × 400 pixels)”“origin.png (WEBP Image, 400 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingAttention

“origin.png (WEBP Image, 400 × 400 pixels)”“origin.png (WEBP Image, 400 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingLow

“origin.png (WEBP Image, 400 × 400 pixels)”“origin.png (WEBP Image, 400 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingHigh

“origin.png (WEBP Image, 400 × 400 pixels)”“origin.png (WEBP Image, 400 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingAll

“origin.png (WEBP Image, 400 × 533 pixels)”“origin.png (WEBP Image, 600 × 800 pixels)”
http://localhost:3333/origin.png?width=400&height=400http://localhost:3333/origin.png?width=400&height=800

InterestingLast

This mode is quite Interesting, we ran an error:

2023/07/09 12:51:15 [VIPS.info] reduceh: 11 point mask
2023/07/09 12:51:15 [VIPS.info] cropping to 400x400
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x2 addr=0x7f32ef4fcb0a pc=0x7f32ef4fcb0d]

runtime stack:
runtime.throw({0x8b43e4?, 0x7f32b801f8c0?})
	/usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f32c3604cd0 sp=0x7f32c3604ca0 pc=0x43d19d
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:825 +0x3e9 fp=0x7f32c3604d30 sp=0x7f32c3604cd0 pc=0x4537e9

goroutine 19 [syscall]:
runtime.cgocall(0x7e4c20, 0xc000080b18)
	/usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000080af0 sp=0xc000080ab8 pc=0x40c57c
github.com/davidbyttow/govips/v2/vips._Cfunc_thumbnail_image(0x7f32b8006020, 0xc000114020, 0x190, 0x190, 0x7, 0x0)
	_cgo_gotypes.go:2232 +0x4c fp=0xc000080b18 sp=0xc000080af0 pc=0x79eccc
github.com/davidbyttow/govips/v2/vips.vipsThumbnail.func1(0x7f32c2dc72c8?, 0x7f32eb94e5b8?, 0x190, 0x190, 0x7, 0x0)
	/home/Nova/go/pkg/mod/github.com/davidbyttow/govips/v2@v2.13.0/vips/resample.go:57 +0xa7 fp=0xc000080b60 sp=0xc000080b18 pc=0x7a3647
github.com/davidbyttow/govips/v2/vips.vipsThumbnail(0x1?, 0x1?, 0x16?, 0x2b4?, 0x26d?)
	/home/Nova/go/pkg/mod/github.com/davidbyttow/govips/v2@v2.13.0/vips/resample.go:57 +0x91 fp=0xc000080be0 sp=0xc000080b60 pc=0x7a3511
github.com/davidbyttow/govips/v2/vips.(*ImageRef).Thumbnail(0xc000112050, 0x1?, 0x33668?, 0x33669?)
	/home/Nova/go/pkg/mod/github.com/davidbyttow/govips/v2@v2.13.0/vips/image.go:1677 +0x27 fp=0xc000080c18 sp=0xc000080be0 pc=0x7a2cc7
webp_server_go/encoder.resizeImage(0xc0001160a8?, {0x14?, 0x4ac456?})
	/home/Nova/workspace/webp-server/webp_server_go/encoder/encoder.go:35 +0xb5 fp=0xc000080c68 sp=0xc000080c18 pc=0x7af7f5
webp_server_go/encoder.webpEncoder({0xc0001160a8, 0x14}, {0xc000146140, 0x3c}, {0x0?, 0x942f30?})
	/home/Nova/workspace/webp-server/webp_server_go/encoder/encoder.go:217 +0x191 fp=0xc000080e48 sp=0xc000080c68 pc=0x7b0871
webp_server_go/encoder.convertImage({0xc0001160a8, 0x14}, {0xc000146140, 0x3c}, {0x89bdda, 0x4}, {0x0?, 0xc0000687d0?})
	/home/Nova/workspace/webp-server/webp_server_go/encoder/encoder.go:115 +0x2d0 fp=0xc000080f58 sp=0xc000080e48 pc=0x7b00d0
webp_server_go/encoder.ConvertFilter.func2()
	/home/Nova/workspace/webp-server/webp_server_go/encoder/encoder.go:72 +0x5c fp=0xc000080fe0 sp=0xc000080f58 pc=0x7afb9c
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000080fe8 sp=0xc000080fe0 pc=0x46f281
created by webp_server_go/encoder.ConvertFilter
	/home/Nova/workspace/webp-server/webp_server_go/encoder/encoder.go:71 +0x215

Some conclusions

From the above example, we can observe that in the case of an original image size of 384 × 512 px, if both height and width are passed in, then for the following Interesting variations, in order to maintain the aspect ratio of the image, only one dimension will be adjusted to match the passed-in parameter.

  • InterestingNone
  • InterestingAll

And other Interesting variations will crop the image to ensure that the output dimensions perfectly match the two specified dimensions passed in.

On the other hand, InterestingEntropy and InterestingAttention are more suitable for generating thumbnails. The former determines the cropping center based on the image entropy, while the latter attempts to find the focal point in the image as the cropping center.

Currently, WebP Server Go and WebP Cloud use InterestingAttention as the cropping method, which facilitates the generation of suitable thumbnails.

References

  1. https://www.libvips.org/API/current/libvips-conversion.html#VipsInteresting
  2. https://pkg.go.dev/github.com/davidbyttow/govips/v2/vips#InterestingAll

The WebP Cloud Services team is a small team of three individuals from Shanghai and Helsingborg. Since we are not funded and have no profit pressure, we remain committed to doing what we believe is right. We strive to do our best within the scope of our resources and capabilities. We also engage in various activities without affecting the services we provide to the public, and we continuously explore novel ideas in our products.

If you find this service interesting, feel free to log in to the WebP Cloud Dashboard to experience it. If you’re curious about other magical features it offers, take a look at our WebP Cloud Services Docs. We hope everyone enjoys using it!


Discuss on Hacker News