这篇文章有简体中文版本，在： libvips, CGO 与 purego——如何让 Go 应用跨平台编译运行

One of the most notable features of Go is its support for convenient cross-system, cross-architecture cross-compilation. Imagine the following scenario:

You have a friend who needs to process a certain number of files every day according to certain rules. They want you to help write a program so that they can just run this program and relax.

Your friend uses Windows, while you use macOS.

Implementing the business logic is not particularly difficult, but the problem is how to make your friend use your code. You can’t ask them to install an interpreter or compiler. And if you use Node.js, you might encounter the following issue:

Go doesn’t have this problem. Ideally, we can easily compile binaries for other platforms and architectures. We don’t need to install cross-compilation toolchains, and more importantly, our applications are statically linked, which means they can run without any dependencies.

Whether you are a beginner programmer or someone who downloads games from a website, you must have encountered a situation like this:

For Go, the weird issue of missing runtime libraries basically doesn’t exist. We can directly compile the corresponding program:

# Compile the Windows exe file
$ GOOS=windows GOARCH=amd64 go build .
$ file awesomeProject.exe
awesomeProject.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows

# Now let's try compiling for Linux, both amd64 and arm64
$ GOOS=linux GOARCH=amd64 go build .
$ file awesomeProject    
awesomeProject: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=tSnBibyCQLPF4pPSyw4d/yFku1u5LTJGphtQfeoc5/EUJDOUquWUDl0G4NOyj2/x8FBO6wejoBCzyF-5sLs, with debug_info, not stripped

$ GOOS=linux GOARCH=arm64 go build .
$ file awesomeProject
awesomeProject: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=JQJBN7vXIJN-OfAptA1U/6uWHhmAeqKPmmO-Sc7h0/gYsHDj13pvNe2sJukdkO/jUkmn9RgZnyNYBs-UBFj, with debug_info, not stripped

# My quirky friend who uses FreeBSD but doesn't know how to program
$ GOOS=freebsd GOARCH=arm64 go build .
$ file awesomeProject
awesomeProject: ELF 64-bit LSB executable, ARM aarch64, version 1 (FreeBSD), statically linked, for FreeBSD 12.3, FreeBSD-style, Go BuildID=SEQmlkCSluwqrdkxTbI2/ej7yuXNnh9AvTQOLuJ8a/cvumRw18eyZLqbVYYlw4/qHSwivX3bgKMoj_JYuXh, with debug_info, not stripped

However, is cross-compilation really that simple with Go?

Implementation in Go

Applications implemented purely in Go can undoubtedly be statically linked. We need to explicitly disable CGO and add some LDFLAGS.

The Go standard library, such as net, uses CGO in some cases, which defaults to dynamic linking. Using the following magical code, we can achieve static linking:

CGO_ENABLED=0 go build -a -ldflags '-s -w -extldflags "-static"' main.go

CGO

Implementing a complex system from scratch can often be a tedious task. For example, to implement WebP encoding functionality, it is much easier to use the existing C library libwebp rather than writing the encoder from scratch. Just like the glue language Python can call C/C++ code, the behavior of calling C from Go code is called CGO: Calling C from Go code.

Suppose someone has already written code calc.c to calculate the volume of a sphere. It uses stdio.h for standard output and math.h to calculate powers and provide the value of pi.

#include <stdio.h>
#include <math.h>

float volume(float radius)
{
    float volume = (4.0f / 3.0f) * M_PI * pow(radius, 3);
    printf("radius: %f, volume: %f\\n", radius, volume);
    return volume;
}

To call this from Go, you just need to import "C" and include. Note that import "C" must be written on the line after include. Here, I also included time.h to generate a runtime timestamp.

package main

import "fmt"

/*
#include "calc.c"
#include <time.h>
*/
import "C"

func main() {
	timestamp := C.time(nil)
	fmt.Println("timestamp from time.h", timestamp)

	res := C.volume(C.float(3.9))
	fmt.Println("Result: ", res)
}

Now, if you want to cross-compile, whether it’s across platforms or architectures, it won’t work. This is because we are calling C code from Go.

The error message can be quite confusing, as main.go is right there, but it tells you there are no corresponding files:

# Try to compile binary for Linux
$ GOOS=linux go build main.go
go: no Go source files

# Try Darwin amd64
$ GOARCH=amd64 go build main.go
go: no Go source files

# Disable CGO explicitly
$ CGO_ENABLED=0 GOOS=linux go build main.go
go: no Go source files

$ CGO_ENABLED=0 go build main.go          
go: no Go source files

# Build binary for the current platform and architecture
$ go build main.go
$ file main
main: Mach-O 64-bit executable arm64

💡 C/C++ can also be statically linked, but that’s beyond the scope of this discussion.

`

C Standard Library and CGO

Most operating systems implement the C standard library, such as glibc for GNU/Linux and musl for Alpine Linux. You can use the ldd command to check.

# Alpine Linux
~ # ldd --version
musl libc (aarch64)
Version 1.2.4
Dynamic Program Loader
Usage: /lib/ld-musl-aarch64.so.1 [options] [--] pathname

# Ubuntu
root@b97e444b77d8:~# ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

When using CGO, part of the functionality of your application is dynamically linked to the corresponding operating system’s C standard library.

# Install gcc and related tools
~ # apk install alpine-sdk

# Compile the program
~ # go build main.go

# You can see it dynamically links to musl
~ # file main
main: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, Go BuildID=EcVShcMJ4BadtKOwCXEA/PmcoroS9BkTvUtAwCGc8/hgBrbqcis0htwacoIGeK/cP45vit8HqiQHniKYo82, with debug_info, not stripped

~ # ldd main
	/lib/ld-musl-aarch64.so.1 (0xffffb9bdd000)
	libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0xffffb9bdd000)

# On Debian, we need to use -lm to link with math.h
root@f3a02e3c52f2:~# export CGO_LDFLAGS=-lm 
root@f3a02e3c52f2:~# go build main.go

# Here you can see it dynamically links to glibc
root@f3a02e3c52f2:~# file main
main: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=e6e82e9f418298d8d833beaea799fc13b6e5e9fe, for GNU/Linux 3.7.0, with debug_info, not stripped

# Run the program
root@f3a02e3c52f2:~# ./main
timestamp from time.h 1688153330
radius: 3.900000, volume: 248.474869
Result:  248.47487

# Check the program's linkage information
root@f3a02e3c52f2:~# ldd main
	linux-vdso.so.1 (0x0000ffff98b69000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff98a90000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff988e0000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffff98b2c000)

# Check the glibc version
root@f3a02e3c52f2:/go# ldd --version
ldd (Debian GLIBC 2.36-9) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

Now, if you copy the main binary to a different Linux system that uses a different standard library implementation, what will happen?

Undoubtedly, if you give a binary linked with musl to a system using glibc, you will get a strange error message: ./main: No such file or directory. Even though the main file is there, the error is actually saying that the system cannot find /lib/ld-musl, not that the main file doesn’t exist. This is one of the quirks of open-source tractors.

root@4035da59313e:~# cat /etc/issue
Ubuntu 22.04.2 LTS \n \l

root@4035da59313e:~# file main
main: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, Go BuildID=EcVShcMJ4BadtKOwCXEA/PmcoroS9BkTvUtAwCGc8/hgBrbqcis0htwacoIGeK/cP45vit8HqiQHniKYo82, with debug_info, not stripped

# The main file is there, but it says it cannot be found
root@4035da59313e:~# ./main
bash: ./main: No such file or directory

root@4035da59313e:~# ls
calc.c	go.mod	go.sum	main  main.go

What if you copy it to an operating system with the same standard library implementation, such as the widely used glibc in most Linux distributions?

Generally, this depends on the compatibility of glibc. In general, higher versions are backward compatible, but lower versions may not be forward compatible. This means that a file compiled with a lower version of glibc will likely work with a higher version, but a file compiled with a higher version may not work with a lower version of glibc.

For example, you can try running a file compiled with Debian 12 on Debian 11, as shown below, and you will quickly see that it fails:

root@5354fbc3d13b:~# cat /etc/issue
Debian GNU/Linux 11 \\n \\l

root@5354fbc3d13b:~# ./main
./main: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./main)
./main: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./main)

# Unable to run because Debian 11 has glibc version 2.31 while Debian 12 has a newer version
root@5354fbc3d13b:~# ldd --version
ldd (Debian GLIBC 2.31-13+deb11u6) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper

However, if you compile the binary file on Debian 10, it can be used on versions 10, 11, 12, and newer.

Therefore, if you use CGO and want the compiled program to be usable in as many places as possible, it’s advisable to compile it with a lower version of glibc, such as CentOS 7, as long as it doesn’t cause any other negative impacts.

Here, I want to praise musl, which has a very stable ABI. Regardless of the version, a file compiled with musl can be used without any issues, whether it’s Alpine 3.14 or 3.6!

purego

CGO can be both loved and hated! Is there a way to call C code without using CGO? Absolutely! You can use the project ebitengine/purego to achieve that.

The basic idea is as follows: we locate the corresponding shared object (so) file for macOS or dynamic-link library (dll) file for Windows, dynamically load it at runtime, register the functions, and then we can call those functions!

Here’s an example that calls the time function from the standard library and our calc.c function. Since this library doesn’t seem to support float32, we’ll make a slight modification to calc.c and change float to double, which corresponds to Go’s float64.

#include <stdio.h>
#include <math.h>

double volume(double radius)
{
    double volume = (4.0f / 3.0f) * M_PI * pow(radius, 3);
    printf("radius: %f, volume: %f\n", radius, volume);
    return volume;
}

Then we use purego:

package main

import (
	"fmt"
	"github.com/ebitengine/purego"
	"runtime"
)

func getSystemLibrary() string {
	switch runtime.GOOS {
	case "darwin":
		return "/usr/lib/libSystem.B.dylib"
	case "linux":
		return "libc.so.6"
	default:
		panic(fmt.Errorf("GOOS=%s is not supported", runtime.GOOS))
	}
}

func getCustomLibrary() string {
	switch runtime.GOOS {
	case "darwin":
		return "calc.dylib"
	case "linux":
		return "./calc.so" // Note the "./" here, otherwise symbolic link to /lib is required
	default:
		panic(fmt.Errorf("GOOS=%s is not supported", runtime.GOOS))
	}
}

func main() {
	libc, err := purego.Dlopen(getSystemLibrary(), purego.RTLD_NOW|purego.RTLD_GLOBAL)
	if err != nil {
		panic(err)
	}

	var timeFunc func() int64
	purego.RegisterLibFunc(&timeFunc, libc, "time")
	x := timeFunc()
	fmt.Println("timeFunc from purego: ", x)

	calc, err := purego.Dlopen(getCustomLibrary(), purego.RTLD_NOW|purego.RTLD_GLOBAL)
	if err != nil {
		panic(err)
	}
	var volumeFunc func(float64) float64
	purego.RegisterLibFunc(&volumeFunc, calc, "volume")
	res := volumeFunc(5.3)

	fmt.Println("volumeFunc: ", res)

}

• getSystemLibrary retrieves the file name of the system’s C standard library.
• purego.Dlopen opens our standard library.
• var timeFunc func() int64 defines a function timeFunc with no arguments and returns an int64.
• purego.RegisterLibFunc(&timeFunc, libc, "time") registers the Go function to the time function in libc, which corresponds to the function in time.h.

Once the registration is complete, we can directly call timeFunc.

For our custom function, we need to compile it into a dynamic-link library first:

gcc calc.c -shared -o calc.dylib

Then, specify the path to this library file in purego.Dlopen:

$ CGO_ENABLED=0 go build main.go

$ file main
main: Mach-O 64-bit executable arm64

$ ./main
timeFunc from purego:  1688157968
radius: 5.300000, volume: 623.614538
volumeFunc:  623.6145379031443

You can even cross-compile and target different platforms and architectures:

$ CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build main.go && file main
main: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, Go BuildID=fE2oBt7SuA3vQ1l6grl3/TMG44vS0AQgizY8N0dgU/fzsb2LmHEV5MewkNePWZ/gZcBLiazVGwhZnCddgc-, with debug_info, not stripped

$ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build main.go && file main
main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=INDTVasUMzUfX4e4TOdd/f0SK_QqErc6rczYl33mA/blCz4OvC5JrgxwccisHF/HKPRj5tDEcSB7MLxefYY, with debug_info, not stripped

$ CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build main.go && file main
main: Mach-O 64-bit executable x86_64

However, please note that your target system should also have calc.so, and in GNU/Linux, don’t forget to link with -lm:

root@50a91fb229b9:~# gcc calc.c -shared -lm -o calc.so

root@50a91fb229b9:~# file main
main: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, Go BuildID=yaDDwT9MZyytBRl8Hgsn/TJAksLImgI7B37_oJor7/RDT3WAei40Odjx7-HbU5/xw7CqO3m_sjSQvcFlX13, with debug_info, not stripped

root@50a91fb229b9:~# ./main
timeFunc from purego:  1688158724
radius: 5.400000, volume: 659.583680
volumeFunc:  659.5836804636092

How is this related to the WebP Server Go project.

In the WebP Server Team (also known as the WebP Cloud Services Team), our goal is not only to optimize images but also to make it easier for users to run our application. We provide a way for users to quickly experience our application by directly downloading the release files from GitHub.

Previously, we used libaom and libwebp, which worked well. However, we discovered that govips had higher performance, and libvips supports multiple image conversion formats. So we switched to libvips as the underlying conversion library. However, things became more complicated after switching to govips.

• govips depends on libvips version 8.10 or above, which is relatively new.
• We want to support both amd64 and arm64 architectures because most users use the amd64 architecture, while we use the ARM64 architecture in the production environment. You can refer to our previous blog post, “Hetzner CAX Series ARM64 Server Performance Review and WebP Cloud Services Practice” (https://blog.webp.se/hetzner-arm64-zh/).
• Using govips means using CGO, and using CGO introduces glibc compatibility issues.

Using purego

Purego is an excellent project that supports cross-compilation without CGO. It only requires referencing the corresponding shared object (so) files and registering functions. However, as we can see from the practical operation mentioned above, using purego is very cumbersome, especially considering that we depend on govips. It would be extremely difficult to change this library to purego mode.

Building with an older version of glibc

Yes, we can build a binary file on an older operating system, but users still cannot use this file directly. They need to install libvips 8.10+ manually before they can use it.

CentOS 7 and libvips

CentOS 7 was released in 2014 and is now close to 10 years old. It is still in the maintenance phase and is suitable for building purposes. Remi maintains a libvips library, but unfortunately, it does not support arm64. Therefore, we need to find a way to build libvips ourselves.

What do we need? We only need a usable, minimal libvips that does not support png, jpg, or webp. In this CentOS, we just need to compile our binary with the help of libvips.so. As for the functionality of the compiled binary, it depends on the version of libvips installed by the user at runtime.

Preparing to build libvips

We need to install some common dependencies, such as gcc.

bashCopy codeyum install -y epel-release wget
yum groupinstall -y 'development tools'
yum install -y glib2-devel expat-devel

Downloading and compiling libvips code

wget <https://github.com/libvips/libvips/releases/download/v8.10.0/vips-8.10.0.tar.gz>
tar xf vips-8.10.0.tar.gz && rm -f vips-8.10.0.tar.gz

cd vips-8.10.0
./configure --prefix=/usr --libdir=/usr/lib64

make && make install
vips --version

The prefix and libdir parameters are essential. CGO will look for the corresponding so files in specific directories defined by ldconfig. In CentOS, this directory is /lib64, and /lib64 is symlinked to /usr/lib64.

We have compiled a minimal libvips! This libvips cannot even read jpg, but it is sufficient.

Installing Go and compiling

Download and install Go from the official Go website and then:

export CGO_CFLAGS="-std=gnu99"

git clone <https://github.com/webp-sh/webp_server_go>
cd webp_server_go
go build webp-server.go

The compiled file can now be distributed to users, as long as they have installed libvips correctly.

If your system is too old…

If your system is too old and does not have the required version of libvips, nor any third-party PPA, then you have to do it yourself.

Compiling libvips yourself is not very difficult. Remember that libvips requires libwebp version 0.6.0 or above, but the version provided in the CentOS 7 source is 0.3.0, which is very old. Therefore, we need to compile libwebp first.

# Also need to install libraries related to jpg and png
yum install -y glib2-devel expat-devel libgif-devel giflib-devel libpng-devel libtiff-devel libjpeg-devel libexif-devel

# Download and extract
mkdir libwebp && cd libwebp
wget <https://chromium.googlesource.com/webm/libwebp/+archive/refs/heads/0.6.0.tar.gz>
tar xf 0.6.0.tar.gz && rm -f 0.6.0.tar.gz

# Configure and compile
./autogen.sh
./configure --prefix=/usr/ --libdir=/usr/lib64 --enable-libwebpdemux --enable-libwebpmux

# Compile and install
make && make install

• prefix and libdir are the same as before.
• To make libvips support webp, we need to install demux and mux.

Why the hassle?

After going through all this, it seems that we have learned about CGO logic and built a bunch of programs, but it seems that we have gained nothing else. Purego is almost unusable. Although the binary file compiled with a lower version of glibc can solve glibc-related issues encountered when running on different systems, users still need to manually install the required version of libvips to successfully convert images.

Not to mention using LDFLAGS to statically link libvips, which is not a practical solution.

As a result, we can see that switching WebP Server Go’s encode library from the previous chai2010/webp library to libvips has brought performance improvements but also introduced some runtime environment issues. In light of this, we recently updated the README of https://github.com/webp-sh/webp_server_go to encourage users to use our pre-built Docker containers to run WebP Server Go (while still retaining the option to run the binary directly).

Furthermore, if you find Docker cumbersome or if your website is a static site that does not require runtime, why not try our WebP Cloud Services?

We are currently exploring a new business model called WebP Cloud under WebP Cloud Services, which is a SaaS version of our open-source component, WebP Server Go. Users only need to log in with GitHub and provide the source site address to obtain a new address with WebP conversion, CDN, and caching features. For example, the address of a 100KB image https://blog.webp.se/hetzner-arm64/c1-board.jpg becomes a WebP version with only 60KB https://p2k7zwb.webp.ee/hetzner-arm64/c1-board.jpg (with almost no loss in image quality).

https://blog.webp.se/hetzner-arm64/c1-board.jpg	https://p2k7zwb.webp.ee/hetzner-arm64/c1-board.jpg
107.81 KB	64.52 KB

In addition, you can also generate thumbnail images directly by passing the ?width= parameter.

https://blog.webp.se/hetzner-arm64/c1-board.jpg	https://p2k7zwb.webp.ee/hetzner-arm64/c1-board.jpg?width=200
107.81 KB	7.27 KB

If you find this service interesting, feel free to log in to the WebP Cloud Dashboard to experience it. If you’re curious about its features, you can check out our documentation at WebP Cloud Services Docs.

References

• CentOS 7 image containing libvips: webpsh/libvips
• webpsh/libvips build code: https://github.com/webp-sh/libvips
• Purego: https://github.com/ebitengine/purego
• WebP Cloud Services: https://webp.se/
• Statically Linking Go in 2022: https://mt165.co.uk/blog/static-link-go/