There are different ways of expressing the "build" process for our software projects, such as make, just, task, etc. But do we need them to do that? What if I tell you that you can express your build process with the language you are already using to write your code?
Today, I'll introduce a new Mage project that allows us to codify our build process with Go and gives us much flexibility. First, we'll make a demo to show you how to use Mage within your projects by providing a real-world example. Then, in this demo, we'll set up a release process that will release binaries and container images using two of my favorite projects, GoReleaser and ko, on the GitHub Actions platform.
If you're not familiar with ko or GoReleaser, don't worry, I suggest you read the blog posts I've written about ko and GoReleaser to help you get started with these technologies.
- Getting started with ko: A fast container image builder for your Go applications
- The fanciest way of releasing Go binaries with GoReleaser
As I mentioned in the beginning, there are different build tools, and the most popular one is make, but using make to assemble the build process is hard for most people. Moreover, it has many problems, such as that make is not cross-platform and will not work in the Windows environment. According to that problem, using Go to define a build process will fix that issue for free because Go is cross-platform. Even though Mage is an entirely new technology, of course, there is good news for make lovers that Mage has the same UX as make; also, it has the same philosophy as make which we will explain in the upcoming sections.
So what is Mage? Mage is a make/rake-like build tool using Go. You write plain-old go functions, and Mage automatically uses them as Makefile-like runnable targets. So, as we said, Mage has the philosophy of make, just, instead of using a bash script to define targets, it uses Go functions, and it has the same CLI experience, mage build instead of make build. So, just as the first step in using Make is to create a Makefile, the first step in using Mage is to generate a Magefile. But since in Mage, everything has to happen through code, this Magefile definition will usually be in the "magefile.go" file at the root of your repository.
There are different options to start using Mage, one of them is using its CLI called mage, and the other one is Zero Install Option that does not require installing mage CLI which is one we are going to stick within this blog post. Mage has excellent documentation; this is where we can find all about Mage. To install its CLI, please refer to the installation page. We can quickly create a template Magefile with the -init option.
mage -init
We may have any number of Magefiles in the same directory. Mage doesn't care what they've named aside from standard go filename rules. All they need is to have the mage build target. Or, we can create magefile.go within your project's root folder, and use mage.go file (but it can be named anything)) as "Zero Install Option" recommends. With that, now, we can go run mage.go , and it'll work just as if you ran mage . The only thing that you need to do is add mage.go file and fill its content with the following:
// +build ignore
package main
import (
"os"
"github.com/magefile/mage/mage"
)
func main() { os.Exit(mage.Main()) }
One good news about using Mage is that there are lots of ready-to-use helper utilities available via the community and Mage itself. So we don't need to discover the wheel again and again. In mage, there are three helper libraries bundled with the mage, which you can learn more about through this page. From the community side, magex will be the most vital option that you want to use, thanks to @carolynvs. Now, let's jump into the details by looking at the code you can find on GitHub.
//go:build mage
// +build mage
package main
import (
"errors"
"fmt"
"os"
"runtime"
"github.com/carolynvs/magex/pkg"
"github.com/carolynvs/magex/pkg/archive"
"github.com/carolynvs/magex/pkg/downloads"
"github.com/magefile/mage/sh"
"sigs.k8s.io/release-utils/mage"
)
// Default target to run when none is specified
// If not set, running mage will list available targets
var Default = BuildImagesLocal
// BuildImages build bom image using ko
func BuildImages() error {
fmt.Println("Building images with ko...")
if err := EnsureBinary("ko", "version", "0.11.2"); err != nil {
return err
}
ldFlags, _ := mage.GenerateLDFlags()
os.Setenv("LDFLAGS", ldFlags)
os.Setenv("KOCACHE", "/tmp/ko")
if os.Getenv("KO_DOCKER_REPO") == "" {
return errors.New("missing KO_DOCKER_REPO environment variable")
}
// echo "${{ github.token }}" | ./ko login ghcr.io --username "${{ github.actor }}" --password-stdin
_ = sh.RunV("ko", "login", os.Getenv("ghcr.io"), "-u", os.Getenv("GITHUB_ACTOR"), "-p", os.Getenv("GITHUB_TOKEN"))
return sh.RunV("ko", "build", "--bare",
"--platform=linux/amd64", "-t", "latest",
"-t", os.Getenv("GITHUB_REF_NAME"),
".")
}
// BuildImagesLocal build images locally and not push
func BuildImagesLocal() error {
fmt.Println("Building images with ko for local...")
if err := EnsureBinary("ko", "version", "0.11.2"); err != nil {
return err
}
ldFlags, _ := mage.GenerateLDFlags()
os.Setenv("LDFLAGS", ldFlags)
os.Setenv("KOCACHE", "/tmp/ko")
return sh.RunV("ko", "build", "--bare",
"--local", "--platform=linux/amd64",
".")
}
func Release() error {
fmt.Println("Releasing greeting with goreleaser...")
if err := EnsureBinary("goreleaser", "-v", "1.10.3"); err != nil {
return err
}
ldFlags, _ := mage.GenerateLDFlags()
os.Setenv("LDFLAGS", ldFlags)
args := []string{"release", "--rm-dist"}
return sh.RunV("goreleaser", args...)
}
func EnsureBinary(binary, cmd, version string) error {
fmt.Printf("Checking if `%s` version %s is installed\\n", binary, version)
found, err := pkg.IsCommandAvailable(binary, cmd, "")
if err != nil {
return err
}
if !found {
fmt.Printf("`%s` not found\\n", binary)
switch binary {
case "goreleaser":
return InstallGoReleaser(version)
case "ko":
return InstallKO(version)
}
}
fmt.Printf("`%s` is installed!\\n", binary)
return nil
}
func InstallKO(version string) error {
fmt.Println("Will install `ko`")
target := "ko"
if runtime.GOOS == "windows" {
target = "ko.exe"
}
opts := archive.DownloadArchiveOptions{
DownloadOptions: downloads.DownloadOptions{
UrlTemplate: "<https://github.com/google/ko/releases/download/v{{.VERSION}>}/ko_{{.VERSION}}_{{.GOOS}}_{{.GOARCH}}{{.EXT}}",
Name: "ko",
Version: version,
OsReplacement: map[string]string{
"darwin": "Darwin",
"linux": "Linux",
"windows": "Windows",
},
ArchReplacement: map[string]string{
"amd64": "x86_64",
},
},
ArchiveExtensions: map[string]string{
"linux": ".tar.gz",
"darwin": ".tar.gz",
"windows": ".tar.gz",
},
TargetFileTemplate: target,
}
return archive.DownloadToGopathBin(opts)
}
func InstallGoReleaser(version string) error {
fmt.Println("Will install `goreleaser` version `%s`", version)
target := "goreleaser"
opts := archive.DownloadArchiveOptions{
DownloadOptions: downloads.DownloadOptions{
// <https://github.com/goreleaser/goreleaser/releases/download/v1.10.3/goreleaser_Linux_arm64.tar.gz.sbom>
UrlTemplate: "<https://github.com/goreleaser/goreleaser/releases/download/v{{.VERSION}>}/goreleaser_{{.GOOS}}_{{.GOARCH}}{{.EXT}}",
Name: "goreleaser",
Version: version,
OsReplacement: map[string]string{
"darwin": "Darwin",
"linux": "Linux",
"windows": "Windows",
},
ArchReplacement: map[string]string{
"amd64": "x86_64",
},
},
ArchiveExtensions: map[string]string{
"linux": ".tar.gz",
"darwin": ".tar.gz",
"windows": ".tar.gz",
},
TargetFileTemplate: target,
}
return archive.DownloadToGopathBin(opts)
}
If you have installed mage CLI, you can run mage -l commands to see what targets you have defined within your magefile.go:
$ mage -l
Targets:
buildImages build bom image using ko
buildImagesLocal* build images locally and not push
ensureBinary
installGoReleaser
installKO
release
* default target
As you can see from the output above, we have six targets, and we marked the "buildImagesLocal" target as a default with the help of a variable called "Default".
I won't go into the details of how ko and GoReleaser work, but as I mentioned, if you want to learn more about them, please read the blog posts above. It is worth saying that if you look at the imports section, you will notice that we use a bunch of helper packages. I want to highlight one of the important ones named "sigs.k8s.io/release-utils/mage". This package includes a bunch of great functionalities. One of them is mage helper utilities and version command. Why am I telling you that the Kubernetes #sig-release team also uses mage a lot in their project's build process? Even better, they provide it as an external library through a project called kubernetes-sigs/release-utils. They also show how we can use this within a sample project in the kubernetes-sigs/bom project. Also, they provide a version command that can be used within our projects with many great features. The only thing we need to do is add this line below:
rootCmd.AddCommand(version.WithFont("doom"))
If you look at the file root.go under cmd package, you will notice that we use the "version" package of the same project. Once you have run the project's version command, you will see that it will give you a bunch of excellent information about the project because the version command provided by the "version" package fills out these Go ldflags for free.
$ docker container run --rm -ti ghcr.io/developer-guy/mage-go-example:latest version
_____ ______ _____ _____ _____ _____ _ _ _____
| __ \\| ___ \\| ___|| ___||_ _||_ _|| \\ | || __ \\
| | \\/| |_/ /| |__ | |__ | | | | | \\| || | \\/
| | __ | / | __| | __| | | | | | . ` || | __
| |_\\ \\| |\\ \\ | |___ | |___ | | _| |_ | |\\ || |_\\ \\
\\____/\\_| \\_|\\____/ \\____/ \\_/ \\___/ \\_| \\_/ \\____/
greeting: Will say hello
GitVersion: v0.1.0
GitCommit: 64252f7
GitTreeState: clean
BuildDate: 2022-08-21T11:27:25Z
GoVersion: go1.18.5
Compiler: gc
Platform: linux/amd64
Let's move on with the details of the release process we defined in a file release.yaml under .github/workflows directory.
name: Release
on:
push:
tags:
- v*
jobs:
build:
name: Build images with ko
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.18'
check-latest: true
cache: true
- name: Build
env:
KO_DOCKER_REPO: ghcr.io/${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
go run mage.go BuildImages
release:
name: Release binaries with GoReleaser
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.18'
check-latest: true
cache: true
- name: Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
go run mage.go Release
`
Please don't worry if you are not familiar with GitHub Actions platform, just focus on the go run parts because this is where we invite Mage into the play. As you can see from the file above, we call Mage's targets by simply running go run mage.go command. Although we use ko and GoReleaser binaries within the magefile, we prefer to install them through the "archive" package provided by the magex library. Still, there is an alternative way to install them. GoReleaser has a GitHub Action that can help install the GoReleaser binary into your environment. Also, Mage has GitHub Action, which can help you install and run mage commands.
That's all for me now, and I hope you like the blog post and the Mage itself. 🫶
Thanks for reading. 🙋🏻♂️