Go Testing Toolbox

A conglomeration of testing tools, from Autotest to Vagrant.

Published on September 28, 2013.

Gopher tools
Toolbox sans box — this is not whac-a-gopher.

Let’s explore some of the tools available for testing our Go code.

Unit Testing

Go has testing baked in. Below we contrast the standard testing package with Gustavo Niemeyer’s popular gocheck library:

// with testing
if !reflect.DeepEqual(value, 42) {
    t.Errorf("Give us the ultimate question then")
}

// with gocheck
c.Check(value, DeepEquals, 42, Commentf("Give us the ultimate question then"))

Gocheck one-liners are a little more concise, and we don’t lose out in extensibility. For example, gocheck doesn’t ship with a delta checker for floats. I wanted something that looked like this:

c.Assert(gear.GearInches(), Within, 0.01, 137.1)

So I wrote a custom checker in less than 20 lines of code. Arguably, I could’ve written a Within() helper function in 3. So you win some, you lose some.

There are ways to write concise tests without pulling in an xUnit assertion frameworks. If you’ve ever seen Cucumber data tables, you will appreciate table driven tests, a simple way of testing many variations with relatively compact code.

Alternatives

Gocheck isn’t the only third-party option out there. Andrea Fazzi’s PrettyTest has a colourful test runner, and it is compatible with gocheck assertions.

Go has anonymous functions, so it was only a matter of time before someone built a Jasmine-style BDD library. There are several options, including Mao/Zen, Goblin, GoConvey and Ginkgo/GΩmega. Unfortunately I haven’t had a chance to try these yet, so I can’t vouch for them.

No doubt there are other tools that have slipped past my radar. Lately I’ve been reaching for the standard testing package, mostly to avoid yet-another-dependency in reusable libraries. Go 1.2’s ability to download test dependencies with go get -t may change that for me.

Mocking

Mocking, stubbing, fakes, test doubles… it can be a bit confusing.

A summer ago, I was learning OpenGL, and wanted a way to stub out the graphics context in tests. I looked at gomock, but I wasn’t a huge fan of the code generation required, and I didn’t actually need expectations.

So I looked for a light weight alternative. The solution was to use interfaces in the production code, which allowed me to write a test double that satisfied said interface. That worked pretty well, and is the approach I continue to use today.

If mocking with an interface doesn’t fit, Karl Seguin has an different approach, by redefining functions in tests. Gomock has continued to improve and may fit your situation, and there is a newcomer to the mocking scene, withmock.

Then there are specialized “mocking” tools. For testing HTTP client libraries, the standard library ships with httptest. Will Norris explains how it works in Testing in go-github.

Coverage

Test coverage allows you to see which methods and conditional branches your test suite doesn’t reach.

Andrew Gerrand and Rob Pike are developing a new coverage tool for Go 1.2. To try it out, you’ll need the Go 1.2 release candidate or install Go from source. After you go get it, a lovely report in your browser is as simple as:

go test -coverprofile=c.out -covermode=count
go tool cover -html=c.out

If the new cover tool doesn’t suit your fancy/needs, take a look at Andrew Wilkins’ gocov. Combine it with drone.io for CI to be able to hook it up to Coveralls. See goveralls for details.

Having played with both, I much prefer the new cover tools UI to Coveralls. It’s snappy and easy to navigate.

Continuous Integration

If you’re a fan of Jenkins, take a look at go2xunit, otherwise you may like one of these services. They all support Go and are free for open source.

Note: If you previously read my Releasing Open Source article, most of this section will be familiar.

GoCI

GoCI is an open source continuous integration service by Jeff Wendling.

While more rudimentary than the other offerings, it is the only CI service focussed exclusively on Go. I’m curious what it could become if the Go community chips in.

drone.io

drone.io is super quick to setup through the web interface, and it supports GitHub, BitBucket and Google Code. I’m impressed.

For those private projects, the pricing is quite competitive, and it so happens to be written in Go too. The default build commands for Go may need a little tweaking.

Travis CI

Travis CI has been around longer than most, and has some nice features. Though it only integrates with GitHub, Travis will let you know if pull requests are safe to merge. It also allows testing against older versions of Go or Go tip.

AppVeyor

AppVeyor is a CI service for .NET developers. I approached them over Twitter with an interest in testing Go libraries on Windows. They added the option to use a PowerShell script for the build phase, installed Go on their servers, and have since officially announced Go language support.

Here is the source code for a sample application with the necessary build script.

wercker

wercker looks to be the most flexible offering, though I found the whole thing a bit overwhelming. It allows the community to build custom boxes for any language or environment required, and includes support for Go.

They blog frequently on Go deployment to Heroku and App Engine, code coverage, and more. Wercker is currently a free beta, but they plan to keep it free for open source.

Cross-Platform Testing

Before contributing to fsnotify, I wanted a way to test my changes across the adapters for kqueue (BSD), inotify (Linux) and ReadDirectoryChanges (Windows). My development machine is running OS X, which currently uses the kqueue adapter (though support for FSEvents is in the works). How would I test the others?

Vagrant

Vagrant is a tool to script the creation of VirtualBox VMs (as well as VMWare and other providers).

When you run vagrant up with the Vagrantfile I wrote, it will download Linux and BSD boxes if necessary, install the DVCS tools and Go binaries, and configure GOPATH and other environment variables.

The way I have this setup, my src/ folder is shared, so I can continue to use my editor and tools on OS X, while being able to issue commands to Linux and BSD VMs:

vagrant ssh linux -c 'cd nathany/looper; go test ./...'

Each VM has its own bin/ folder so go install won’t stomp on the binaries created by your host OS or the other VM.

If you would like to try it out, there are more details in the Vagrant Gopher repository. I welcome contributions, but be warned, you’ll be dealing with Bash script embedded in Ruby. Lotsa fun!

Windows

I’m not aware of a way to work with the Windows command line through Vagrant, so I had to set all this up manually.

You can download Windows Virtual Machines from Microsoft for free, but I went with a proper Windows 7 Professional license to avoid the constant “Genuine Windows” nagging.

Using VMWare Fusion, I shared my project folder which contains src, bin, and pkg. It becomes Z:\project on Windows.

Fortunately, this doesn’t cause conflicts, because binaries on Windows have an .exe extension and Go puts static libraries under pkg\windows_amd64.

With the Go binaries installed, I can use PowerShell to run my tests.

Autotest

Many people like running tests directly in their editor or IDE. I prefer to have a separate Terminal window for running tests.

This past February I wrote an autotest tool at a hackathon. Looper uses fsnotify to watch your file system. When you hit save it runs the tests for that package (folder).

Looper still has a long ways to go, but even at the start it had some nice features:

  • Recursive watching, when you add a subfolder it will start watching it.
  • It has a prompt that uses readline to manage history.
  • ANSI colours indicate pass/fail.

I’m already thinking about automatically running tests across multiple VMs using Vagrant, and maybe adding one-off commands for shortcuts, like generating a coverage report. My goal isn’t to replace Bash though, so there is much to consider.

The Looper README maintains a list of alternative tools for all your autotest and file watching needs.

PASS

Well, I hope you found this article helpful. If it was a FAIL, please let me know how I can improve it.

Comment on Hacker News, reddit, or the Go Nuts mailing list.

Related Articles:

Nathan Youngman

Software Developer and Author