Johann Höchtl asked a few interesting questions on the Google Go mailing list regarding my experiences with using Go for a sizable project. Let me answer these questions in turn:

The Go package system

Quite simply, in my experience the Go packaging system is already mature enough. I ran into no troubles with it whatsoever. A few months ago the packaging system was slightly immature in that you could not have packages with names clashing with the Go packages, but this has been fixed for a long time now. And so, the packaging system is good to go for large projects, in my opinion.

What was a headache?

There are a couple of GO features that one needs to understand carefully, otherwise the bugs they produce can be time consuming to find.

The first one: If you implement your own io.Reader (or io.Writer) interface make sure it really truly behaves as the specification of io.Reader requires. In particular, you will notice that a Read() method is never supposed to return a negative byte count, even when an error has occurred. This is an easy mistake to make for those used to POSIX conventions. If you do make this mistake and use your Reader in conjunction with, say, bufio.Reader you will be faced with bizzarre behavior because bufio.Reader assumes proper behavior from your io.Reader and makes no error checking. I gather (from the Go Team) that this is an intended design decision (not to check for correct input/output argument behavior, or correct interface behavior). It makes sense and, of course, it does mean that in the future there should be an automated way of running your program in a “debug” mode where proper behavior is ensured at real time and bugs are caught. Or one might even have a static analysis tool.

The second: This code is flawed. Do you see why? The err initialized inside the blocks is not the same as the return parameter err. As a result, when you use return with no arguments, you are returning the wrong err (the one that is nil). I quickly learned to avoid this by not using argument-less return, however the bugs caused by this kind of code were even harder to catch than the ones given above.

What is missing from the Go language?

There’s nothing missing that was really annoying or disturbing. Once in a while I could have saved a few lines of code if there were generics. But such cases were neither common nor particularly crucial. I would not pressure the Go Team to have generics until they have a very good design in place.

Debugging and profiling without the tools

The main thing that you cannot debug with ease using Printf’s is catching deadlocks, because in a system as complicated as Tonika, they could be anywhere. For this I built a very handy profiler package (given below) that monitors time spend inside each lock and you can get a textual printout at any time and see which locks have not been released or have been held for too long. This package is a real treat, but I have not submitted it to the Go repository because I believe that the long term solution will be different (using a real profiler and debugger). But what I have was more than sufficient. The bottom line with this package is that anytime you use a mutex, you will define your variable as prof.Mutex, rather than sync.Mutex and nothing else changes. See the source code below:

  • Statistical functions for keeping running average and variance of time locks held: avgvar.go
  • Main profiler type: prof.go
  • Convenience type for profiling Mutex’s: mutex.go

Thanks for your questions!