golang-pros-cons-devops-part-3-speed-lack-generics

This is our six-part series on Golang Pros and Cons for using Go in a DevOps development cycle. In this one, we discuss Golang’s runtime, compilation, and maintenance speed (the pros); and lack of generics (the) con.

Be sure to read up on the last post about “Interface Implementation and Public/Private Designations” if you missed it, or subscribe to our blog updates to be notified when the rest of the series is published. (We are doing these about every other week, but since we were busy launching our platform beta, we are admittedly a little behind.)

Golang Pro: Speed

I’ve broken the benefit of speed down into three different categories, based on what matters to us when writing the Blue Matador agent: (1) runtime, (2) compilation, and (3) maintenance. The amount of definitive data I have decreases with each passing category — you’ll see why.

Runtime Speed

Because our agent runs on customer servers, runtime speed was a top priority, along with security and auto-updates. Unfortunately for us, we didn’t determine this priority set until after our first stab at the agent, which was written in Python. Looking back, I’m glad we moved away from Python.

As a general runtime speed comparison, we scraped some numbers off of benchmarksgame.alioth.debian.org and put them into this nice chart. This is the number of seconds each language took to run a standard test. The longer it took, the higher the bar. Notice that Python is slow across the board, except for the pidigits test, where it failed to give accurate output. The rest of the languages are all in the same class.

golang-language-runtimes

Since this is a post about the relative speed merit of Golang, and not about why we moved from Python to Golang, I’ll remove the two languages that are consistently slower than Golang: Python3 and Node.js. Now with the cruft removed, you’ll see how Golang compares against the top competitors.

golang-language-runtime

It’s easy to tell in this graph that C++ is the fastest language — no surprise there. In fact, even Java beats Golang, but there’s two good reasons: (1) the Java Virtual Machine (JVM) has been under development since 1995 — 17 years longer than Golang; and 2) the JVM tests took 2 to 30 times as much memory as Golang — meaning the Golang garbage collector is, in general, working harder than the JVM one.

Compilation Speed

Our previous agent was written in Python (which isn’t a compiled language), but that doesn’t mean we’re strangers to the compile-time advantages of Go.

From the beginning of Golang, fast compile times have been a strict requirement. Go was created at Google by Ken Thompson and Rob Pike. Google, with over 2 billion lines of code, undoubtedly has serious issues related to time wasted compiling.

compiling

There’s some really good information posted here, which I’ll summarize (I do recommend reading their post as it’s pretty short but very detailed).

First, take this graph of compile time for a relatively large codebase. Details for the code can be found on the link previously mentioned. Notice how well Golang does compared to everything but Pascal. If you’re unfamiliar with Pascal, the language was designed to only need a single pass through the code — essentially the compiler is guaranteed to be O(n). Considering the Golang language spec is still usable, unlike Pascal, I’d consider the relatively narrow margin a victory.

golang-language-compilation-time

I don’t understand all of why the compilation times are so fast, but I do understand that at least a portion of it is due to the dependency management system. When a file is compiled, the compiler only looks at the immediate dependencies listed in the file. It doesn’t have to recursively load all the dependent files’ dependent files.

The Blue Matador agent has 29 packages, 116,824 lines of code. It has 3 target OSes, 2 target architectures, and 3 modules. All of this compiles in parallel (on an 8-core development laptop) in under 10 seconds. The good news is that we’re never really waiting on builds. The bad news is that we don’t have time to joust like in that XKCD comic.

Maintenance Speed

Now, the detour in murky waters. Before I dive in, let me make a couple points clear:

  1. Our agent Golang codebase is relatively small intentionally.
  2. Probably because of its size, there have only been 2 reported bugs.
  3. We currently only have 3 full-time developers, and only 2 of them are even touching agent code.

So, understand the incredible lack of evidence and experience when I say that maintaining our Golang code has been very simple and not at all time-consuming. It could be I’m wrong about this entirely. I’m sure I’ll find out when programmers down the line are cursing my name.

That said, here are the reasons why I think the code maintenance is better in Go:

  • No memory management. In C/C++, there’s a lot of code dedicated just to memory management. You end up doing wacky things to be able to manage memory. That’s not the case in Go because it’s garbage collected.
  • Robust core libraries. Except as required by the error-return system, our code was very slim. That’s because the core libraries have everything we needed, from HTTP calls and JSON encoding/decoding to forking and IPC pipes.
  • No generics. Yes, I’m about to name the lack of generics as a severe drawback to the language; however, without generics, types are always explicit and known. When you read a class file, you know exactly what to expect. This tends to make modifications easier and faster.

Additional Resource for Speed

Note: I found this after I wrote the post. I recommend reading it if you want to know more about the speed of Go: 5 Things That Make Go Fast.

Golang Con: Lack of Generics

Golang doesn’t have generics. I hate it. No doubt there are benefits, but I really hate it.

Imagine working in C++ without the standard template library, and no ability to create your own. Imagine the heartache of wanting to use data structures, but only having an array and a hash map at your disposal, knowing that if you create any other structure, you won’t be able to reuse it for anything else. Imagine all the benefits of a typed language, but the impossibility of reusing logic in strongly typed classes. It reminds me of a time when I created custom websites, and for all my ability to write code, the best I could do was copy-and-paste function.php to all my clients’ servers.

When it comes to writing bad code to bypass the lack of generics, I’ve either done or considered doing the following things to make Go work like I want it to.

Empty Interface

This solution requires the use of the empty interface. Variables of type interface{} can be absolutely anything. This is great for flexibility in typing. It’s terrible for different reasons.

If you use an empty interface as the underlying value in a data structure, then every time you interact with the data structure, you end up littering your code with type assertions, which should never return an error. Ignoring the potential error is a recipe for disaster though, since a simple refactor is no longer type safe.

If you use an empty interface as a parameter to a function, then you will likely have a type assertion switch statement in the function. Reusing the function with a different type means adding a new row to the switch statement. I’d rather have a lobotomy, because this is not really “reuse,” and it’s not scalable to many developers.

Copy/Paste

Our free Watchdog module monitors various system metrics — CPU, load, disk, network, etc. The first metric I wrote was CPU percentages. While writing these, I assigned a float32 type. Next up was number of processes running, clearly the job of a uint.

It’s here that I realized Go hated me personally.

I tried so many different ways getting the query and persistence layers to work with both a float32 and a uint. The last thing I wanted to do was copy/paste. Luckily, mine was a success story, because I didn’t copy/paste; I used suboptimal typing (up next). But to show how frustrating it was, I had copied/pasted all my code — all the logic, parsing, persistence, etc. — and was ready to commit. I just couldn’t do it.

The mere fact that the Go language spec had me ready to commit a copy/paste commit of complicated query and persistence code is a strong persuader to consider another language.

Suboptimal Typing

What I actually did, instead of copying and pasting, was to just use floats across the board. Yes, the Watchdog module uses floats to track number of running processes, number of disk writes, and number of swap ins/outs.

This method is largely benign for numerical types, which is why I ultimately used it for the query and persistence layers. There are far more cases where you can’t use this method than those in which you can. If you have numerical types, consider it. If you don’t, you’re back to empty interface and type assertions or copying/pasting.

At least it’s not as bad as storing a number, 0 to 100, that represents a percentage of CPU as a 4 byte int.

Unions

Just kidding. Go doesn’t support them, but they would have been really helpful. What do you think?

Every other week or so, we are posting a new guide like this in our six-part series on “Golang Pros & Cons for DevOps.” Next up: Date Parsing and Method Overloading.