The Challenges of Writing a Massive and Complex Go Application

The Challenges of Writing a Massive and Complex Go Application



Author: Ben Darnell

Abstract:

We opted for Go when building CockroachDB, a scale-out, relational database, because of its support for libraries, interfaces, and tooling. However, it has come with its own frustrations, often related to performance and synchronization. And as for Cgo, RocksDB, and other critical external libraries, we’ve had to hunt down or develop creative workarounds to ensure they work well the rest of the toolchain. In this talk, we’ll share how we’ve optimized our memory usage to mitigate issues related to garbage collection and improved our use of channels to avoid deadlocks. We will also share creative techniques to integrate non-Go dependencies into the Go build process.

ACM DL:
DOI:

11 thoughts on “The Challenges of Writing a Massive and Complex Go Application

  1. Curious what exactly the style was that helped alleviate bugs with having incorrectly assumed inner fields were mutable

  2. There used to be a joke that you could make FORTRAN programmers write in another language, but they would still write FORTRAN programs, just with different syntax. This sounds a lot like making C++ programmers write in Go, and they wrote C++ programs using Go syntax. Personally I think C++ syntax is horrible enough to make that sound like not a terribly silly thing to do.

  3. Thanks for the level of detail, and congrats on reaching 1.0 and production readiness.

    The Jepsen write-ups are at https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ and http://jepsen.io/analyses/cockroachdb-beta-20160829.

  4. I'm building a CNN using Go, and I was having the same issues that they were running into. Especially the memory issues with slices, and the deadlocking with goroutines.

  5. Why can't some technical people give proper talks? All those ahms and repeated words really detract from the talk.

  6. I haven't used CockroachDB, but I am anxious to try it out since I absolutely love Tornado, which Ben wrote.

Leave a Reply

Your email address will not be published. Required fields are marked *