Looking back on "Building Better Controllers" 2 years later

Over 2 years ago, I started working on some ideas to build better Kubernetes controllers. In this post, I wanted to give a bit of a retrospective on how things have gone since then. Over the years working on Istio and other projects, I observed a number of major issues with controllers: Most code was about error-prone event handling and state reconciliation, rather than business logic. Most tests, in turn, were about the same. This, in turn, made the code extremely complex, brittle, and often incorrect. This complexity lead to user facing compromise: incorrectness and performance issues. You might argue I should just write a better controller that is faster and without bugs. Maybe, but probably not.