About Adding Features (part 2)

This is the second part of the “About Adding Features” series of posts. The first post is here.

At some point, we are convinced that a new feature belongs to a library. It fits well, improves the ergonomics, and on top of that - doesn’t seem like that much work. The costs that immediately come to mind are the engineering time needed to implement the feature and the long-term maintenance burden. This is not the whole picture, though. There are many pitfalls one has to avoid when extending or changing a public interface.

Cognitive Load

Undeniably, the architecture of a library has the most significant impact on the user’s experience. It affects the organization of the public interface, how it needs to be used, and - perhaps most importantly - how programmers need to think about the problem they are trying to solve.

Explaining how to choose a suitable mental model is well beyond the scope of this post (and I don’t really have an unconquerable answer). Assuming it has been picked - we should strive to keep it as simple as possible. Even if we break DRY¹ here and there.

Complexity and Quantity

Would you rather fight 100 duck-sized horses or 1 horse-sized duck?² It’s a silly question, but an interesting one. It shows that it is not obvious to us how problems scale, neither with size nor quantity.

We can change the theme to coding and ask an analogous question: would you rather use 100 small interfaces or 1 big one? In many cases, the number 100 would be an exaggeration. One rarely has to deal with 100 interfaces to solve a problem. It is also not the case that - with the big interface - one has to deal with the whole thing at once. As usual, it seems that the answer is both “it depends” and “somewhere in between”.

To stay pragmatic, though, we can follow the rule of thumb: do what reduces complexity, or increases it the least. But how to apply it in practice? Take Go’s standard library package strings as an example - it offers many functions for inspecting or manipulating text. Amongst other functions, it includes:

func Contains(s, substr string) bool
func ContainsAny(s chars string) bool
func ContainsRune(s string, r rune) bool
func HasPrefix(s, prefix string) bool
func HasSuffix(s, suffix string) bool

func Trim(s, cutset string) string
func TrimFunc(s string, f func(rune) bool) string
func TrimLeft(s, cutset string) string
func TrimLeftFunc(s string, f func(rune) bool) string
func TrimPrefix(s, prefix string) string
func TrimRight(s, cutset string) string
func TrimRightFunc(s string, f func(rune) bool) string
func TrimSuffix(s, suffix string) string

Specialize or Parametrize?

To make the package smaller, one could instead come up with³:

func Contains(s string, r Rule) bool
func Trim(s string, r Rule) string

func Substring(s string) Rule
func CharSet(set string) Rule
func Func(f func(rune) bool) Rule
func FromLeft(r Rule) Rule
func FromRight(r Rule) Rule

Now, we replace:

• ContainsAny(s, substr) with Contains(s, CharSet(substr)),
• TrimLeft(s, cs) with Trim(s, FromLeft(CharSet(cs)),
• TrimPrefix(s, prefix) with Trim(s, FromLeft(Substring(prefix)),
• and so on.

As you can see, we have managed to cut the number of different functions almost in half (13 vs. 7). Is it really an improvement, though? We have increased the complexity of the mental model by introducing a new concept - a Rule. Now, every developer needs to understand what a Rule is within the strings package, and how it’s used. We also added Rule transformations, namely FromLeft and FromRight, in order to support TrimLeft and TrimRight (along with their Func variants).

YAGNI

One could argue that introducing the Rule interface opens up more possibilities. We can now trim log statements (e.g. "13-03-2021 15:35:21 foo/bar/baz.go:123: error during frombulating") by creating a new Rule that matches the timestamp and code path prefix. This saves us the cost of reimplementing the Trim part of the function ourselves!

In such cases, it is worth it to consider the YAGNI⁴ (You aren’t gonna need it) principle. If not taken to extremes (pun intended), it can be beneficial to determine whether additional complexity is warranted. Consider also what would be the cost of adding the thing you wanted now in the future. Would it be roughly the same as if you did it now? If yes, then postpone it.

There is usually little harm in adding an extra convenience method, e.g. adding Contains, even if Find could be used instead. It is still important to not introduce inconsistencies - e.g. if Find works with empty strings, Contains should too.

But what if we actually need to trim the log statements?

Utility Packages

In such a case, consider creating a utility package that works with the interface instead of extending it. For the sake of argument, let’s assume that Trim is heavily optimized under the hood - and we don’t want to reimplement it. We can implement the prefix matching logic and then simply use TrimPrefix to cut it out.

What if it turns out that this is not the only complex trimming operation that is commonly needed? Well, we can come up with a trimutil package that implements TrimPrefix(s string, m Match). Whoever needs the complexity of matching the string according to its format will easily understand what’s going on. Who doesn’t - won’t have to.

Not Enough Pieces

It is not always the case that we can build things on top of a package. Many libraries contain enough functionality to make it difficult to accommodate the wishes of all its potential users. Take the example of the Go HTTP library that provides a simple file server:

func FileServer(root FileSystem) Handler

It’s a fantastically simple interface that gets the job done. Under the hood, it handles the intricacies of the HTTP protocol, range requests, redirects, If-Modified-Since headers and so on. But what if we’d like to use it to serve our web application? It’s all well until the user asks for a non-existing file. They’ll be greeted with a rather ascetic "404 Not Found" plaintext message. This is a very different user experience from any well-polished service, where the error site has pretty graphics and offers ways to help the user reach where they need to be.

Is there a way to configure the FileServers error responses? No. Is there a way to build on top of it, keeping all the much-needed functionality that hides underneath? Yes, but it’d be pretty complicated⁵.

Consider the Problem

The library authors could’ve done the following instead:

1. create a new FileServer type with configurable error responses; or
2. add a parameter to the FileServer call that allows for providing its own error responses.

The options above are by no means an exhaustive list. Both would extend the http package (which is already pretty big by Go standards) and add some complexity to the implementation. If there was ever a discussion about doing this, they have decided against it.

One good argument supporting such a decision starts with a thought experiment: **who are the users of this API? **It is clear that this was meant to be a simple file server. There is no support for compression, generating index pages , and so on (like e.g. fasthttp does). You could safely say that it is not meant for advanced users - and that’s ok!

Always ask yourself who the users are and what problem they are trying to tackle. Look at it holistically - quite often, it will turn out that providing just one feature is still not enough to solve the task at hand. It can also lead you to a completely different path - and that’s ok too.

If you want to discuss this post or give me feedback (much appreciated), please use Twitter: https://twitter.com/kele_codes/status/1424303354449174529.

Notes