This is the second part of the “About Adding Features” series of posts. The first post is here.
At some point, we are convinced that a new feature belongs to a library. It fits well, improves the ergonomics, and on top of that - doesn’t seem like that much work. The costs that immediately come to mind are the engineering time needed to implement the feature and the long-term maintenance burden. This is not the whole picture, though. There are many pitfalls one has to avoid when extending or changing a public interface.
Undeniably, the architecture of a library has the most significant impact on the user’s experience. It affects the organization of the public interface, how it needs to be used, and - perhaps most importantly - how programmers need to think about the problem they are trying to solve.
Explaining how to choose a suitable mental model is well beyond the scope of this post (and I don’t really have an unconquerable answer). Assuming it has been picked - we should strive to keep it as simple as possible. Even if we break DRY1 here and there.
Complexity and Quantity
Would you rather fight 100 duck-sized horses or 1 horse-sized duck?2 It’s a silly question, but an interesting one. It shows that it is not obvious to us how problems scale, neither with size nor quantity.
We can change the theme to coding and ask an analogous question: would you rather use 100 small interfaces or 1 big one? In many cases, the number 100 would be an exaggeration. One rarely has to deal with 100 interfaces to solve a problem. It is also not the case that - with the big interface - one has to deal with the whole thing at once. As usual, it seems that the answer is both “it depends” and “somewhere in between”.
To stay pragmatic, though, we can follow the rule of thumb: do what reduces
complexity, or increases it the least. But how to apply it in practice? Take
Go’s standard library package
strings as an example - it offers many
functions for inspecting or manipulating text. Amongst other functions, it
func Contains(s, substr string) bool func ContainsAny(s chars string) bool func ContainsRune(s string, r rune) bool func HasPrefix(s, prefix string) bool func HasSuffix(s, suffix string) bool func Trim(s, cutset string) string func TrimFunc(s string, f func(rune) bool) string func TrimLeft(s, cutset string) string func TrimLeftFunc(s string, f func(rune) bool) string func TrimPrefix(s, prefix string) string func TrimRight(s, cutset string) string func TrimRightFunc(s string, f func(rune) bool) string func TrimSuffix(s, suffix string) string
Specialize or Parametrize?
To make the package smaller, one could instead come up with3:
func Contains(s string, r Rule) bool func Trim(s string, r Rule) string func Substring(s string) Rule func CharSet(set string) Rule func Func(f func(rune) bool) Rule func FromLeft(r Rule) Rule func FromRight(r Rule) Rule
Now, we replace:
- and so on.
As you can see, we have managed to cut the number of different functions almost
in half (13 vs. 7). Is it really an improvement, though? We have increased the
complexity of the mental model by introducing a new concept - a
every developer needs to understand what a
Rule is within the
package, and how it’s used. We also added
Rule transformations, namely
FromRight, in order to support
(along with their
One could argue that introducing the
Rule interface opens up more
possibilities. We can now trim log statements (e.g.
"13-03-2021 15:35:21 foo/bar/baz.go:123:
error during frombulating") by creating a new
that matches the timestamp and code path prefix. This saves us the cost of
Trim part of the function ourselves!
In such cases, it is worth it to consider the YAGNI4 (You aren’t gonna need it) principle. If not taken to extremes (pun intended), it can be beneficial to determine whether additional complexity is warranted. Consider also what would be the cost of adding the thing you wanted now in the future. Would it be roughly the same as if you did it now? If yes, then postpone it.
There is usually little harm in adding an extra convenience method, e.g. adding
Contains, even if
Find could be used instead. It is still important to not
introduce inconsistencies - e.g. if
Find works with empty strings,
But what if we actually need to trim the log statements?
In such a case, consider creating a utility package that works with the
interface instead of extending it. For the sake of argument, let’s assume that
Trim is heavily optimized under the hood - and we don’t want to
reimplement it. We can implement the prefix matching logic and then simply use
TrimPrefix to cut it out.
What if it turns out that this is not the only complex trimming operation that
is commonly needed? Well, we can come up with a
trimutil package that
TrimPrefix(s string, m Match). Whoever needs the complexity of
matching the string according to its format will easily understand what’s going
on. Who doesn’t - won’t have to.
Not Enough Pieces
It is not always the case that we can build things on top of a package. Many libraries contain enough functionality to make it difficult to accommodate the wishes of all its potential users. Take the example of the Go HTTP library that provides a simple file server:
func FileServer(root FileSystem) Handler
It’s a fantastically simple interface that gets the job done. Under the hood,
it handles the intricacies of the HTTP protocol, range requests, redirects,
If-Modified-Since headers and so on. But what if we’d like to use it to serve
our web application? It’s all well until the user asks for a non-existing file.
They’ll be greeted with a rather ascetic
"404 Not Found" plaintext message.
This is a very different user experience from any well-polished service, where
the error site has pretty graphics and offers ways to help the user reach where
they need to be.
Is there a way to configure the
FileServers error responses? No. Is there a
way to build on top of it, keeping all the much-needed functionality that hides
underneath? Yes, but it’d be pretty complicated5.
Consider the Problem
The library authors could’ve done the following instead:
- create a new
FileServertype with configurable error responses; or
- add a parameter to the
FileServercall that allows for providing its own error responses.
The options above are by no means an exhaustive list. Both would extend the
http package (which is already pretty big by Go standards) and add some
complexity to the implementation. If there was ever a discussion about doing
this, they have decided against it.
One good argument supporting such a decision starts with a thought experiment: **who are the users of this API? **It is clear that this was meant to be a simple file server. There is no support for compression, generating index pages , and so on (like e.g. fasthttp does). You could safely say that it is not meant for advanced users - and that’s ok!
Always ask yourself who the users are and what problem they are trying to tackle. Look at it holistically - quite often, it will turn out that providing just one feature is still not enough to solve the task at hand. It can also lead you to a completely different path - and that’s ok too.
If you want to discuss this post or give me feedback (much appreciated), please use Twitter: https://twitter.com/kele_codes/status/1424303354449174529.
Don’t Repeat Yourself, see also The Wrong Abstraction by Sandi Metz ↩︎
The Atlantic pondered what would President Obama do in such a situation. ↩︎
A bit more effort, and we’re going to reinvent regular expressions. ↩︎
You’d need to intercept the calls to
ResponseWriterand listen for errors. After that, anything else that’s written would need to be ignored. Feels like a hack. ↩︎