How to keep the code coherent
There are two approaches to creating consistent programming code. For the same quality, one requires more skill and less time, while the other is the other way around.
Here I understand the word “coherent” in the sense of “forming a unified whole” as well as “logical and consistent” (from https://www.oxfordify.com/meaning/coherent). The easiest way to make the codebase coherent is to build it from a single root, avoiding duplication and repetition from the very beginning.
The power of coherence
For example, it is the usual goal of refactoring to change the code to a more compact form, where repeated code has been “factored out”. While the end result is the same, I’d like to argue for a more systematic approach, where such refactorings are needed much less frequently: When the code is coherent, it means, it is being born that way, from the beginning!
Natural phenomena are usually quite coherent, because our Universe was born in the Big Bang from a single singularity or created by one God (or both). Physicists routinely discover that coherency, so the idea I express here should sound quite natural to them.
Mathematics in it’s essence is coherence itself. There is simply no need to rediscover or rename what is, for example, a monoid or a group. This is where the power is. Once we understand that we deal with a certain mathematical structure (for example, by proving certain properties), a vast amount of results opens to us.
Haskell programmers (and Generic Programming gurus) can experience this first-hand. They do add typeclasses like “Monoid a =>” to their code. They are also aware that a monoid is a semigroup, because it can be seen from the definition “class Semigroup a =>”. Haskell is of course not a mainstream language, but it has tools for programming coherently, when one is comfortable mapping the problem domain at hand to the eternal mathematical abstractions. The example shows that the power of coherency is achievable. It is enough to tell the compiler the most suitable class (or generic programming concept) the entity we model conforms to, and some code just snatches into place after we add specific methods.
And the reward is generous: We have added a piece, which will fit in all places we need it to fit, no more and no less! We may have one case in our mind (for instance, concatenating one string with another), but when we spend a few minutes more on choosing our abstraction, we may be sure, that when our client comes with an extra feature (e.g. empty string), we already have that case covered. And when our client will ask us about some property (for example, whether the order of concatenation of three strings matters), we can answer easily and reliably, because we leverage mathematical theorems. There is also much smaller risk for a confused developer to change the implementation and break the contract. In the case of Haskell, the type system will intervene.
When we program all that in a programming language, in a do-it-yourself manner, the consistency is to be maintained manually. Of course, we can make Semigroup and Monoid types in any decent programming language, but a rare “normal” programmer would be (a) comfortable of matching problem domain with ultimate mathematical abstractions (b) that will pass the review of colleagues who will most certainty label this as overdesign, violations of KISS and YAGNI. It is so much easier to re-implement “monoid” twenty times than to recognize the fact, isn’t it?
And then the same people will tell you that strict static typing is the thing. Yes, it is, but I see no point in static typing unless the concept in the data type also contains all the relevant constraints. Roughly speaking, the degrees of freedom in the concept are adequate. In the case of a monoid, using the string data type as an illustration, we know that “string1 + string2” is not the same as “string2 + string1”.
And this is at the heart of the coherency: Rules for the entities of our solution are there by design, they are not maintained separately throughout the code, core abstractions make sure operations on those entities are correct, we only need to map those abstractions with specific types. More specific types and classes may of course have even stricter constraints (for example, we may want a Group, not just a Monoid), but the solution will remain coherent.
The above does not even depend that much on the programming paradigm one uses. Both functional and object-oriented programming can be used.
Back to Earth
I can hear more rumblings like “don’t follow DRY too strictly”, KISS, and YAGNI. However, this only reflects how software developers, especially those without computer science or math background, reinvent mathematical concepts as they gather experience over the years. Experienced developers still have that voice inside, telling them “this need that thing I did in the project X… let me recall… it had some associative operation, and then it needed some empty element… and then we can concatenate them”. Their brain is thus having a notion of a monoid we use as an example here, without realizing it. And it is but one example. There are more “patterns” like that stored in the software developers brains as they gather experience.
No, I do not insist every developer should remember what Monoid or Monad is, but it is good to know, that there is the source of “platonic realism”, where those things have their names — mathematics. For example, we can roughly talk about string type, when we mean monoid. String is of course loaded with all kinds of auxiliary methods, which belong to linguistics, so the analogy is noisy while the algebraic definition of monoid is exact.
There are people who advise not to spend time on factoring out commonalities at all. They advise us to code separately, because our code will get a lot of boolean flags to choose a more precise mode of operation. However, while it can be the case we have not recognized some exceptions or extra constraints at first, and adding a boolean flag or copy-pasting our implementation may be the fastest way forward, but is it wise to do?
Are those concepts really that different? Will adding more structure to reflect the difference work better than conditional processing? In any case coherent software is usually also highly composable, because consistency makes it much easier to connect code together not only at the interface level, but also a bit deeper (refer to https://en.wikipedia.org/wiki/Concept_(generic_programming) here). Combining two simpler things may be a better way to keep complexity low than making god-classes with methods full of flags and if-statements. It is of course easier to say than to do, especially for one not yet experienced enough.
What is it for humans in it?
Human reality is much less coherent, and that is why we give up so often to find a better root for everything we want our computer to do for us and our users. Layers of existing software skew our picture of the software world even more. Leaking abstractions are unavoidable. So how can we still write software coherently given all this heritage of our prior solutions and users’ mental models of them? Here the user’s mental model being aligned with that of software is understood as a prerequisite of any good user experience with possible exception of games.
There is no conflict with design methodologies like DDD (domain-driven design) here though. Software always has a stratified structure, even if we write our GUI application in assembler, there are still layers of CPU (micro)code, which ultimately get executed, so the problem-solution semantic border ought to be present at some level.
End user does not need to know that we use strings (which are instances of monoids) to express a person’s name, or that we use a graph to represent a workflow of some procedure. Unless we really want to educate power users, these implementation details can be hidden.
Once we match a problem domain concept with an adequate mathematical concept (workflow and graph in the example above), we can also easily leverage a rich set of graph theory algorithms or we can even jump into matrix representation of graphs when our solution requires certain optimizations.
Here some readers can remind me that “normal” developers very rarely encounter the need to implement highly efficient algorithms like solving optimization or operation research problems. We usually have some cross-related entities and accompanying metadata, and the hardest challenge may be optimizing misbehaving SQL queries from time to time. (I am not touching concurrency here.)
To the point
Developing software goes farther and farther from it’s mathematical roots, and in the name of practicality it is common to assume quality software requires extra efforts and experienced developers. What is probably not understood is that experienced developers are mathematicians, who rediscovered what has been already known for quite some time. There is also no incentive to learn more fundamental approach to creating software because “everyone can code”, “we do not need all that abstract garbage in our front-end development”, etc, etc. Educating oneself in abstraction-making, learning from fields where abstractions are used fluently is the only road to become the master quicker. This requires reflection on the ways one thinks. To benefit from “abstract ideas” one needs to learn to see the connections of those abstract concepts and the real world. I am pretty sure having these abilities would not harm anyone. The most obvious winners are those, who learned to reflect on the ways they think from the childhood. Abstract — concrete bridge can be discovered then, and become a very sharp tool by the age one goes to the university, because having the bridge makes studying at school much easier.
In addition
This article was almost ready when I understood one fundamental difference in programmers’ approaches to building software.
Imagine we have an equation and we want to find its roots. One mathematician can sit down and come up with an analytic expression. Another one will apply numerical methods to find the roots. Analytic solution may require much more experience, but it’s offering utmost accuracy, and is re-usable (inserted to another equation). Numerical methods are iterative and the solution found may have errors in it, which depend on how many iterations have been done. Each consequent insertion of the root lead to less accurate results.
The same happens in programming. Coherent solution is like an analytic expression we get for the problem domain. Numerical approach requires divergence. If the programmer fixes more bugs than adds — solution with required precision will emerge at some iteration. The only problem is the number of iterations can’t be large, so remaining bugs will most likely be quite insidious. Analytic solution does not suffer from that problem, because only a very small number of (numerical) examples is needed to be more or less confident in it. (I am simplifying here. If the development project involves stakeholders, who judge how near iterations are to their liking, we have a parameter identification problem from estimation theory.)
The point being many developers jump too fast into “numerical”, even for simple linear equations…
