Framework as a way to painless software development
Low viscosity, easily-maintainable systems are a dream of software developers. Here a generic design approach to such systems is described. Everyone experienced enough will recognize the pattern and start applying it.
The foundational ideas here are not my own. I’ve familiarized myself with them while trying to understand the essence of what a framework is some years ago. References are given at the end in Sources of Inspiration.
The ideas presented here are somewhat above the level of programming paradigms, meaning, that they are applicable in many of them. This does not prevent, of course, to apply the ideas also at the lower level, and maybe in a nested fashion.
The problem
Software systems usually change a lot during their lifetime. Bug-fixing, adding new features, removing features is a never-ending process. One of the desirable properties of the software is low viscosity. Here I understand viscosity as described in Cognitive dimensions of notations. Unfortunately, English Wikipedia article on viscosity itself is not helpful. Simplifying, viscosity relates with amount of efforts required to change a program expressed in some programming language. I believe every programmer has heard also about DRY (do not repeat yourself) principle, so reasoning on how many places in the program need to be touched and the pain related to it can be understood.
The question of this article is: How to organize the program or it’s parts to minimize the pain, inflicted by changes?
Analysis
Intuitively we understand that ultimately followed DRY principle should make it very easy: It is always just one place to make a change. However, DRY tells nothing about how exactly not to repeat yourself. A lot of techniques and patterns, SOLID principles, etc, are there to help, but experience matters.
One of the powerful techniques is something we usually do not think of first though, but we all know and use it: Frameworks. (Please, read on.)
At a very general level, a typical framework provides us with “holes” we fit our “pieces” into. Those holes and pieces can be of different shapes, but one property, which we usually do not think of much is that the “pieces” of the same “shape” are homogeneous!
Another observation everyone even minimally experienced can make is that our programs are usually composed of some blocks, and if we are accurate, the order of those blocks is not totally chaotic, but has some order. There is, for example, a layered architecture, where layers containing those blocks are “on top of each other”. Please note, that in particular situations those blocks can be anything from a (micro)service to a module to a class, you name it.
We can (visually) put one “row” of homogeneous “pieces” horizontally.
However, vertical structures are inevitable as well. It is well-known that new feature usually requires changes from UI down to the database, and possibly layers in-between. The same is true for other kinds of architectures, which may have inner-outer axis instead of top-bottom, and so on.
With a framework, we have a structure, which has “slots” where “blocks” can be inserted. In that structure a block is not in the vacuum: It gets its environment. Good frameworks achieve an optimal environment for the system to function by minimizing framework-slot interfaces.
Synthesis
Let’s imagine for simplicity (and as Gorbunov-Posadov and other authors he refers to suggest), that our system consists of “rows” of blocks stacked upon each other, and each feature uses some of those blocks, one (or zero) in each row.
We may need to throw away any preconceptions about how we build software so far and do not try to think of the blocks as modules, classes, etc: They can be anything, even lists of declarations!
Gorbunov-Posadov uses a text editor as an example. And in that example one row can be just a (declarative) description of the menu, where each “piece” is a menu item, and the row of blocks is just a list of objects (like in JSON). Scripts those buttons refer to can reside at a lower row.
As you may have already guessed, adding new features may require adding blocks to each row and form a vertical column. Some may even reuse underlying blocks. It may also be that a specific feature does not need a block from some row from below. It is fine.
Note, that the decomposition into a stack of rows is not equal to the architectural layers. Just think of some big table or sparse matrix if it helps. Architectural layer can have many rows of blocks.
Here we need to recall that the table (matrix) itself represents the variable parts of the solution. It’s elements are supposed to fit to the framework’s slots. Nowadays, Docker containers can be an approximate analogy here every software developer knows at least something about.
This description is so general, that it does not matter whether we have data in the blocks, code, classes, objects, templates, modules, database tables or even abstract data types. It can even be all of the above in the same solution.
Lets consider an extreme example. The core of the Semantic web application can make use of a framework, which contains a query builder and an inference engine. Our list of homogeneous blocks is just a list of potentially trillions of RDF triples. Adding a “new feature” is then as simple as adding new triples. It’s not of course the same as adding new lines of code because RDF triples are meaningfully interconnected and readily registered for the inference framework.
More moderate example is component architecture. This is probably the most accurate analogy for the painlessly modifiable system. Component architecture of course needs some framework to connect components to each other. Here also adding a feature is about adding (and registering!) new components, spanning different levels.
The list of blocks does not even need to be explicit, it can be distributed in the code (for example, localization strings can be automatically gathered by some tool into lists). The most important thing is, that the blocks can be easily accounted for, that is, located, added, referred to, removed. This is where frameworks differ. A lot. File system folders are used to group modules, standard directory layouts invented, gluing XML-specifications added, convention over configuration used… Many choices.
Show me the magic!
In a sense, the magic is really everywhere. Frameworks and component architectures are already widely used.
The idea of framework has been rediscovered a lot of times. And surely, it has been mostly found useful. So why do we still complain about software, which is hard to maintain?
The main suggestion here is to… make frameworks early, to make them in narrower domains, to pay attention to so-called hot-spots: Spots in which software grows or is most likely to grow. Are more menu items likely? Yes! Will there come more actions? Most likely! Can there be more types like these, representing the same concept? Sure! All those are potential hot-spots and thus candidates for homogeneous “lists of blocks” to be kept in the registries throughout the code base.
It’s of course easy to claim all such cases need to be captured, but hard to do in practice. In the beginning of the project our code base is like a universe at the time of Big Bang. We do not know for sure how it will develop, and trying to provide for everything is, well, over-engineering.
What can help though is always treating software development as building frameworks (even if you do it inside some other framework already), being attentive to the way system grows, noticing emerging candidates for collections of horizontal blocks, and refactoring early (having 2–4 emerged homogeneous blocks may already justify such a refactoring). Some subsystems can be treated as a framework from the very beginning. For example, accounting for API routes, localization strings, collections of assets, templates, widgets may be good to start from the very beginning.
It does not always require a trained eye to see new components as belonging to a set of similar ones. Even an act of copy-pasting (performed by beginners) can be a good hint. Developing skills of abstraction is still needed though to see how two or more blocks can be unified to work with one slot of the framework. Abstraction skills are needed for programming anyway, so this is hardly an obstacle. Note, that it’s not important to get rid of all the boilerplate code right away: It’s more important to recognize the homogeneity of the blocks under refactoring and register them. Reducing redundancy can be done later, and can be done with more confidence, because we already identified the slots blocks should fit into.
Special words of caution here about the magical environment some frameworks provide, making everything available to every slot. At first it feels good to use services of the framework in the block, but as the system grows the bloated slot interfaces can backfire. Same motivation probably led to development of mini-frameworks, microservices, microkernels.
For a very different framework, consider Erlang programming language and especially it’s OTP, as a brilliant example of finding homogeneous blocks (Erlang’s processes) and placing them in OTP’s “slots”. OTP captures several typical communication patterns, so code of the “blocks” can be to a large extent freed from concurrency programming concerns.
Is that all?
One of the side-products of “framework-oriented” design practice is that it is easy to factor out some declarative configurations, which can be easily provided to advanced non-programming users. In other words, when software is being deliberately and systematically built with identifying and isolating homogeneous blocks, unification of those blocks may produce declarative configurations — one step away from“zero-coding” solutions. (For the attentive reader, here I defy “Prevent over-configurability” clean code rule as it rarely helped in practice.)
Needless to say also, that for some parts of the system third-party frameworks can be used. For example, it rarely makes sense to write your own web-form library. Choose one instead. One of the criteria of the library can be how well it supports adding new blocks (in this example — new input widgets).
Takeaway
- Many successful software systems today are frameworks over one or more kinds of homogeneous blocks
- Consider software to be built as a framework from the very start
- Make it clear for everyone on the team how horizontal lists of blocks and feature verticals are built
- Stick with the definite architecture (top-bottom, outer-inner)
- Refactor (unify) emergent homogeneous blocks (together with your framework) to achieve slot-block match
- Minimize slot interface and dependency of blocks on environment
- Use the presence of “framework-oriented” design as a quality criteria for choosing third-party software
- Consider the whole table of the framework, not only horizontals or only verticals
- Frameworks can be nested
Sources of inspiration
Articles of the Russian computer scientist M.M. Gorbunov-Posadov listed here: https://www.keldysh.ru/softness/gorbunov/public.htm
