Software Architecture: Making Legos

In practice, not being a startup means that I need to create the software equivalent of legos. Most people seem to understand this analogy as creating reusable bricks that I can snap together in order to de-duplicate my work. While this is correct, I also need to create a standardization system similar to what legos create as an ecosystem. In other words, I need to come up with a system of things that snaps together as cleanly and elegantly as legos do. As you can imagine, the latter is by far the harder and more ambitious part of the work.

Of course, many people know the software engineers’ secret: we write only a very small proportion of our code ourselves. By that I mean that that the vast majority of us rely upon the plethora of existing libraries and modules of code written by others. With these libraries we can avoid reinventing the wheel at every turn—without them building even a simple project could easily take years, decades, or even a lifetime.

Given the existence of these libraries, one might be tempted to declare that software legos already exist—after all, isn’t each third-party library, module, or service a lego block? Are we not already snapping these together?

Yes, and no. The genius of legos is in the simplicity of how they fit together. I could buy three very different lego sets and still expect that if I cannibalized them, the pieces would fit together nicely and I could create something different and new to my liking.

As is, slapping the various available libraries and modules together is more akin to randomly grabbing bricks of several different lego competitors and lookalikes and then haphazardly building them into the larger shape I desire. The endeavor might require a bit of extra frustration here, a bit of duct tape there, and even the occasional super glue to bind particularly stubborn and dissimilar blocks. For all that effort, at the end of the day I get something less than ideal—not the smooth and easy connections of a well-designed lego set, but rather a Frankenstein that looks close enough to what I was going for if I squint hard enough and swallow my pride. Worse, many components of this monster are fused together so I cannot unsnap the blocks to change them or reassemble them into something else.

The closest thing we have to pre-designed lego systems would be frameworks like Ruby on Rails, Django, or NextJS. Each of these has a community and an ecosystem of lego blocks built to play nicely with them. I’ve spent a good amount of my career working with frameworks, even reaching for or guiding clients towards them when one wasn’t available.

Frameworks are great! They’re especially great if you want to get started quickly, mostly agree with the underlying philosophy of the framework you’ve chosen (or aren’t experienced yet to have your own opinions), and are working on something of small to medium scope. A lot of things come for free with a framework, the most important of which is a modicum of architecture informing a mental model of what goes where and how things should fit together.

It could be argued that what I’m creating for myself are the bones of a personalized framework. For the sake of argument, let’s ignore the nuances and assume that this is true. Why not just use an existing framework then? Put simply, frameworks have their weaknesses, too. I’ve reached a point in my career where I have strong opinions along with the knowledge, experience, and confidence to back them up. I want the space to explore, express, and develop my own software philosophy. (This is also the next obvious frontier for growth in my career as a software engineer.) I want the flexibility to tailor something exactly to my needs instead of bowing to what any particular framework thinks is right.

I’m not abandoning frameworks, per se, and I certainly have no interest in reinventing the wheel. Rather, I’m defining a way to write code that is, as much as possible, framework agnostic and platform independent. When I’m done, I should be able to snap together a few lego blocks to define all or part of a new product, and I should be able to plug that into a framework if needed. (Case in point: Midana uses NestJS on the backend and everything I’m creating now is designed to be plugged into NestJS rather than supersede it.)

This way I play by my own rules, and then find a way to adapt that to a framework rather than play by a framework’s rules from the start. The difference is subtle, but it gives me the flexibility to switch frameworks or abandon frameworks altogether at any point in the future for any reason. It also allows me to build on assumptions defined by my own rules that might otherwise be out of the scope of what any particular framework or loosely associated set of blocks aims to accomplish.

Outside of the lego system (read: pseudo-framework) itself, I also generally won’t be building my own lego blocks. Blocks already exist for foundational things like authentication (user accounts), permissions, notifications, storage, and in-app purchases. I want to leverage, not duplicate, this effort. My challenge is to take each of these things and distill their essence so I can create a lego that encompasses those ideas with a simple and elegant shape compatible with my lego system.

For example, there might be five existing blocks for authentication. All five blocks solve the same  problem (or at least have a high degree of overlap), meaning any of the five would fulfill the same  purpose within my product: allow users to register and login. Each authentication block is made of a different material (implementation), though, and each has a slightly different shape (API).

If I understand authentication deeply enough, and glean the essence of the blocks well, I can discern commonalities between the shapes of these blocks. Then I can mold a standardized shape that feels right to me, plays nicely with my lego system, and encompasses the shapes of other blocks well enough that any of those blocks (and any future blocks) could be adapted into the shape I want without too much extra work.

Doing this allows me to avoid duct tape or super glue, and it affords me the flexibility to switch authentication blocks at any point for any reason without having to change how other parts of my lego system snap together. Maybe the block I’m using has become expensive, and I could increase profit by switching to something else. Or maybe I discover the block I’m using has deal-breaking bugs or limitations I hadn’t anticipated. (Side note: this has happened while working on Midana once or twice now, and is a natural risk of experimenting with relatively young technologies or community-maintained open source code.) Or maybe I just want to evaluate a new authentication block for a new product, but I still want everything else to snap easily like it did before, and I want to be able to fall back to something more familiar if I hate it.

To realize this vision, I need to refine my own software philosophy enough to invent an elegant and cohesive lego system. Then I need to distill each foundational problem (authentication, purchases, notifications, etc.) into a standardized mold. Finally, I need to create adapters to massage incompatible blocks solving these problems into the right shapes. Most of the hard work is conceptual, thinking through the merits of various designs and capturing the complexity of an entire ecosystem of existing blocks behind relatively simple shapes. Writing the code itself won’t be too hard, getting the design right will be.

By the time I’m done, I’ll have a lego system and a small, but growing lego set powering a wide variety of rich applications across mobile, web, and desktop. I’ll spend 2-3 times longer creating each block, but I should spend a fairly minimal amount of time revisiting blocks in the future. It will take longer to finish the first product, and I’ll occasionally need to detour to create new blocks, but time to market for each subsequent product should decrease precipitously. Built this way, I should be able to de-duplicate much of the work to maintain a full fleet of products even if working alone. Each new product pays forward for the next product, allowing me to recycle the time investment for failed products. If all else fails, I can use this technology for greenfield consulting projects in order to deliver higher quality work at a much faster pace.

I recognize that it’s all very ambitious, but these days I often find myself so excited about the vision, that I just can’t wait to get to work on it. That, more than anything, signals to me that I’m headed in the right direction.