This is a follow up to my earlier article on unit testing focused on why unit testing leads to better internal design especially when developing complex software. Recently, I had a conversation with a friend of mine (who is not a software developer) about why I love being a programmer, how I develop software, and why I use automated unit testing and especially the practice of Test Driven Development (TDD) (first popularized by Kent Beck) in any work that I undertake. During this discussion I also shared my frustrations with him that despite TDD (and automated unit testing) becoming a proven and well-established practice within many organizations that aim to develop high quality software, most programmers that I run into (and others who work with programmers closely) are not always enthusiastic about adopting it. The two main reasons I hear are: one, they don’t see the value of it because they have a dedicated QA team or end users who will catch any defects during formal testing phase to happen near the end of the project, and second, it supposedly takes twice as long to develop software as you now have to account for the time to write test-related code. Proponents of TDD usually counter with reasons such as: tests act as specification/documentation, help create loosely coupled code by helping in isolating dependencies, help reduce time when enhancing the application by reducing regression testing times, etc. However, after almost 10 years of practicing TDD and writing automated tests, my belief is that its ability to provide the programmer with short feedback loops that help in decision making on the design or structure of the software is single-handedly the most important reason for me for using it although all the other reasons are also very important and beneficial.

The short and fast feedback loops that the practice of TDD provides then enables the programmer to make a continuous series of micro-adjustments to the design of the software sometimes until the very last line of code is written. This enables a programmer to write the software program in small steps one module/unit at a time while always ensuring that the boundaries of behavior for each of those units of code are verified and validated through the application of testing code that can later be run on demand very quickly even when these tests number in the hundreds if not in the thousands as the software grows larger and in complexity.

The legendary Chicago architect Louis Sullivan (also known as the “father of skyscrapers”) writing in 1896 said:

All things in nature have a shape, that is to say, a form, an outward semblance, that tells us what they are, that distinguishes them from ourselves and from each other. — Unfailingly in nature these shapes express the inner life, the native quality, of the animal, tree, bird, fish, that they present to us; they are so characteristic, so recognizable, that we say, simply, it is ‘natural’ it should be so.

Over the years, I have come to increasingly appreciate the deep insights of what this really means which is that the inner and outer style of anything should reflect its true purpose nothing more, and nothing less (or essentially “form follows function”). I am sure Louis was not thinking of computers or writing software programs when he wrote this, but he might well have been as his beliefs are only confirmed by the emergence of design principles such as Service-oriented architecture, SOLID principles, Micro-services, etc. that are becoming increasingly popular in the software industry these days. All these principles essentially stem from a core idea that any moderately complex system should really be decomposed into an ecosystem of collaborating units of code or “services” each of which is focused on one thing only, and these units should be contractually committed to play a specific role within a specific ecosystem using a formal specification to avoid any ambiguity. All these principles emerged from the recognition in the industry that writing software to fit in the highly collaborative, mission critical and varied environments of today is not a trivial task, and therefore new techniques and approaches were necessary. And writing software, especially in short timeframes, to conform to these stringent specifications requires many “micro-decisions” to be made both during initial design as well as subsequent development work. Accomplishing all this without fast feedback cycles that enables one to understand the impact of these decisions is somewhat akin to having blinds on when walking through rush hour traffic. One needs to see or feel the results of making decisions immediately in these situations, and TDD happens to be one such tool that provides you with great feedback loops when programming. For a successful software development project, other feedback loops are also necessary such as close interactions with the customer, doing wire frames/prototypes, frequent and iterative delivery of software, etc. Using these feedback tools and approaches makes the process of creating anything especially software go faster and smoother, and they ensure that you are building exactly what is being asked, and you are indeed headed in the right direction.

I have written my share of good and bad programs over the last 17 years using TDD and without TDD. And looking back now, I can safely say that programs I have written using TDD (in the last 10 years) have often been much more compact (and even dare to say elegant) than any other ones when I hadn’t begun to use this practice. This has been especially true of complex software that I have developed over the years that were required in the medical or the financial industry for example that need to run, or at least be available all the time. Often, the practice of TDD has resulted in the delivery of a program very different from what I conceptually envision it to be since there are often many paths and possibilities that I simply cannot not anticipate or see when I first begin designing or writing the program. This fact has only convinced me and shaped my thinking that the discipline of writing software is very much a creative process (so much for my long held belief that programmers including me were mostly left brained) even though the eventual outcome or product may still be considered purely a functional artifact, and may have no artistic relevance in many people’s eyes, and especially for those who are outside of this process. This fact in my opinion is also sometimes a cause of frustration for those who want to control the outcome of software project in a predictive manner.

To be able to try ideas as you think of them. If there is any delay in that feedback loop, between thinking of something and seeing it, and building on it, then there is this whole world of ideas which will just never be. ~ Bret Victor

When I began to investigate the area of creativity in general, I stumbled into other insights as well as evidence that short feedback cycles are extremely important to any creative process including computer programming, and therefore the tools and techniques programmers use also need to provide them with rapid feedback loops. Anyone who has read articles or watched videos by Bret Victor (designer and influencer of products such as phones and tablets from Apple) will note that he often makes the strong case that short feedback loops help develop and nurture ideas, and when a creator cannot see the immediate connection to what they are designing or developing, the ideas become shunted and fade away. Many in the profession of software development including me would completely agree with this sentiment. Time and time again, we have all either worked in or witnessed projects that go for months and sometimes years without delivering anything useful for end users, and when something does get released eventually, it is often not what the end users desire. This is either because the requirements were misunderstood, or the landscape of the business and technology has changed significantly that the software is already obsolete and considered a legacy application. No amount of requirements gathering and initial design can really validate what the users really want until the software (at least in some shape or form) is made available to them for use. So, feedback cycles are needed in every aspect of the software development cycle.

Although we have come a long way since the 60s when the field of computer science formally emerged, we still get very frustrated at times and wonder where the profession of software development is headed and what the tools, techniques and programming languages of the future (5-10 years from now) might look like. Although it is hard to predict the future, I am certain of one thing at least around programming. The software construction tools of the future will provide the programmer with shorter and faster feedback loops than they are able to do now, enabling us to visualize, design, decompose, and construct the software in ways even faster than we are able to now. These faster feedback loops provided by these tools will help the programmer to recalibrate the behavior of the program much more easily and much more often by providing enhanced tactile and graphical visualizations of the program behavior that are not (at least commercially) possible right now, and should enable the programmer or the team to make the right decisions at each step of the programming process as they progress towards their goal.

By the way, if you want to watch a video of Bret Victor explaining why feedback loops are very important during the creative process, you can watch it here. Happy 2016 and happy coding!!!