Michael Feathers

Unconditional Programming

Control structures have been around nearly as long as programming but it's hard for me to see them as more than an annoyance.  Over and over again, I find that better code has fewer if-statements, fewer switches, and fewer loops.  Often this happens because developers are using languages with better abstractions.  They aren't consciously trying to avoid control structures but they do. 

If we are working in an object-oriented language, we can replace switch-statements with polymorphism.  The same trick works well for if-statements too, but it can be overkill in simple cases. When we use languages with functional features, we can do most of the work that we do in loops using maps, filters, and folds. Control structures end up disappearing, and that can be a good thing.

The problem with control structures is that they often make it easy to modify code in bad ways.  Let's take a simple if-statement:

  if ...
    ...
  else
    ...
  end 

Every place that we have ellipses in that code is a place where we can put more code. Those places can access variables outside of the if. It's very easy to introduce coupling.  Moreover, people do routinely nest conditionals inside of conditionals.  Some of the worst code I've ever seen is a cavernous nightmare of nested conditions with odd bits of work interspersed within them. I suppose that the real problem with control structures is that they are often mixed with the work. I'm sure there's some way that we can see this as a form of single responsibility violation.

What can we do?  Do we have to live with control structures?  In general, I think that we do, but it's an interesting exercise to see what we can do to reduce our use of them. Often we can learn new tricks and make our code clearer in the process.

A while ago, I was working on some Ruby code and I needed to write a 'take' function to take elements from the beginning of an array.  Ruby already has a take function on Enumerable, but I needed to special behavior.  If the number of elements I needed was larger than the number of elements in the array, I needed to pad the remaining space in the resulting array with zeros.

This seems like a job for a simple if-statement:

  def padded_take ary, n
    if n <= ary.length
      ary.take(n)
    else
      ary + [0] * (n - ary.length)
    end
  end

Let's look at this code carefully.  There's nothing in it which tells us anything about what padding is, and how the pad relates to the array we're making. We can see what is happening if we look close, but we don't see the concepts in the code.

We could introduce some functions to make it clearer, and simplify the conditional by using a guard clause:

  def padded_take ary, n
    return ary.take(n) unless needs_padding?(ary, n)
    ary + pad(ary, n)
  end

That's short and sweet but it doesn't take advantage of a simple fact - we can use a null object to get rid of a conditional.  An empty array is a wonderful null object. Let's start over.

We don't need a conditional to compute the length of the pad.  The length is number of elements we want to take minus the length of the array, if the number we want to take is greater than the array length - it's the maximum of the difference and zero:

  pad_length = [0, n - ary.length].max

If we have that, we can pad the array first and then take the number of elements we need from it:

  def pad ary, n
    pad_length = [0, n - ary.length].max
    ary + [0] * pad_length
  end

At this point, we can write our padded take:

  def padded_take ary, n
    pad(ary, n).take(n)
  end

What we've done is eliminate an if-statement by forming a computation that always appends a pad. However, sometimes that pad is just an empty array.

I'm not going to argue that this code is simpler than the if-then-else code we started with but it is more declarative and, in general, I don't think that code using this strategy is as prone to abuse.

There's a clear advantage to being able to think at this level of abstraction. It pays dividends when we confront larger problems.

The Single Responsibility and Open/Closed Principle are the Same

Consider a class C and a set of potential feature changes F'.  If class C has two responsibilities, it is likely that many F in F' will only affect code for one of those responsibilities in C.

We apply an F to C but, since there are two responsibilities in C it has less closure - it was open to F and to feature changes that applied to the other responsibility.

Now consider separating responsibilities by breaking C into C1 and C2.  If F only applies to C1, C2 is closed with respect to F.

Smaller, focused abstractions lead to closure.

Flipping Assumptions with 'Programmer Anarchy'

I ran into a blog about Fred George's Programmer Anarchy concept the other day and it felt a bit weird.  By some quirk of fate, I've seen Fred give a talk on the topic three times and it made a strong impression on me a few years ago, but I think that my takeaways were very different from those outlined in the blog.

The first takeaway I had is that it is a thing that works.  I've visited the company where Fred was working when this concept was developed and, sure enough, they are making money.  This is not science fiction.  The second takeaway is more subtle.  I'll get to it in a little while.

For people who haven't seen Fred give his presentation, I'll just outline a few of the things that stand out for me in the barest form of the process. Teams, being anarchic, of course, can add anything need to get the job done:

1. There is no automated testing.  Code is put into production very rapidly in a monitored environment, A/B tested to a wider group and left if it is found working.
2. Code consists of micro-services that can be written in a heterogenous set of languages.  They are small: 10 - 100 lines, and they are expected to make money and then die quickly as the organization goes on to make money in other ways.
3. Programmers work rapidly with non-programmers in an egalitarian environment, trying things out in a market and seeing what works.
4. There's no need for management. It all just works.

It sounds idyllic.  I know that I would like to work in an environment that free of constraints, particularly at the technical level.  As  "the legacy code" guy, I've dreamt for years about what we would need to do to carry less code around in the industry.  Yes, code makes money, but it can be a drain.  There's a distance between what is easy and what is hard with regard to new features as a code base grows. Things that lead to that distance are often a mystery and organizations are left with persistent burdens that they didn't anticipate. 

If code can just be good enough to make money and then die, in short order, it makes software development much easier.

What does it take?  Well, I think that the key thing is a radically unstable domain. If a team creating an ERP system attempted Programmer Anarchy it would find that the code persists.  What's useful today is useful tomorrow and while that's good by most measures (you are able to make money from existing code), it's also bad because it is the first step toward a legacy trap.  Yes, we can do artificial things like rewrite code periodically whether it needs it or not, but frankly the economics are muddy.  It's hard to make the case for that. 

To me, this leaves me with the idea that the piece of Programmer Anarchy that I've described is really a tight fit for a particular type of problem space, and that gets me to my final takeaway about the concept. When we are presented with something like this that works, we can either bemoan the fact that we don't have the same problem, or we can do something more radical - we can try to find way to make money with a process like this.

It's backward from the way that most people think about software and business. Usually, we go from business idea to software, but it can be valuable to flip it and say "we have a way of working that is very streamlined - how can we make money with it?"

Avoid Null Checks by Replacing Finders with Tellers

One of my pet peeves in programming is null checks.  Many codebases are littered with them and, as a result they are often very hard to understand.

It's easy enough to complain about null checks, but it's harder to root out all of the places they occur and find alternatives.  The typical advice is to move your code toward using the null object pattern or to just check for null immediately and make sure that you don't pass nulls along in your program.  After all, we do often need to retrieve objects and sometimes they aren't there. 

Let's re-examine that.

Do we really need to retrieve or find objects in our programs?

The other day, I saw this example on StackOverflow:

    Person person = dataSource.getPersonById(personId);
    if (person != null) {
        person.setPhoneNumber(phoneNumber);
        dataSource.updatePerson(person);
    }

This looks like a typical case where we need a null check.  We attempt to find a person from a dataSource and we could get a null return value.  We could change the code to return a null object, a Person object that simply does nothing and that could solve the problem.  The code would look exactly the same except that it would be missing the null check.  The setPhoneNumber function would simply do nothing, and the updatePerson function would have the good sense not to add the null object to the database.

Yes, would could do that, but it is a bit of work. There is an alternative that we can use if we get ourselves out of the mindset of asking for objects.

Consider this bit of Ruby code:

    data_source.person(id) do |person|
      person.phone_number = phone_number
      data_source.update_person person
    end

The person method accepts an id for a person and a block.  If the person is found the block is called with the person.  Otherwise it isn't. Elegant, eh?  And, there are no nulls.

Lots of luck doing this in a language without blocks or lambdas.  I think that when the history of computing is finally written, one of the chapters will be about how much insanity thrived in the era before blocks were considered mainstream.  In fact, I think we can argue that null checks became common precisely because most languages haven't had blocks until recently.

Imagine doing this in Java without lambdas:

    dataSource.person(id, new Action<Person>() {
        public void act(Person person) {
            person.setPhoneNumber(phoneNumber);
            dataSource.updatePerson(person);
        }
    });

My eyes bleed. My soul too. It's no wonder why null-littered code is common in today's Java.

This teller method pattern may seem like a special case, but it isn't.  One of the core ideas in object-orientation is that it is better to tell than to ask. When we tell someone to give someone else a thing, it either does or it doesn't. There's no need to check for an error.

So, whenever you receive null from an API you've written, don't complain.  After all, you asked for it. Consider telling the result to someone instead.

 

Guiding Software Development with Design Challenges

There's quite a bit of bad software design in the industry and that's surprising.  After all, there hasn't been any lack of guidance about design over the years. From the earliest days of programming, there have been people who've shared their experience with creating software that is easy to change over time. As a result, we have an incredible number of concepts that we can fall back upon - Coupling and Cohesion, the Single Responsibility Principle, the Law of Demeter, the Dependency Inversion Principle, Postel's Law, RESTful-ness.  The list is nearly endless.  Surely, we should have good design by now. If only it were that simple.

The fact of the matter is that many organizations do excel at design.  Not as many as I wish, but there are quite a few.  But, the teams I find most interesting are the ones that miss the mark by being overly allegiant to design guidance.  There's always that team that takes some piece of advice too seriously and ends up creating a mess.  A classic example is over-doing the Dependency Inversion Principle (DIP).  DIP tells us that it is better to depend on abstractions than it is to depend on implementation-esque entities. Some people, in statically typed OO languages, reduce it to  sound-bite and write code where every single concrete class has an associated interface. You can't really fault them on following directions, but when you look at the resulting mess, it's obvious that they lack judgment.  Or, that they don't feel the wiggle room that is necessary to apply judgment.  Recently, I've arrived at the idea that we can fix that. 

Most pieces of design guidance are presented in a context free style.  Essentially, they are stated as ideals - "a good class in OO should look like this."  What would happen if we focused on the problem we're trying to solve rather than the solution?

Imagine that you are working with someone and you see what would classically be called a Single Responsibility Principle violation. It's obvious to you that the class you are working on does several things, and the reason that you can see that is because you have wide experience with software.  The person you are working with may not yet have that sort of tuned design sense.  What can you do?

Well one thing that you can do is launch into a discussion of Single Responsibility and work through some examples.  Your hope at the end of the process is that they've absorbed the idea and, having seen some examples, can apply the concept in new situations. Maybe they can, maybe they can't, but there is a more direct way of approaching the problem.  We can simply say "if we were going change this class to do <some new thing> what would we have to change?"  Notice that this is completely open-ended.  You have a conversation, and in the process you deal with context. You may see that there is no need to split the class, or that in this particular situation you can, but it's not worth it.  In any case, you are dealing with something real - a particular piece of code and thoughts about how it has and will change.

It turns out that nearly every design principle or piece of design guidance can be framed as a set of challenges to your code.  To approach Liskov Substitution you can ask the question "will a caller be surprised if we replace every object of type A with an object of type B?"  We can also approach bigger issues of Open/Closed Principle violations with hypothetical features - "Suppose the business changes in this way and we need to add this feature. What classes will change?"  Again, it is all grounded.

I see the use of Design Challenges as a mental hack that is analogous to the move from Test-Driven Development to Behaviour-Driven Development. It's a re-placement of emphasis. I encourage you to try it out. The starting point is to learn how to take things you know about good design and frame them as questions.

Scalars as Implicit Collections - Removing an Edge

Every once in a while I get irritated by the edges in programming languages.  One notorious edge is the absence or presence of a method on an object.  The method is either there or it isn't and if you guess wrong you have a program that simply doesn't run.  

In some programming languages, there is a way around this.  You can hook into the runtime to generate a method on an object when it is called but doesn't yet exist.  Of course, that leaves us with the decision of what method to generate. And, I suspect that some of you reading this are wondering now whether this is some sort of rabbit hole - what problem am I trying to solve?  In most cases, the fact that a method doesn't exist is important. We shouldn't gloss over it. But, sometimes we can gain advantage when we do.

Consider finder methods. You have some criterion you want to use to find an object. Maybe some unique id.  You call your finder and you have to deal with possibility that the thing you are looking for doesn't exist.  Most of the time this means that you are obligated to use a null check in your language, but that's messy and people can easily forget to do it. Some functional programming languages get around this by using a Maybe or Option type, but there's another way to punt.  You can make your finder always return a collection. If the thing you are looking for is not found, you receive an empty collection. "Okay," I can hear you say, "that is just passing the buck. You have to check later."  The fact of the matter is that you may not have to. You could use a map to perform the same operation on each of the elements, provided there is no chance of having more than one.

    finder(criterion).map(&:run)

This makes me wonder about something.  What if we were were able to treat values as collections when we are programming?

Consider this in a dynamic language.

    x = 4

When we type that, we expect that x[0] will yield an error, and we expect that x[1][59][3] would yield an error also.  But what if they didn't?  What if each of those references returned 4. What if values could be treated as infinite collections of themselves in programming?  Could that be a creative way of erasing an edge? 

Consider this in a Haskell-ish language.

    map (+1) [1,2,3]

We know that would yield [2,3,4]

Okay, how about this?

    map (+1) 3

Couldn't that evaluate to 4?

With an operation like map, this is easy.  What about fold (codenamed reduce or inject in some languages)?

If we did something like: [0,1,2].reduce(:+) in Ruby, we'd expect the sum.

What should we expect for this?

    2.reduce(:+)

The sensible answer is: 2. A case could be made for infinity as it is the sum of an infinite list of 2s, but I think we can just say that it evaluates to 2 and maintain consistency.

Maybe there is an inconsistency in this and it all falls down. But I hope that it works and that there is an efficient way to merge scalar values and collections of the same value. I suspect I'm just reinventing APL in conventional programming languages.

 

The Fallacy of One Definite Meaning

I saw an interesting article on Hacker News today.  It was called The Meaning of 'su', and it outlined the author's quest to understand the name of the su command in Unix.  It's a great read.  He starts with the assumption that su stands for super-user, and then infers that it means switch-user, and, well, it goes on from there.  But, there is an assumption behind all of it (and the comments on Hacker News) that really should be explored. The assumption is that things have one meaning.

Among friends (ones who will put up with it, at least), I have a habit of saying one thing that can be taken three different ways and meaning every single one of them. It's fun, but I think it is more than fun. It highlights what are, to me, a few deep truths:

1. Nobody has just one opinion about things. We are complex swirls of ideas and feelings, many of which may conflict yet be simultaneously true for us in our experience.
2. Meaning isn't objective, it is subjective. When someone asks for the meaning of something, it's fair to ask: for whom? Meaning is a form of valuation, and without someone for whom there is meaning, or a mean-er, it doesn't really exist.
3. Your meaning is as good as my meaning. And, in fact, your meaning can mean more to you than mine.  In other words, it doesn't matter what the authors of the su command intended, you get to decide what su stands for too.

Here's the thing. I enjoy etymology. It's great to see how words came to be and what was originally intended by them.  But, the unstated assumption is that once we understand that, we have some advantage. For instance, if we understand the core of the word property, we understand what is common between the notion of, say, personal property, and the notion of an object having properties.  Powerful stuff.  But, I think there's also a lot of power in recognizing that maybe there never was one single intended meaning. Maybe there were many. It's possible that the people behind Unix were having some fun with multiple meanings too. I wouldn't put it past them. 

 

A Type Driven Approach to Functional Design

InfoQ just put up a presentation I did at StrangeLoop last year.  It's a bit fuzzy as my talks go because I'm describing a way of thinking about design while doing it.

The crux involves thinking about the shapes of data and articulating intermediate steps between the input and output of a piece of a program.  I find that this helps me jiggle my thoughts about structure and arrive at the names of functions later.  It's sort of a sandwich between top-down and bottom-up.

Subtree Selection with Flatten/Select

Trees are wonderful data structures, but traversing them can often be a pain.  You can use recursion, as you long as you are confident that you won't overflow your stack.  But, if you want to do anything non trivial when you've found the nodes you care about, it can be hard to keep that work separate from the traversal code.

A few weeks ago, I ran into just this problem.  I was using Ruby's Ripper library to generate ASTs from Ruby source (Ripper calls them s-expressions).  Here is a simple example.

This call:

  Ripper.sexp("class A; def a; @a = @b; end; end")

yields this AST for the code:

  [:program, [[:class, [:const_ref, [:@const, "A", [1, 6]]], nil, 
    [:bodystmt, [[:def, [:@ident, "a", [1, 13]], [:params, nil, nil, nil, nil, nil],
      [:bodystmt, 
        [[:assign, 
          [:var_field, [:@ivar, "@a", [1, 16]]], 
          [:var_ref, [:@ivar, "@b", [1, 21]]]]], nil, nil, nil]]], nil, nil, nil]]]] 

 

Essentially, it is a tree constructed from arrays showing the structure of the program along with some text locations.

If we wanted to find all of the nodes for classes, it's simply a matter of searching for the arrays that start with a  with a :class symbol.  To find uses of instance variables, we search for a :@ivar symbol.  But, how do we traverse this?

Well, we could write that recursive routine, or we could take advantage of the fact that our trees are stored as arrays and use some of the functional goodness that we have on the Array class.  In particular, the flatten function. 

Flatten takes takes an array that may have elements that are arrays and merges those arrays into the top level.

For example,

  [1,[2,3],[4,[5]].flatten

yields:

  [1,2,3,4,5]

 

We can have more control if we provide an argument.  When you pass an integer, it tells flatten how many levels to merge into the top level.

So,

  [1,[2,3],[4,[5]]].flatten(1)

yields:

  [1,2,3,4,[5]]

 

Sweet, eh?

Here's our strategy.  Take the array that represents the tree and flatten it one level.  Then, select all elements which are arrays that start with the symbol we care about.  Then, flatten the array to the next level, and select sub-arrays at that level which start with the symbol. We can group all of these selections together and end up with an array of all of the subtrees regardless of the level at which they were located in the original tree.

If we have a sense of the maximum depth we can have in our tree, and we don't mind some inefficiency, we can do this using a side-effect free style:

  def sexp_select sexp, symbols
    (1..(max_nesting_level = 20)).map {|n| sexp.flatten(n)
                                               .select {|e| nonempty_ary?(e) }
                                               .select {|e| symbols.include? e[0] }}
                                 .select {|e| nonempty_ary?(e) }
                                 .flatten(1)
  end 

 

The code above takes a tree and produces a list of progressive flattenings of it.  The first element contains the tree flattened once, the second twice, etc.  It selects from each of them the arrays that have the symbol we care about as the first element.  We then select all of the non-empty arrays from that array (after all, there may be some levels that do not have the symbol we are interested in).  Afterward, we flatten that selection to bring all of the sub-trees from each of the flattenings to the top level.

This expression:

  sexp_select(Ripper.sexp("class A; def a; @a = @b; end; end"), [:@ivar])

yields:

  [[:@ivar, "@a", [1, 16]], [:@ivar, "@b", [1, 21]]]

which is an array of the subtrees that define the instance variable references in the AST.

Yes, that function is an eyeful, and there are things that can be done to make the code clearer, such as assigning the subexpressions to variables so that we can name them well, but if we don't mind the extra processing that we are doing, this is a rather elegant way to find subtrees within a larger tree represented in an array. As well, it is a decent example of functional style in Ruby.

 

 

 

 

 

 

The Framework Superclass Anti-Pattern

Unfortunately, this anti-pattern is too common.  It doesn't hurt until it hurts and when it does hurt, it hurts a lot.

If you are developing frameworks do not provide superclasses that framework users must inherit to use your framework.  Inheritance is the one of the tightest forms of coupling you can use in OO.  When you force your users to inherit from you, you:

• Make it nearly impossible for users to test their logic independently of your framework
• Make migration away from your framework difficult, or impossible.

The testing issue is subtle, but very real.  If you write tests as you develop you notice it immediately. When you inherit code from a framework,  it is mixed with your logic.  Often you are obliged to run that inherited code with the code that you really want to test, along with all of its dependencies, start up time, etc. 

Migration, as well, is a very real issue. If you've written logic important to your domain, there is nothing preventing you from being able to use that logic with other technology - nothing except coupling.

Yes, users of frameworks can guard themselves from that coupling by making their own adapters, but why put them through that?  Most of us hate vendor lock-in strategies in business.  Framework Superclasses are a very direct form of vendor lock-in. 

As a framework designer, you have many other choices: eventing, listeners, and object composition.

Consider them.  Thank you.