Michael Feathers

Measuring the Closure of Code

One of the most interesting maxims of OO (to me) is the Open/Closed Principle.  It states that modules should be open for extension but closed to modification.  Bertrand Meyer coined the term and, although he was thinking of inheritance as a reuse mechanism, he hit upon something central in software development. It turns out that regardless of whether we are doing OO or not, we know we are programming well when we find ourselves not modifying our existing code as much.  Code should grow by addition rather than mutation.  Need a new feature?  Introduce some new classes.  If you find yourself adding all of the code inline in existing methods and classes, know that down that path madness lies.  If you keep your classes and methods small and focused and you avoid tangling responsibilities, closure is often an emergent effect.

The Open/Closed Principle has been around for a while.  Many developers attempt to use it consciously, and others do things which tend to promote closure without thinking about it.  What is the net effect on code?

One of the nice things about having a version control history is that we can query it.

Here is a graph of the per file commit frequencies of source files on a random open source project.  Every point on the x axis is a file and the its height on y is the number of times its been modified.  The frequencies have been sorted in ascending order to make the picture sane:

  

One thing that's obvious is that there are a lot of files which have only been modified a very small number of times.  There are also some files which get an extremely large number of commits.  It's worth taking a look at the latter files and wondering why.  In some cases, it's because the files are configurational - they consist of things like data-based settings in code that change frequently.  At other times, they are havens of run-away complexity - snarls of logic that seem to accrete more and more code. Clearly the right side of the curve is where you want to expend some refactoring effort.  

Another thing that is nice about this view of code is that you can get a sense of what your payback will be for your refactoring.  The area under the curve is the total number of commits for the project.  If we inch backward from the right, we can calculate something like the 5% mark - the files which you are likely to touch in one out of every twenty modifications.  This may not seem like much, but it is over the duration of a project, especially when you factor in the effort of continually trying to understand that code.  I have no doubt that there is a decent formula that we can use to calculate the likely savings from refactoring based on the ratio of adds to modifications, and the commits in a code base.

Side note:

It's relatively easy to make these diagrams yourself.  grep the log of your VCS for lines that depict adds and modifications.  Strip everything except the file names, sort it, run it through 'uniq -c', sort it again, and then plot the first column.

 

 

 

The Thing of Software Development

It's been an interesting week in the software craftsmanship blog space.  Dan North wrote one up that expressed the issues that he had with the notion of craftsmanship, and Jason Gorman just wrote one calling for an end to holy wars in the development community.  I look at those opinions, and I can't say that I disagree with them. In fact, I think I'd carry it a bit further.

Every once in a while, when I'm talking to someone they identify me as part of the Software Craftsmanship movement.  I guess I am, but I don't go out of my way to present myself that way.  The same is true with 'Agile.'  I hardly ever use the word anymore.  It's just a label for a set of values and practices.  When Jason talks about 'Holy Wars', I see something far more innocuous.  What is see is sets of people looking for banners to rally behind.  It's a very human thing.  People form movements, they identify with them, they persuade others to their cause, and, frankly, a lot of good can come from it if the goals are good and people walk the walk that they talk.  But, at the end of the day, it's all philosophy.  It's all of us huddling beside a campfire and trying to figure out the right way to be in our work.  It's more than that, actually.  It's inward facing.  A collective form of self-preoccupation.

As much as I love working with people and aligning values and practices, I love the "thing."  The "thing" I am talking about is the set of extremely concrete challenges that we confront as software developers in code and in teams, and the way that we confront them.  Last week, I visited some people from Obtiva and we started to identify a set of ways that Rails applications commonly go wrong.  It's the thing that many people don't want to look at.  They know that code gets harder to deal with over time, and they know that refactoring is the path out, but how many of us are actively looking at why?  Or, developing roadmaps to confront these technical challenges.  There are two forces at work here, there's the Larry Wall-ish "the best programmers are lazy programmers." They find the easier way. But, there's also the fact that as much as we try to dodge, sidestep, or creatively finesse away problems, many are still there.  These problems are part of the "thing" and we have to talk about the "thing."  We have to talk about it a bit more than we need to talk about "us", in my opinion.

I guess I can explain what I am after by referring to something Kent Beck did years ago.  When he wrote the book 'Smalltalk Best Practice Patterns', he set himself the goal of not doing anything in his code until he understood why he did it.  The set of patterns in the book resulted from that reflective exercise.  It's very deliberate practice.  I think we need that sort of thing at the macro scale now.  We need to know why we do things.  The things that work and the things that don't.  We have to explore the sinew and shape of software as it grows and learn how it's impacted by us.  Learn the ramifications of our early design choices -- how to make broad changes in code and when to throw it away and start over. And, yes, we can call ourselves craftsmen, or engineers, or whatever.  And, we can have great values, but we have to concentrate on the "thing."  

Don't take this wrong.  I know that many people do this.  It happens in parallel with talk about craftsmanship, etc.  We are advancing the state of practice. But, what would it be like if the "thing" were the top topic of conversation?  If blogs, discussions, and conference sessions were all about the "thing"?

 

The String Calculator Kata in Haskell

Finally got around to recording this one.  Here's the String Calculator Kata in Haskell using a bit of REPL-based TDD through ghci.

The final refactoring felt like a bit of a stretch.  I'm still not sure how far to go when point-free style seems to be just in reach.  In this case, using a pair for the delimiter and the calculation string allowed me to avoid passing the input string as a parameter in the middle of a train of function compositions.  I wonder whether there is a name for this technique?

 

String Calculator Kata in Haskell from mfeathers on Vimeo.