Michael Feathers' Blog

The Cult of Language Expertise

It happens time and time again on mailing lists and discussion forums – someone posts some code in the context of an example and then there is an outcry. “The code is lousy!”, someone says.  So you look and you realize that “lousy”, to this person, means that that it doesn’t use language feature X, that just came out in the last release; or that its author didn’t use idiom Y, which someone in some recent article proved is necessary to write “good” code in a particular language. 

Sound familiar? It happens all the time.

The truth is, I was one of those guys. Years ago, I took a lot of pride in the fact that I was knew all of the nooks and crannies of C++. I nearly memorized the ARM (Annotated C++ Reference Manual), and in flamewars on the net, I did my language-lawyerly best to point out little problems in posted code, and (I thought) advance the state of practice a bit. But, I always felt odd about it. If writing code one way was okay before the introduction of a new language feature, what made it bad immediately afterward? 

Java and C# offer some great examples. If you are working in Java 5 or 6, you must use varargs when you can. If you’re working in C# 2.0, well, you must make a class static if all of its members are static. It’s just the right thing to do. Then, of course, there’s the issue of generics. Before their introduction, people would cast objects as they retrieved them from containers. Afterward, “good practice” dictated that you use type-safe collections and avoid the casts. It was just seen as “the right thing to do.” But, of course, if you work in JavaScript, Python, or Ruby, you can play by different rules because these "rightness" rules vary across languages.

It’s hard to fault anyone for trying to keep up with language developments. If there are, now, safer and better ways of writing code, isn’t it important to move along? I think it is, but only to a point. 

The fact of the matter is, there is a strong psychological component to language expertise. The people who become language lawyers (and I was one of them) aren’t doing anything complex, they are actually doing something very simple. They are sitting down and learning all of the rules and using them as a point of leverage.  They think, "I am valuable if I know the right way to use the language." But, frankly, there is harder work out there.

If you want to see that harder work, try to give advice to people who are working in less than current environments. Try to write libraries and tools for people who are working in antiquated environments. That is the point where you see the continuum.  You can write good code without the latest features. People do it all the time.

There isn't some continually moving standard for good practice in a language, there are hundreds of islands, places where people have discovered what works for them in their environment.  Sure, there are incremental advantages to using the new bells and whistles but they aren't going to make you or break you.

Language expertise is fine, but it isn’t the most valuable thing out there. If someone programs conscientiously, I can work with them. I have a lot of respect for people who write solid code despite not having completely up to date language knowledge. Language knowledge is easy: you read, you think, you try.  And, you can catch up.  Conscientiousness, though, is the thing that really matters.  Next to it, language expertise is easy.

Alpha, Beta, and Gamma Software Organizations

An Alpha software organization uses the latest stable versions of their tools and development languages. It moves as rapidly as prudent and possible to the latest release of, for instance, Python, gcc, or .NET. 

A Beta software organization may not be using the latest versions but it has plans to move to the latest as soon as possible. Beta organizations have solid dates or triggers for the migration. They are either waiting for particular tool vendors to catch up with a language release or for some other part of their own organization to do a roll out.

Gamma software organizations are in limbo. They have no plans to upgrade. These are the organizations which drifting along in Java 1.3.1 or .NET 1.1 or some pre-Cambrian version of C++.

Organizations can get caught in Gamma-hood for a variety of reasons.  One is client dependency.  If you are shipping software to external clients and they can't/won't upgrade their systems, you're caught.  If you're running on your own servers, Alpha-hood is yours to lose, and it's a shame that so many organizations do lose it. 

It's easy to fall off the upgrade path, but, having visited many teams, I notice that the ones that don't fall off perform better. 

A team's upgrade policy seems to be a reliable measure of health and seriousness. 

Culture of Severity

A month ago, I was at a conference in the U.K. I always enjoy going over because I get a chance to catch up with people I’ve known for a while, but this time, it wasn’t very pleasant. I met up with a friend who I’d seen at conferences on and off for years and I asked him how he was doing. He told me that he was fine now, but recently he’d been deported from the US. There was some kind of a problem with his paperwork when he arrived from the U.K. so he was invasively searched, handcuffed, placed in a cell overnight, and deported the next day. According to his lawyer, he may never visit the US again.

This friend of mine isn’t alone. I’ve been reading a number of stories recently about people who’ve been treated like criminals by US Immigration for rather minor issues. 

We’re supposed to be better than that.

The thing I want to know is: what makes this sort of culture of severity happen?  It's easy to blame it all on 9/11, or claim that it is solely the result of the current administration's policies, but I think that there is more to it than that.  It seems that this sort of severity is very close to the idea of zero tolerance, the idea that if you crack down on the minor infractions, people will fall in line.  It seems to work to some degree, but it also tends to make us somewhat less than what we were.

The big question is: why?  The Zero Tolerance explanation is appealing but I don't think it's the whole answer.  The whole thing feels much less calculated than that.

Suomi Fever

I'm in Helsinki now.  This is the first time I've been here in the winter.  The days are about five hours long, and it is cold.  The darkness takes a while to get used to.  It's odd feeling fully awake when it is pitch black outside.  There's nothing quite as disorienting as a 4PM that looks like a 9PM.  The positive thing I'm getting out of being here over a weekend on a consulting trip is that I have plenty of time to write and code without distraction.  If I do want distraction, I wander the city a bit, but, again, it is cold.

The people here seem to cope well with it, though.  Mexican food is big, and I guess I understand why.  Mexico is the absolute anti-thesis of the Finnish winter experience.  But, it is funny to see Mexican restaurants with huge snow cakes outside. 

I've found a couple of Mexican restaurants I like, but one puzzles me more than the rest, it's called Amarillo.  I've gone there twice so far.  The first time, I was met at the door by a coat room attendant and he said "Oh, if you want to eat it will be about an hour."  Having nothing else to do, I said "okay."  I usually have a notebook on me and I can do just about anything with that notebook that I'd want to do with a word processor on my computer, so I went and stood where he told me to.  A waitress came over and asked me if I'd like to be seated.  I said "yes."   She seated me, I ordered, and I got my food within about fifteen minutes.  Odd, but workable.

Last night, I went to the same restaurant and was met at the door by a different coat room attendant.  The people in front of me were told that it would be about an hour and they walked off.  I asked the same question, and I was told, as well, that it would be about an hour.  I put on a big grin and said "okay."  Now, the coat room attendant wasn't expecting that, so he started to look a little sour.  But I told him that the last time I was there, I was told it would be an hour, I was seated in a minute and got my food promptly so I decided to take a chance again.  He told me where to stand, and again a waitress came over and sat me immediately.  It took a little longer to get my food but not much.

I tried asking the waitress about it but she smiled and didn't have much to say.   It's good food, so I'll probably go  back, but I'll definitely be tempted to use a classic Larry David Stare Down the next time the attendant mentions an hour.

On this trip, I'm going to get a chance to meet up with some people from Agile Finland.  I'm looking forward to that.

From Disaster to Decorator

The other day I ran across a blog that showed a good way of cleaning up a mess.  Joe Lowrance had a class that did a large amount of validation logic and he decided to separate it out in different classes.  After he had, he noticed that it could compose it all as a decorator.  His description is very much like Move Embellishment to Decorator in Josh Kerievsky's Refactoring to Patterns.  Well worth a look.

If I had to go further with his example, I might move toward using the prototype pattern to create the decorated objects, but prototype is a big gun.  I'd hold back until I had a few more cases.

Porn for Consultants

I don’t watch much TV, but there’s one show that I’m addicted to. It’s Kitchen Nightmares with Gordon Ramsay. There are two versions of the show, the British version and the American version. The British version is a bit better – it’s not quite as fluffed up, but regardless, both shows are worth a look. In them, Gordon Ramsay, a famous chef, goes to failing restaurants and tries to turn them around in a week. And, the way that he does it is incredible. He walks in, tries out the food, criticizes the hell out of it, and then proceeds to tell everyone on the staff just how insanely bad everything is and precisely how responsible they are for it. He doesn’t mince words either. Often every other word is bleeped. A couple of times, he’s gone to restaurant owners and told them to f--- off. It might just be that that’s how you say “pay attention” in the restaurant trade, but I’m pretty sure that it doesn’t work in software development consulting.

Ramsey has some advantages, as a consultant. Every meal service is a new iteration. He can walk in, teach the staff something, and see immediate consequences. He tears people down and builds them up, as needed, but when he leaves, the restaurant is a bit better off than it was before and people know what the problems are. Ultimately, they have to decide whether they are going to solve them. 

The fact is, this is very much like software development consulting, except for the “rough and tumble” bit. And, that’s why it’s porn to me. I think that ultimately, in the restaurant business, you can get away with a bit more because it’s a less polite industry. It’s seen as blue-collar. People yell at each other to get dishes out of the kitchen. It’s expected. In most software development teams, if anyone raises their voice it’s a big deal. Developers walk eggshells with each other, and it can make communication less direct than it should be. But, really, we need to call each other out when good practice lapses in a team. If we don’t, it’s not politeness; it’s abdication.

Ramsay gets results but sometimes it doesn’t work out. In the British version of the series, he goes back to restaurants he’s visited in the past and sees how they are doing. Some take what they’ve learned and continue to make progress; others fall back on old behaviors. I noticed early on that some teams I work with fall back also, and it led me to take a bit of a different tack than Ramsay takes. The question that I always have in mind when I’m working with people is what they’ll be doing after I’ve left. How do I give them something that they can build on and know whether they will own it. Maybe that’s what politeness is for.

XPDay London

I'll be speaking at XPDay in London this year.  The motto of XPDay is "more than XP, more than one day" and, sure enough, XPDay is being held November 19th and 20th.

The program is very good and I'm disappointed that I won't attend more of it since I'll be doing a class on TDD and a workshop on determining the organizational contexts which help, hinder, and challenge agility.  But, having said that, I am looking forward to the talks/tutorials I will be able to see.. those by Lasse Koskela, Elizabeth Whitworth, Jeff Patton, and Keith Braithwaite.

The TDD class is called TDD: The C++ Variations.  It's going to be a nuts-and-bolts session on making TDD work in C++.  Unfortunately, C++ hasn't received as much attention in the agile community. The class is hands-on, and we'll confront the issues that you run into immediately when you want to want adopt a test-driven approach.

From Objects to Functions in OCaml

I’ve been diving into OCaml recently and I’m using my usual diving board. Whenever I want to learn another programming language, the first thing that I write is an xUnit-style testing framework.

I don’t do it just because I love testing frameworks, although that’s reason enough, I guess – writing one makes learning and working in a new language much more enjoyable. No, I do it because implementing xUnit forces you into the nooks and crannies of a language early. The first thing that you do is look for reflection, exceptions, and object structuring. Once you’ve used those parts in a language, you may not know the best way to do things yet, but you can survive.

So, here's the core of my first-attempt OCaml xUnit – the test_case class:

class virtual test_case test_name =
object(self)
  method set_up = ()
  method tear_down = ()
  method virtual run_test : test_result -> unit
  method run result =
      result#test_started;
      self#set_up;
      try self#run_test result with
        TestFailure message ->
          result#add_failure test_name message;
      self#tear_down
      
end;;

The run method contains a straightforward implementation of the template method design pattern.  In OCaml, virtual means abstract.  The run_test method is implemented in subclasses of test_case.  Any exceptions that are thrown in a test are caught and passed to an instance of a class named test_result.

As it stands, this little xUnit works.  It's functional, but it isn't really functional.  In OCaml, you'd expect to see some functional idioms in use, but it's hard to see where they should be used.  In xUnit tests have state.  They call set_up to create it, and tear_down to get rid of it.  How do you handle state in a functional way?

The typical answer in functional programming is to parameterize.  Imagine that we don't have classes and we want to do everything that test_case run does.  We'd end up with a function that looks something like this:

let run_test setup test_function tear_down result = 
  set_up;
  try test_function result with
    TestFailure message ->
      add_failure result message;
  tear_down

Nice, but look at all of those parameters!  Makes you wish for objects again, doesn't it?

Well, not so fast.  One of the cooler things you can do in functional languages is partial application.  You can create a function that is defined as the application of another function to some subset of its arguments.  So, if I want to have a set of tests that use the same setup and teardown, I can define a new function like this:

let xpath_test = run_test xpath_setup xpath_teardown

If you remember from above, run_test takes four arguments: set_up, tear_down, test_function, and result.  What we've done now is bind xpath_test so that it uses xpath_setup and xpath_teardown any time that it is called.  However, you still have to pass the other two arguments (test_function and result):

xpath_test a_test_function a_result

So, really, you don't need objects for template method-like work in a functional programming language.  But, there is one little problem: state.  When you're working in  an object, you have shared state among your functions by default.  When you working with functions alone, you have to pass state.

Here's how Ounit (a much more functional xUnit ported from Haskell) solves the problem:

let bracket set_up f tear_down () =
  let fixture = set_up () in
    try
      f fixture;
      tear_down fixture
    with e ->
      tear_down fixture;
      raise e

The bracket function assumes that set_up will return a fixture of some sort.  It also assumes that the test function and tear_down will accept the fixture.  It looks a little convoluted, but I suspect that template method looks convoluted to many people also.

I suspect that over time I'll start to see functional solutions from the very beginning, but it's nice to see that the path toward more functional style isn't very rocky.

Hamatai!

I’m back home now.

Last month involved a lot of travel.  I was with a client in Spain, then I was at the JavaZone conference in Oslo, followed by a trip back to the US for SD East, and finally back to Denmark for the JAOO conference. And, as if that wasn’t enough, I made a detour to Oslo on the way back from Denmark just to see a band. 

I know.. traveling specifically to see a band sounds bizarre, but people have been known to travel for football so I don’t think I’m that far nuts. The truth is, though, I’m not alone. The band I went to see is known for inspiring obsessive behavior. Steve Davis, an English snooker champion, blew his prize money in the 80s, getting them to re-form and hold a series of concerts in London.  That's obsessive.. and I think beats out the last minute trip I took from Miami to San Francisco to see them play during their first North American tour back in 2000.

The band's name is Magma, and they're hard to describe.  Take every intense form of music you've ever heard in your life.. avant-garde jazz, modern classical ala Stravinsky and Carl Orff, African-American gospel music, blistering rock and African chant.. undercut it all with furious drumming and distorted bass played at twice the volume you'd expect.. put a choral group on top of it.. and then make it intermittently violent and intensely beautiful, and you've got Magma.  Not everyone's cup of tea, but they are playing tonight and tomorrow night at a jazz festival in Nancy, France, and I kind of wish I was there. :-)

Assorted Links:

MDK Part 3
Kohntarkosz Anteria Extrait 1
Kohntarkosz Anteria Extrait 2
Magma's MySpace Page

What I learned from C that made me a better Pascal programmer

I snuck into programming through the back door. At first I wanted to be an architect, but I went to school in the days before CAD and, well, India Ink and I didn’t get along. You could easily spend hours/days on a drawing and if, at the end, you made one little botch you had to start over.   I think I wanted refactoring before I even knew what it was.

After architecture, I started taking math classes and I considered majoring in math, but it felt a bit selfish to me. I didn’t think that I could feel useful plugging away at theorems. Also, I wasn’t sure I really had the aptitude. I liked math but proof was harder than I expected.

My next stop was engineering. It looked like fun, but I started to have nightmares about designing vacuum cleaners for a living. It's not that I don't like vacuum cleaners, it's just that I don’t have a deep passion for them (applause for the people who do).

It all changed for me when I got a co-op job at a company that was writing demographics software. I worked with a computer running SAS-PC programs, and I became curious about programming.  It looked like fun. I started to learn more and then, with all of the fire of youth, decided that if I was really going to make a career of programming, I had to know that I could really be good at it. I had to know that I could really excel.

How did I convince myself? I decided that I would teach myself C. I’d read that C was the hardest programming language out there, so I figured that, hey, if I could learn C as a first language, by myself, programming might be thing that I should throw myself into. I started writing little graphics programs, displaying the results of formulas I’d read about from chaos theory. It was exasperating but fun. I stayed after work, exploring all of the nuances of C compiler command line. I stumbled through pointers and indirection, manual memory management, and printf debugging. It was hard slogging through alone, but I did it. I convinced myself that I could program.

The next step was to change my major to computer science and start taking classes. My first programming class was Introduction to Programming with Pascal. For a self-taught C programmer, nothing could be more gruesome. You couldn’t just open a file, you had to pass an input or an output parameter to your program. You couldn’t really touch memory, you had all of these little bits of syntax in the way that didn’t seem to map to anything real.

It was hard for me to know what was really happening behind the scenes, and I was a bit shaken by the experience. But, nonetheless, I thrived. I took more courses and I dutifully went to the computer lab every afternoon and logged onto the VAX/VMS operating system(!) and went about my work – until – one day late in the term – I looked at the terminal of the woman next to me and I saw something strange. It was an array subscript out of bounds error.

I’d never seen one before.

After a bit of reflection, I understood why. When you are programming in C, nothing is going to save you. After you spend hours debugging an errant pointer dancing over memory, you just learn not to do that.

These days, I have a greater appreciation for Niklaus Wirth's work (although I don't really have fond memories of Pascal) and I do appreciate it when a language's runtime or static checker lets me know when I've flubbed up in a particularly grievous yet stupid way.  But, after all is said and done, I'm glad that C taught me that lesson. 

Safety is good, but so is the possibility of danger.  It keeps us thinking rigorously, and there's value in that.

 

This blog was inspired by Reginald Braithwaite's Three blog posts I'd love to read (and one that I wouldn't)