Software development, like any other industry, has trends. Over the past couple of years it's been fun to watch the ascendance of the Domain Specific Language meme. I’ve loved language design since I first started programming and I like to see people work in that design space. Language design is challenging and immensely satisfying work. It's been said that one of the oddities of software design is that we suffer from a lack of constraints. At times, we have severe memory or time constraints, but for the most part, there is nothing that forces us to choose, say, one object structure over another other than our understanding of how it might impact future change. When you design an embedded DSL -- a DSL built of constructs in a surrounding programming language, the structure of your classes has an immediate impact on the expressivity of your language. It's API development writ large.
But, embedded DSLs are just one part of the DSL space. The rest of the space is occupied by external DSLs -- standalone languages that compile down to some other language or a run-time. In general, they give you a freer hand. You can create a complete syntax and tune it directly to a particular domain. Most of your constraints evaporate. It’s a very tempting approach but it does have some pitfalls. When you are a consultant, you end up seeing many proprietary technologies that you can’t talk about. There are more external DSLs out there than most people realize and more often than not, I see companies trying to get rid of them.
Well, there are a couple of reasons. One is the cost of maintenance. Someone in the organization needs to know enough about the DSL to make the inevitable changes. In a way, this isn’t much different from the cost of maintaining a framework, but many organizations see it as a difference in kind because the tools that you use in an DSL implementation are often not the same as the tools that you use in conventional business applications. It’s one thing to hire developers who know C# or Java or Ruby and yet another to hire or make sure that you have some who know ANTLR or YACC. In some companies this is absolutely no problem. They hire very knowledgable developers as a matter of course and they are sufficiently polyglot to make the cost of maintaining a language implementation negligible, but on the other hand, there are many companies that look at the cost side. They write software, but their business isn’t software, so they are constantly trading off the cost of maintaining their DSL with the cost of dedicating those people to work that is closer to the business line.
One other reason why companies move away from external DSLs is that they can complicate hiring. Years ago, I designed an external DSL for an organization that felt that they could save money by developing a high level language that their domain experts could use directly. I spent a good amount of time on the implementation and it was used successfully for long time. After a while, however, they dropped it. The domain experts were technical enough to use a mainstream programming language, but beyond that, moving away from the DSL made it easier to hire them. Let’s face it, if you are looking for a job what do you want to have on your resume? A language used by precisely one company or one that is known or used throughout the industry?
Like the cost of maintenance, hiring cost is just one piece of the total cost of ownership for a DSL but it can be the one which tips a decision. One very public example is Erlang. People are paying a lot of attention to Erlang today, but we shouldn’t forget that it was dropped by Ericsson in part because they were concerned with their ability to hire and train developers. Many people know the story of Erlang, but what they don’t realize is just how common this scenario has been in the industry. I get to see a lot of legacy code in my work, and a good amount of it is in homegrown languages that companies have trouble supporting.
The thing to remember about DSLs is that they are a commitment and the commitment doesn’t end when the initial implementation is done. This is definitely true for external DSLs, but it’s also true for embedded DSLs, albeit at much smaller scale. When you make a commitment, you have to make sure that you have the will to follow through and you also have to make a sober assessment about whether you will keep the commitment even under adverse business conditions. If you can, it’s worth doing. DSLs are powerful technology.
Enough about failure. How can you succeed with DSLs?
Here are a couple of ideas:
- If you are creating an external DSL, consider open sourcing it or, at least, consider forming some sort of an industry group around it. This partially externalizes the cost of maintenance and avoids hiring trouble.
- If you can’t or won’t open your DSL(s) to the outside world, make sure that your developers are not expected to use them exclusively. If you don’t, you can expect to pay higher wages for developers of equivalent skill than you would otherwise. In negotiations, they will factor in the cost of working with little known technologies.
- Consider embedded DSLs as an alternative. Often they are easy to construct and they can give you many of the same advantages. Dynamic languages like Ruby are ideal for embedded DSL construction, but embedded DSLs have been written in just about every mainstream programming language.
I like external DSLs and I think we'll see more of them. But remember, success is a function of far more than the technology.