Open Source != Open Distribution Apple sea change in effect? Control and Data
Aug 05

The secret to productive software development: Creating the Happy Path

Tech with tags: , Add comments


I am reading A Theory of Fun written and illustrated by Raph Koster. It is a fantastic little book that the renaissance man of Mozilla, Atul Varma, passed over (to Ben actually, and I nabbed it for a quick read!)

In theory, the book is about computer games, but in practice it is applicable to Everything. The content is deep in a way that has you applying it to your own areas of interest. For me, my thoughts kept rolling over to two loves…. software development, and parenting (education).

Some of the early thoughts were about how the brain works, and the “chunking” that the brain does with information. We are not robots or computers. Most of us are unable to recall exact events. If someone says “what searches were run at 12:34pm last Sunday” to Google, it could tell you. If you asked me to explicitly name the events of the day, the detail would be replaced by the general. “Got out of bed, got dressed, went to work” etc. The brain is chunking away the details that don’t matter.

You can then jump from that to think about what happens when you master something. We talk about “muscle memory” which has less to do with muscle, and more to do with the brain. There is research that shows that thinking about actions helps grow muscle memory…. so Chevy Chase is right when he told us to be the ball This is old hat for sporting chaps who are often told to think about events with positive outcomes, preparing for that fantastic cross from Cristiano Ronaldo and your split second attempt to glance a header into the bottom corner. Practice is more than physical.

It is interesting to think about how the brain constantly internalizes patterns. Raph mentions the obvious data point of how our eyes see our nose, but we don’t see it. In fact, most of the time we are filling in the blanks. There is a reason why eye witnesses are so awful. We think that we “see” things in the literal sense, but in fact we are seeing our own doctored reality.

Anyway, how does this relate to software again?

Creating the right path for a challenge

If something is too easy, we find it boring. If something is too hard, we dismiss it and don’t want to play. We can see this with Tic Tac Toe (discussed in the book). If you are playing a kid, they tend to lose all the time, until they grok it and then they can draw all the time and it instantly becomes no fun.

To do our best work, we need to cut the right path. When I think back to the best teachers that I had, they were the ones that created that path. They balanced enough repetitive thinking so my brain could happily learn patterns (it reeeeally likes patterns), but in a way where you were constantly growing and building. We have all heard the following said: “That teacher was great. They made Math fun.” Sometimes you may think it was the funny teacher, or the young teacher who let you get away with anything that would be the most fun. For me though, that wasn’t the case at all. It isn’t fun to not push yourself, or feed the brain at all. If you think back, you will probably find that your favourite teachers actually had very high expectations of you, they just also gave you a path with a good learning curve.

ASIDE: This is probably why I have found that my best teachers weren’t necessarily the smartest ones. It is hard to teach a tough concept to a student when it is innate and obvious in your own mind. A seminal moment for me in Math was when I sneakily changed teachers in high school.

The goal of the team on a software project should be to recreate this curve. That arc of the project needs to keep you growing in a way that has you enjoying the growth.

Why Rails?

There is another tension in software development. Once you get really good at a given technology, when does it make sense to jump to the next one versus staying the course?

I have been through this quite a few times. Take Web development as an example. I had my own Web framework when that was all the rage, and it evolved with the needs of my projects. I knew it intimately, it was nice and lean as it only had what I wanted in it, and there was room to grow. Fast forward to 2009, am I better off having fulfilled projects across a series of frameworks or platform, or would I actually be better off staying the course with my and friends?

My personal framework breaks down fairly quickly when you think about working in teams with varied backgrounds, but beyond that I also find myself back thinking about “fun”.

A lot of developers jumped onto the Rails bandwagon as they thought it made Web dev fun again. Writing code that does real stuff feels more “fun” to me than the scaffolding of XML required by the big Java frameworks of the time.

I also believe that Rails did a fantastic job at giving people one of those nice paths to success. It is “easy” to get started using it, but there is enough to learn to keep you enjoying life in the framework.

There are a huge number of other factors of course, and the marketing and community that DHH created have a lot to do with the path that Rails itself has taken (good and bad). Both marketing and community are key as they can change the needle on perceived fun which is all that really matters.

Refactoring as a way to help the path

How can you help build the magic Happy Path? One way is by giving yourself the gift of time, and using it correctly.

Over time, code on a project can get crufty. If you are running fast and hacking together new features your source will become sludge and your velocity will suffer. At this point in time the coding becomes Less Fun to deal with (productivity goes down) and the brain runs into a steeper path which borders on “too hard”. Give yourself the gift of frequent refactoring and you may well be able to do the gardening that will keep the path green. Agile folk have talked about the benefits of paying forward through refactoring, and using testing to enable safety as you change things. You may also enjoy the act and art of refactoring itself, and creating functionality afterwards.

Refactoring also offers something else. It offers a way to better yourself as a developer. If you only ever write an API once, then you can only learn so much. If you are continuously evolving your systems you are trying out new tactics and approaches which you learn from.

“Project Plan” == Happy Path

This comes together and makes me want to revisit the light “project plan” and make sure that it is leading ahead, shining the light on the happy path itself.

Chunking your code

As I thought about the fact that our brain chunks what it does, and how it is probable that it doesn’t see the reality of the code, it makes me wonder. How does this affect our code? Shortly after we write it, we forget what we did. We probably forget some of the reasoning that made sense at the time. You may remember to put some inane comments in there if you are lucky, but that still won’t help too much with the big picture.

Since we can’t keep the exact code in our brain, how does that affect what we write? It makes me feel like the importance of a clean architecture, small modules, and simplicity really kicks in. Patterns can really help here too. When you cracked open an early Rails project, you would be able to understand a lot about the project based on the patterns that you are used too. That enabled a higher level of thinking about the project. If you can create your own higher level modules that build on each other then a lot of understanding comes along with you when you crack open a nice small module.

DSLs: Patterns are brain food

The DSL movement continues to grow in some ways (they had a DSL conference!) and in others it has always been there (SQL is a DSL!)

When you think of the pattern matching brain food, it makes me think of DSLs. Once more, if you let your brain learn those DSLs then it makes life easier to build products based on them.

Functional Programming == Pattern Matching

And the pattern matching logic of the brain brings me to another hot spot of current dev talk….. the re-rise of the functional languages!

Testing builds stairs

There aren’t many insights here per se. Refactoring, simple code, keeping things fun…. all good stuff that we kinda know.

We have all been on projects that aren’t fun, and progress is sloooooow. On the other side, we have also all been on projects where you can’t step away from the computer.

I had a reprieve of this only a short week ago. I was staring at some functionality in a project that was a lot of messy work and my brain wasn’t ready for it. It was only when I took a step back and plotted the Happy Path that things came together. By plotting “first, lets just get up the first stair” it kept me engaged. Then I continued to climb the stairs and the brain kicked in. Before I knew it I had reached the next level and things were done.

One way to build out the stairs on the path is by creating unit tests. Knowing that you have 40 tests completed lets you focus on the first. You know that if you keep learning and climbing you will get there.

If unit tests only helped you build the foundation for the stairs then rather than being “a waste of time and code we need to maintain” they become something much more. And of course, this adds to the fact that they are insurance for the future, and help you become a client of your own APIs immediately.

As you can see, I thoroughly enjoyed reading the book and day dreaming my way to apply it to software.

17 Responses to “The secret to productive software development: Creating the Happy Path”

  1. Raph Koster Says:

    Wow… you took that in a fresh and unexpected direction. :) Glad you enjoyed the book!

  2. dion Says:


    Thanks so much for checking out my post, oh and for creating a fantastic book that was both fun to read, and got me thinking.



  3. Mark Holton Says:

    Great post — found it a little inspiring. Agree about Ruby/Rails too. It helps me get the fires burning. cheers-

  4. Paul Barry Says:

    Great post, almost a Steve Yegge-esque rant. I mean that in a good way. :)

  5. Andy Atkinson Says:

    Cucumber has made test-first development fun for me, and delivering working (Rails) software that is well tested is very satisfying.

  6. Erik Says:

    Very interesting post!

  7. Joeri Sebrechts Says:

    Another book on memory and the human brain/mind that I found very interesting is “On Intelligence”, by Jeff Hawkins. It gave a me a lot of those “aha!” moments as well.

  8. Edwin Khodabakchian Says:

    Great article with a lot of interesting points.

  9. dion Says:


    Thanks! I will check out “On Intelligence”.

  10. dion Says:


    It would be honour to write a quarter of a Steve Yegge post…. in many ways :)

  11. wavded Says:

    Thanks for the post. Very insightful. I’ve been up against refactoring a couple of projects and honestly I just shut down mentally because its a “huge” task to wrap my mind around but when you talked about unit tests and small manageable parts, it gave me some hope in getting things done without being the brain bloat.

  12. @F1LT3R Says:

    Dion… this was so awesome to read! It really resonated with my own experience, making sense of the way I have been feeling about various projects and what I can do to stay on the happy path.

    I would love to hear more about mapping out the happy path. How do you know ahead of time, where to draw that curve of inspiration and ingenuity?

  13. Brad Neuberg Says:

    I love that you have a picture of Chevy Chase from Caddyshack on here! I grew up on those films….

  14. geeks Says:

    Interesting note about the “fun” aspect of development and technology.

    Haha! I haven’t been convinced that it will be more fun to give up my framework in favour of Rails yet.

  15. Aliens InfoTech Says:

    Computers are now invariably used in every area of business. It is being seen as a symbol of pride and if activities are automated it will save the much needed time and energy. Your article highlighted the importance of developing Software for minimizing users intervention. We also understood the various issues associated with software development like portability, reliability, response time etc.

  16. Henry Walts Says:

    “Challenge” always needs a right path to make it achieve.You made a brilliant trial to remind people about their right path and hence they know how to achieve their challenges.It is really good one..Thanking you

  17. Anonymous Says:

    I agree with what you are saying about “fun” being important for productivity, but: (1) RoR, refactoring, or TDD/BDD are not necessarily “fun”. TDD/BDD by average developers can produce a large number of brittle tests which make things no fun at all when you need to change existing behavior of an application. Yes, you could whack out those tests and fix them, but then you have to determine what each of them tested to make sure you are testing the same thing. Then the lack of structure, typing, and decent debug logging by most rails/gem/plugin developers (really RoR developers in-general) leads to situations where you can develop some things uberquickly (somewhat fun, but not challenging enough at times) and then get stuck in the mire for days tracking down bugs (no fun!). (2) If you have a crappy work environment (poor accountability, poor organization/process, poor leadership, poor communication, lack of mentoring/training, low morale) it is definitely not “fun”, and those things you mentioned won’t help. The best course of action then is to find another job you are happy with, but it is difficult to leave in a poor job market and/or when you are getting paid over what is required for the job they want you to do. It would be great if you could reply with what you think on some of these issues.

Leave a Reply

Spam is a pain, I am sorry to have to do this to you, but can you answer the question below?

Q: What is the number before 3? (just put in the digit)