Articles Feed

Reflection on Hangman

by: micah | October 4th, 2009 |

Last week concluded the Hangman Ruby Sparring Tournament. The results are below.

Unlike the previous Battleship Tournament, I put the effort in to write a competitive hangman player (of course he was banned from competition). This made things interesting. By knowing precisely what it takes to create a player, I was able to precisely tune the scoring metrics. My goal with the scoring metrics was to evaluate the code, without actually looking at it myself, and rate it’s quality/cleanliness on a scale of 0 to 100, where 100 was asymptotically impossible to reach.

It is not easy to score well on ALL the metrics. Each metric will push your code in a different direction some of them are opposing. For example, to score well on the flog metrics, you have to break your code up in to lots of tiny methods. However, extracting methods increases the amount of code you have which brings down the simplicity score. To get an overall high score requires compromising between opposing metrics over and over again. Realistically, writing clean code requires the same type of compromising between coding principles. I am thoroughly impressed with the high scores that people were able to achieve in this tournament.

In the end, I believe that the highest scoring players do fit many of the criteria for “clean code”. Which is to say that to metrics used to evaluate the code are fairly complete. However, there is one gaping hole. The highest scoring solutions look like this:

1. def b
2. @r.fill(0)
3. c
4. @l = @l.sort_by { |x| @r[x] }
5. end

People discovered that by using short variable names and method names, they could considerably reduce the “simplicity” of their solution. For this tournament, the simplicity score measures the code mass (compress all the code and count the bytes). It’s a fair metrics because in general, the less code there is, the less code people have to read to understand it. I especially like the fact that comments hurt the score. Anyhow, clearly the code above is not good code.

What’s missing is a analysis tool to measure “Readability” of code. I imagine such a tool would be similar to flog and flay in that it parses your code and examines all names used for variables, methods, classes, etc. When it finds 1-letter names like above, it punishes. When it finds names containing english (or other languages) words, or derivatives of other names, it rewards. For example, it I had a class named “Player”, that’s an english word which is readable. Good. Now a class named “ThingAmaBob” is not english but that not necessarily bad. And you’d expect variables with names like “mythingamabob” or “thingama_bobs” which would be good. There are plenty of ways to expand on the idea. Such a tool would bring our repertoire of metric tools one step close to completion.

7 Responses to “Reflection on Hangman”

1. Markus Gärtner Says:
   October 4th, 2009 at 01:08 PM

   For Java PMD does some of this. I am sure you can write a rule that could do a lookup of the namings provided in some dictionary, too. I'm surprise that there is nothing likewise in Ruby.

2. ehsanul Says:
   October 4th, 2009 at 11:11 PM

   I found that the flog scoring method actually made the code less readable, in a different way, as I tried to raise my player's flog score.

   The contest counted how many of the methods went over the set threshold of 10 points in flog.Though it does help make the code cleaner in many cases to try and reduce a method's flog score to below 10, and delegate some complicated bits to other methods, one can also take it too far. Sometimes, it makes sense for a method not to be ridiculously short. If you divide a task that really feels like a logical unit, dividing it into several methods can make it harder to follow for someone unfamiliar with the code.

   For real world development, taking note of average flog scores, and looking at just the worst offenders in code would probably be a better approach than trying to limit everything to a score of 10. But of course, if that were done in this tournament, you would end up with players minimizing average flog score by making many empty methods.

3. Joel Westerberg Says:
   October 5th, 2009 at 09:12 AM

   Actually I choose short method names because it was significantly faster in jruby.

   At one point I had the solution down to almost 3 lines of ruby. Chained Map - Inject - Map or something like that. Not very readable, but very short. I had to break it up to several methods, and remove inject.

   def guess(word, guesses_left)
     matcher = Regexp.new("^#{word.gsub("_", ".")}$")
     @word_list.reject! { |w| !( matcher =~ w ) }
     rating = @word_list.inject({}) { |r,w| w.split(//).each { |i| r[i] = (r[i] || 0) + 1 }; r }
     @left = @left.sort_by { |x| (guesses_left < 4) ? (rating[x] || -1) : (%w{a e i o u y}.include?(x) ? 10 : 0) }
     @left.pop
   end
   

   I really like chaining methods that take a map. It creates a functional chain and the code becomes very free of error.

4. Joel Westerberg Says:
   October 5th, 2009 at 09:32 AM

   I think the speed increase was something like 10-20% on one letter method names.

5. Krister Says:
   October 5th, 2009 at 10:43 PM

   Looking forward to the next sparring match.

6. Kerry Buckley Says:
   October 6th, 2009 at 09:55 AM

   I found it pretty hard to keep within the 'code volume' limit required to get 100% for simplicity without compromising readability of method and variable names, but I think I more-or-less managed it. I did have to resort to some nasty parameter list trickery with a couple of methods I didn't implement.

   Perhaps a better simplicity metric might count tokens in the parsed code, rather than source characters? Also, reek by default only reports one-character and one-character-plus-digit names as 'uncommunicative', but can be configured to be stricter.

   Oh, and thanks again for running this contest!

7. Denny Abraham Says:
   October 6th, 2009 at 09:07 PM

   It's interesting to see the interplay between fit-for-purpose and principles in the source code. Out of the solutions I saw, there were definitely one or two that did not sacrifice readability for score.

8. Josh Cronemeyer Says:
   October 17th, 2009 at 07:37 AM

   Joel,

   I read your tweet about short method names running faster in jruby. My robot has some pretty long method names and I tried going down to short names of just 2 or 3 characters and there was no change in the running time of the code. I can't think of any reason why a short method name would perform better in Jruby than a long one. Do you have any idea why there would be a speedup?

Leave a Reply

Name (required)

Mail (will not be published)

Website

#