Articles Feed

A Software Craftsman's New Year's Resolutions

by: colin | December 30th, 2009 | 1 comments »

There is so much to learn in this field, and the many new languages, frameworks, and APIs that are developed every day only add to our task. Sorry for the title-bait, but no self-respecting iterative-developing programmer would make resolutions for an entire year of learning. I expect to re-evaluate every few months, but making a long list of goals is a fun way to brainstorm paths to improvement.

Let me first take a moment to reflect on the past year. I've learned so much in 2009, beginning with my discovery of the Software Craftsmanship movement and its focus on apprenticeship. I leapt at the opportunity to come to 8th Light as an apprentice under Micah Martin, and was excited to come onboard long-term at the conclusion of my formal apprenticeship. It's important to remember that only a year ago, I wasn't doing TDD or Agile and hadn't really used any other technologies professionally beyond the Rails ecosystem. This year, I've done quite a bit of Java and JRuby, and I've studied Scheme through SICP, as well as some Clojure (mostly exercise-type practice) and Scala (just enough to write a Tic-Tac-Toe program). My progress speaks both to the time and effort I invested in learning (see Uncle Bob's tweet about professionalism) and to the tremendously positive peer pressure that surrounds me here in the Chicago area, the Software Craftsmanship community, and 8th Light specifically.

However, a software developer can't be caught looking backwards for very long. With that in mind, I present my goals for 2010.

---

1. Check email and Twitter less frequently.

Like many other developers (and musicians), I have a bit of an obsessive personality, so I end up interrupting my work and personal life to get my news fix much more often than necessary. Besides, it's going to be illegal to text, email, or tweet while driving in Illinois beginning January 1, so it's absolutely necessary (not that it was safe while driving before).

2. Take more breaks when coding alone.

When programming without a pair, I tend to stay very close to the code and neglect the big picture. With some open-source work over a recent vacation, I found myself coming to an impass nearly every evening, and invariably I'd have a simple solution the following morning. It's less necessary with a pair (like Doug Bradbury) who's able to step back and evaluate bigger-picture ideas, but our minds don't have infinite endurance, even with the labor split up.

3. Timebox spike code.

I'm improving my test-driven development habits, but there are so many unfamiliar areas to me as a programmer that I have a hard time telling whether I'm going in the correct direction until I'm there. I don't write tests for most spikes, so this means that I sometimes end up with a couple hundred lines of working code that I need to retrofit tests onto. Obviously, this ends up being more work than it should, and while I can refactor to make the code more easily testable (especially with automated refactoring tools), I'd prefer to have tests in the first place. And in the worst cases, I might miss some tests that need to be written and allow bugs that should have been caught. So my intention is to put a cap on the amount of time I spend on a spike before I backtrack and start writing tests. Something like 30 minutes seems reasonable. Besides, most of the spike code I write is alone, so I should be taking more breaks anyway.

4. Focus language study.

There's something to be said for the tremendous breadth in the languages I studied this year (Java, Ruby, Scheme, Clojure, Scala, and C), but I don't want to be a jack of all trades, master of none. It's time to get serious and get some real depth. Since my current project is mostly in Java and JRuby, it seems logical to get really in-depth with those two languages. My study at home would be reinforced by what I do daily at work, and vice versa. On the other hand, it also seems smart to go after an up-and-coming language and get ahead of the curve there. So I won't rule out more Clojure or Scala study, but it's my intention to pick one at most.

5. Focus open-source efforts.

I tend to leap around open-source projects a bit, looking into a few projects just enough to catch some low-hanging fruit from the bug or feature lists. I'd like to get more depth on a project or two, ideally in the same languages I'm planning to study more of: Java and Ruby. Limelight is the no-brainer choice, and I've already started a bit of work there. The best news is that I already know Micah, the creator/maintainer, and he's very willing to help me learn and contribute to the project. I highly recommend the project, as well as Nokogiri (the Java branch would be exciting to get working) and JRuby, for anyone with Java and Ruby skills looking for a project.

6. Really learn testing frameworks well.

It's not very often that I run into a testing problem that's limited by my lack of knowledge of the testing framework, but it sure is embarrassing when that does happen. The acceptance tests for my project are in Fitnesse, and the unit tests are in RSpec and JUnit. Following my current rationale, I should get really good at those. I have the longest to go in Fitnesse, where I still feel like a beginner, so I plan to start there.

7. Schedule study / practice.

My evenings often end up being extended mashups of coding, reading, and television. It's nice to have a relaxed feel about practice, but too often, it means less quality time with my wife and dogs. I'd like to schedule my study, both to increase the quality of my work during that time and to increase the amount of quality family time.

---

I'm sure the ways to achieve these will vary, but I think they're specific enough to drive my growth as a developer. I'm excited about continuing the improvements of the past year in 2010, and I hope that by focusing on these "resolutions", I'll even accelerate things a bit. I'd love to hear other ideas people have for improving their skills in the coming year!

Up and running with TDD on Android

by: colin | July 11th, 2009 | 2 comments »

A couple of weeks ago, I happened to be in the right place at the right time (the first ORD Session) when Google hooked a bunch of developers up with an Android Dev Phone 1. I'd been interested in Android for awhile because it's a more open platform than the iPhone, and the code is Java (I've never worked in Objective-C), so I was excited. After my initial excitement of hacking around and getting things to work, I decided to regain my discipline and figure out a workflow for TDD. The good news is that JUnit is built right into the Java framework that Android app have available. The less-good news is that writing and running tests on Android isn't as well-documented as many other facets of the application framework. I'd like to share my setup, which doesn't depend on any specific IDE or text editor. I'm assuming Mac or Linux here, but I'm sure Windows would only require minimal changes. Other assumptions:

• The Android SDK is installed
• Android tools directory is on the PATH
• Ant is on the machine and on the PATH

Note: the "tools" directory is the main one, not the one below "platforms" - in my case: '/Users/colin/lib/android/tools' Let's get started by creating a project with the command-line utility, which will give you the test directory structure and build files that you need (you won't get these with the Eclipse or IntelliJ plugins, for instance). Of course, you can read documentation for the android command to see more details, but here's what we're doing:

1. $ android create project -t 1 -p tictactoe-android -a TicTacToe -k com.colinwjones
2. $ cd tictactoe-android

All of these options are required (-n, the project name, is optional, and I've left it out above)

• -t is the build target platform (1 refers to Android version1.1 on my machine, see the ids in $ android list targets for yours - this is important to remember as you'll need to refer to it later)
• -p is the path of the new project to create (make up a folder name for your new project
• -a is the first Activity that you want to create
• -k is your package name

Next, we want to create an AVD (Android Virtual Device) if one doesn't already exist - it's an emulator for the phone operating system. We can check to see if one exists already first:

1. $ android list avd

If you don't have an AVD already, create one:

1. $ android create avd -n ColinPhone -t 1

You'll be asked if you want a custom hardware profile (I don't, so I just hit Return).

• -n is the name of the new device
• -t is the target platform, as before (and it should match your project's platform)

At this point, let's go ahead and fire up the emulator, making sure to match the name to the one you created. It's good to make sure that our state at this early point is a good one.

1. $ emulator -avd ColinPhone &

Here I'm launching the emulator (AVD) in the background so I don't need to open up a new Terminal window. You'll get some output in your terminal window; if it bothers you, Ctrl-L will freshen it up. Now we'll build and install both of your apps (the test package is really a separate application, which is a good idea to keep the tests out of the eventual deployment package anyway).

1. $ ant debug
2. $ adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug
5. $ adb install -r bin/TicTacToe-debug.apk

The debug target isn't actually defined in either buildfile (build.xml or tests/build.xml), but is available nonetheless (see $ ant help for other targets you might not otherwise find). This takes care of bundling resources, compiling, converting classfiles to Android's .dex format, and packaging. Note that adb is NOT ant - it's the Android Debug Bridge, and it's invaluable for working with the emulator. The -r option to adb install reinstalls the package if necessary. Now, this is pretty redundant-looking, but just remember that the tests directory is a sort of parallel structure with your project directory, and you need both. It's time to run the default test suite that the android create project call has given us (this can be run either from tests directory or from the root of the project):

1. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

Of course, you'll want to substitute your package and activity names for the ones in my examples. It's very important to realize that you've just compiled and installed your software on the emulator and are running tests on it there. In order to do TDD, you'll need to recompile changed code and reinstall on the emulator. I hope that one day there will be a way to avoid going through the emulator each time (or that there already is one that I've been unable to find), but this is the only method I've been able to get working so far. Now we need to actually add a real test to (in my case) tests/src/com/colinwjones/TicTacToeTest.java. Here, an IDE like IntelliJ or Eclipse comes in handy, especially if you're just starting with Android and aren't sure of the methods you might want to use.

1. // with imports at top:
2. import import android.widget.Button;
3. /* some code
4. * ...
5. * ...
6. */
7. // inside your test class:
8. public void testNewGameButtonExists() throws InterruptedException
9. {
10. Button button = (Button) getActivity().findViewById(R.id.new_button);
11. assertEquals("New Game", button.getText());
12. }

Building our test package at this point will fail, since no resource with the new_button id exists yet. Let's do it anyway to see the first failure and guide us to our implementation code (running this from the tests directory):

1. $ ant debug

The error tells us where to go next: we'll implement the button in /layout/main.xml (making sure to set the right ID on the button). Since we changed the main layout, the implementation package needs to be built first, then the test package:

1. $ cd ..
2. $ ant debug && adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug && adb install -r bin/TicTacToe-debug.apk

Now run the tests again:

1. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

Great! Now we have a proper failure:

1. com.colinwjones.TicTacToeTest:
2. Failure in testNewGameButtonExists:
3. junit.framework.AssertionFailedError: expected:<new game> but was:<>

We just need to add the right text in the XML layout: Let's rid of the default "Hello, World" TextView in main.xml while we're at it. Now rebuild, reinstall, and re-run tests

1. $ cd ..
2. $ ant debug && adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug && adb install -r bin/TicTacToe-debug.apk
5. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

This is getting annoying typing all these commands: we're going to want to write a shell script or Ant task to do this for us. But for the time being, we'll plod through (that was the last time, though). Now we're passing:

1. Test results for InstrumentationTestRunner=..
2. Time: 1.009
3. OK (2 tests)

It's a bit strange that the test runner claims we have 2 tests: each test class will add one of its own. Now that we've seen how to get the tests running, let's automate it by adding Ant tasks to the build.xml in the main project directory. You'll need to set the environment variable ANDROID_TOOLS for this exact task to work, or you can provide the full path to adb.

1. <target name="clean">
2. <delete includeemptydirs="true">
3. <fileset dir="bin" includes="**/*" />
4. <fileset dir="tests/bin" includes="**/*" />
5. </delete>
6. </target>
7. <target depends="clean, debug" name="test">
8. <property environment="env" />
9. <property name="android-tools" value="${env.ANDROID_TOOLS}" />
10. <ant dir="tests" antfile="build.xml" inheritall="false" target="debug" />
11. <exec executable="${android-tools}/adb" failonerror="true">
12. <arg line="install -r bin/TicTacToe-debug.apk" />
13. </exec>
14. <exec executable="${android-tools}/adb" failonerror="true">
15. <arg line="install -r tests/bin/TicTacToe-debug.apk" />
16. </exec>
17. <exec executable="${android-tools}/adb">
18. <arg line="shell am instrument" />
19. <arg value="-w" />
20. <arg line="com.colinwjones.tests/android.test.InstrumentationTestRunner" />
21. </exec>
22. </target>

Now we can just run

1. $ ant test

from the project directory (assuming the emulator is already up and running with $ emulator -avd ColinPhone &), and we'll be good to go. Honestly, it's a pretty simple process: the key for me was in using the Android command-line tools rather than IDE plugins. It helped me to understand the build process and get beyond the initial frustration of not having the IDE do the work for me. I imagine things will change a bit for Windows users, so please leave comments if there's anything drastically different (and also if things change for Mac/Linux users as the framework develops). I do hope this will help someone else to get set up and save the headache I had when first discovering Android.

A Functional Refactoring in Scala

by: colin | June 16th, 2009 | 3 comments »

Many of us have heard a lot of talk about functional programming and its benefits, especially when it comes to highly concurrent applications where thread locking and synchronization would be necessary in order to avoid mangling state among processes. Languages like Scala, Erlang, and Clojure are increasingly coming to mind as people look for ways to take advantage of multiple-core processors without the headaches threading can create. While I'm not quite ready to take the plunge and completely eliminate state from my programming, I'm looking for more and more ways to get rid of it when possible. In doing so, I've come across several situations where a simple refactoring to a more functional style can yield rewards in readability and testability, and I'd like to talk about one of them here.

Simple while loops are one of the imperative programmer's most basic tools, and it might seem difficult to eliminate mutable state. However, a little persistence can pay off. Let's say we start with the following (admittedly simplistic) Scala code:

1. def printUpTo(limit: Int): Unit =
2. {
3. var i = 0
4. while(i <= limit)
5. {
6. println("i = " + i)
7. i += 1
8. }
9. }

It's clear that state is changing inside this function, highlighted by the use of Scala's var keyword that specifies a variable that is allowed to change (val variables are unmodifiable). While this state change isn't visible from the outside (a new copy of the variable is created each time the function is entered), var often means there's unnecessary clutter in the code and that it can be simplified. And indeed it can. First of all, any programmer worth her salt would first change this to a for loop. A Java implementation might look like this:

1. void printUpTo(int limit)
2. {
3. for(int i = 0; i <= limit, i++)
4. {
5. System.out.println("i = " + i)
6. }
7. }

Scala's for loops look more like Java's foreach syntax:

In addition to the Java-like iterating structure (left-arrow instead of :); the range specification (0 to limit) is a Scala built-in, but there are also libraries for ranges in Java, Ruby, and plenty of other languages. The code in the last example seems to more clearly encapsulate the structure of the function. The first and last lines in the while-based version of printUpTo are just boilerplate code, and while most any programmer can easily follow the structure of the code, we should always be striving for improvement. In general, I find that I'm less likely to have bugs with a smaller codebase. Certainly there are exceptions (regular expressions comes to mind), but assuming that two pieces of code both read well, I'd rather have less to read.

In the preceding Scala for loop syntax, i is implicitly a val within the context of the loop, which means it can't be reassigned to a new value. This is a great step forward, as we could initially have created an infinite loop by manipulating the value of i inside the while loop. We have now insulated ourselves against this type of change.

Another functional way to write essentially the same loop would be to use Scala's foreach, a method available on Lists and Ranges that takes a function as a parameter.

1. def printUpTo(limit: Int): Unit = {
2. (0 to limit).foreach {
3. i => println("i = " + i)
4. }
5. }

This seems more expressive to me, as it emphasizes the importance of the range object in the loop. It would also have given us the ability to use a shorthand if we'd not needed the "i =" text: (0 to limit).foreach(println _), or even (0 to limit).foreach(println). But there is also something to be said for the familiarity of the more traditional for construct.

The code above is still not completely functional, however, because the println statement depends upon the definition of standard output at the time (imagine redefining it to a java.io.ByteArrayOutputStream, for example). This also makes the function hard to test. Now, if we're going to build an application to work at the console, at some point we're going to have to actually use println if we want the user to know that the application is working. Does this mean that the functional approach completely eschews input and output streams? Certainly not, but we have two problems in the example at hand:

1. printUpTo is responsible for both the construction of each member of a group of output strings, and for the actual printing of those strings. Therefore, the function has two reasons to change. For instance, we might want to change the information that gets printed on each line (e.g. "z = " + i) or its formatting (e.g. two newlines between each line of text). We might also want to change how the output gets to the user (e.g. a GUI such as Swing). This problem is akin to a Single Responsibility Principle violation, but on the function level rather than the class level.
2. printUpTo is awkward to test. In order to verify that the correct output comes goes to the user, we'll need to either find a mocking library to change the way the Console object behaves, or redirect standard output to another stream that we can read to verify its contents afterwards (maybe a ByteArrayOutputStream).

I'd prefer a further refactoring in order to alleviate these issues. The problem of verifying console output in our test code won't be completely eliminated, but we'll at least isolate it to its own test. Here, we're extracting println from the string-building part of the code.

1. def printUpTo(limit: Int): Unit = {
2. applyUpTo(limit, println)
3. }
4. def applyUpTo(limit: Int, action: String => Unit): Unit = {
5. (0 to limit).foreach {
6. i => action("i = " + i)
7. }
8. }

Now we can test applyUpTo by passing our own custom action in, and tests for println-based behavior can be isolated, so that we isolate some of the necessary mocking and redirection. As a word of explanation, the action passed in needs to take a String as an argument and return the Unit value: (). Now we can also move printUpTo to a different class if we so desire (assuming this function is part of a larger application), eliminating our need to recompile the code in printUpTo each time the string format changes. I'd most likely put the applyUpTo call on its own in a main() method if there's really nothing more to the application.

There is another problem with the code above, however: both of the functions are returning Unit values (the equivalent of Java's void). This means that both of these functions rely on side effects to get their jobs done. If we were to pass a side effect-free function into applyUpTo, we wouldn't actually have any output from the applyUpTo to tell us that we did something. A side effect is any effect of a function beyond its return value, so we can say that if a Scala function has a return type of Unit, there are either side effects, or it is a trivial function that we can safely eliminate. We can make the helper function side effect-free like so:

1. def printUpTo(limit: Int): Unit = {
2. buildUpTo(limit).foreach(println)
3. }
4. def buildUpTo(limit: Int): Iterable[String] = {
5. (0 to limit).map("i = " + _)
6. }

In this case, we're relying on a bit of syntactic sugar for transferring the iterating value of foreach to println, and a placeholder (_) for map's iterating value. The buildUpTo function is now more functional and more easily testable (we can do a fairly simple comparison with the returned Iterable), but it's important to note that memory usage might be a concern, as a map of all the values could become pretty large. Another issue with this is speed: in the more functional version, we're iterating through the collection twice rather than just once. We don't often get things for free, and indeed there is a trade-off here between memory/speed and functionalness.

The example here has been on a very small scale, and perhaps a bit contrived, but we can extrapolate the ideas to a wide range of refactorings. We want to remove as much code with side effects and mutable state as we can from the rest of our application. This way we can divide up the functional parts of our application among processors in order to take advantage of the multiple cores that we all have in our machines these days. There are numerous other situations in which functional programming can lead to cleaner code, and there are also places where it might be inappropriate or impossible. It's important to remember the most common reason we might want to write functional code: concurrency. If I'm only going to be running this small application on my local machine, from the command line, and printing is going to be the limiting factor speed-wise (and it probably will in this case), I would most likely stop at the simplest foreach version above and be done with it. It will be efficient with memory and fast. But when the function buildUpTo becomes more complex, or we need to run numerous instances of the application, that's where concurrency, and functional programming, are best applied. Our goal should be to find, and use, the right tool for the job.

###