Articles Feed

Craftsman Swap at Relevance - Day 4

by: colin | July 9th, 2010 | 0 comments »

Today was my last day at Relevance, pairing again with Stuart Sierra. Along with Alan Dipert, we arrived at a solution for our data woes from yesterday, and deployed it. The approach we chose exposed a problem with data interpretation when a Clojure-precondition-thrown AssertionError came back to bite us later in the day. Note to self: AssertionError, which preconditions throw, is a java.lang.Error, not a java.lang.Exception. Catch it accordingly, especially if you're running these preconditions in a thread pool!

Today may have been the most fast-paced of my time here. In the morning after the project standup, Alan did some whiteboarding with Stuart and me to explain the architecture and requirements of a spike that we spent the rest of the day on. It's going to be a pretty cool system that integrates with the rest of the client system. The interesting bits include writing to Redis (potentially with a C application) and a web service (in Clojure) that reads from Redis server to provide data to end users.

So Stuart and I started from scratch on a new Compojure app with Clojure 1.2, Leiningen, and redis-clojure. We got set up relatively quickly, and implemented a tiny web service that parses a request to find what to read from Redis, does all the proper error handling, and returns a reasonable result. We had ideas about providing several types of web services (HTML, JSON, XML, perhaps even plain text), but for the purposes of the spike, we just used the browser.

I had fully expected Stuart's knowledge of Clojure to be impressive, and I wasn't disappointed. I didn't see too many things I hadn't seen before, but having seen things is different than knowing just when to use them. There was a new macro, condp, that I hadn't seen before, and it seems like a great substitute for cond in situations where all the predicates look very similar. And if you've ever been frustrated by needing to restart a Clojure process in order to add a jar to the classpath, you'll be excited to know that Stuart is working on an interesting solution involving Nailgun.

It's also been cool to hear a couple of questions Stuart Halloway asked us about Clojure idioms (he kindly included me in the questions, which were really for Stuart Sierra). It's sometimes easy to forget that everybody's human, and the smartest people will often ask for help rather than forge ahead as a lone warrior, and they're better off for it - even when it's just rubber ducking.

On that note, my experience here has been that almost everyone seems to be pairing all the time, and those who aren't pairing are either working on project management or QA-type tasks, or have their code reviewed by someone else later on. My own experience at 8th Light has always been that two heads are better than one, so this was cool to see. Additionally, as I've said before, the developers all seem to be very good at client-relations issues, and I discovered today that, as I suspected, Relevance uses its own developers as project managers for all projects. This seems very noteworthy to me - it's been rare that I've seen a developer as project manager, but it seems to be the rule here. This is also often the case for the QA role. Certainly there is a lot of testing going on here at both the unit and acceptance test level, but there is often a developer doing QA on other developers' work.

My experience this week was great. I would highly recommend anyone considering doing something like a craftsman swap / journeyman pairing tour to just do it. This has been a wonderful experience, and it's easy to see why people like Corey Haines have been such proponents of this sort of thing. Relevance would be an excellent choice for a visit if you can make it happen - I've learned a lot, and I look forward to bringing more of that back to the team at 8th Light when I get back to Chicagoland.

Many thanks to 8th Light and Relevance for the opportunity to do this!

Craftsman Swap at Relevance - Day 3

by: colin | July 8th, 2010 | 0 comments »

Day 3 at Relevance began, as the previous days, with the company standup. Stuart Sierra, of clojure.test and Practical Clojure fame, was my pair for the day. Yep, there are two Stuarts at Relevance with Clojure books! If you're interested in Clojure, and you haven't already purchased the Apress book, electronic or print, you should do that.

We started off with some Bash scripting for a cron job. Mostly pretty straightforward stuff, though I did really like the way the structure and logging behavior in the script turned out. I won't go so far as to say that this (or any) Bash script is a thing of beauty, but it's very clear, concise, and is going to do the job very well. One cool thing I hadn't realized was possible (and Stuart mentioned that a decent amount configuration is necessary for this) is that we used using sendmail to, well, send mail - but directly from the command line. So we had something along these lines:

  message=`cat email_file_contents.txt`
  sendmail -t $message

Our needs weren't very complex with regard to things like email headers, so it really was dirt simple to use, which was a very pleasant surprise.

We worked in Emacs, which was a bit of a struggle for me, but a reminder that I ought to be able to get around a bit better in that editor, as I'm interested in Clojure and many people in the community are gung-ho about it. org-mode was one notable takeaway there - we used the tables there to structure our table-based tests, and it can do some pretty cool formatting. I'll also say in Emacs' favor that users and authors sure are serious about documentation (just look at the previous link or the GNU Emacs Manuals), which I love to see. However, it can be a bit intimidating when you feel like you have to read a few books to get to know the editor.

At the project standup with Stuart, Alan Dipert, and Shay Frendt, the client reported a problem with a story that's still in progress, so Stuart and I spent the remainder of the day chasing this bug. There were several bright spots, however: we added several higher-level tests around some data-translation layers of the system (which includes JRuby on Rails and Clojure applications). Our tests were on the Clojure side of things, which was great to get a chance to see. Stuart had this great idea to write table-based tests a la Cucumber or Fitnesse, and we wrote a quick and dirty parser for a syntax similar to Cucumber's, plumbed a few tests in, and found one of our problems after a bit of data correction. We found the other, more mysterious problem soon afterward, and spent most of the rest of the day trying to track down the exact reason for it. It's interesting that neither of these problems had much to do with code - they were both problems with data. In one case, we received data in a format we didn't expect, and in the other, a subset of data suddenly went missing. I generally feel like good practices like TDD and pair programming can save you from many unexpected errors, but I don't know that there's a good solution for cases where data is mucked up, besides deploying, seeing what real data looks like, and adapting based on those results. I would suggest something like Haskell's QuickCheck (and similar ideas elsewhere), to try out more possibilities in the tests, but even if we used something like that here, I have a feeling we would have still ended up with an incorrect constraint, so wouldn't have caught these problems.

These problems make me wonder if, as external data drives more and more of a given application, it becomes more and more important for the data itself to be tested. That is, it would be ideal to be able to write tests that make assertions against the structure and contents of the external data. I'm not sure whether that would have been possible in this case, due to privacy concerns, but it's an idea that I'd love to hear feedback on anyway. And now that I imagine such a test system more carefully, maybe that's just design by contract in a different light?

Looking forward to tomorrow!

Craftsman Swap at Relevance - Day 2

by: colin | July 7th, 2010 | 4 comments »

Today was my second at Relevance for our craftmanship swap (see my first post for the goods on Day 1). Chris and I continued work on the project from yesterday. I was glad to have a chance to continue the story work we'd started, and see it through to completion by this afternoon, eliminating a few timezone bugs along the way (I'll get back to that).

We started the morning off with the company-wide stand-up meeting again, followed by a retrospective for the project I was pairing on. Chris explained that every iteration on a given project, Relevance does either a retrospective or a risk assessment meeting. After some scheduling and telephone difficulties on the client's side were (mostly) sorted out, Jason kicked us off as the retrospective facilitator. I haven't been on a project that held retrospectives on a regular basis before (though we've certainly done them at times), so it was interesting to see the very structured approach Relevance takes. As a brief outline, the team (and customers) began by outlining their expectations and hopes for the retrospective, followed by ratings of how several areas of the project (communication, ROI, building the right thing, etc.) were going, and finally a review and a set of action items with due dates attached. The team members - Chris, Jason, Don Mullen, and Michael Parenteau - were all excellent at interacting with the customers. I like to think that I'm pretty good at that kind of interaction, but there's a lot I picked up in a short time on that conference call. I heard a lot of rephrasing the customer's concerns to verify that the team always had the correct interpretation, and I'm sure the customer came away feeling like their needs were really being addressed, especially with the clear action items to address any issues that came up. Michael's design work on this project, incidentally, is pretty killer and got rave reviews from both of the client representatives.

In yesterday's installment, I neglected to mention the project management tool the teams use, Mingle. It's a very nice tool, which looks to have a ton of features, but (unlike some other many-featured products I've used) manages to keep the interface very clean and simple. I'm not sure in what situations I'd feel compelled to switch to it over something nice (and free) like Pivotal Tracker or Agile Zen, but it is an impressive piece of software that I'm glad to have had a chance to take a look at.

So... time zones. Every time I deal with them it's a mind-bending struggle to keep all the concerns in their proper place, and today was no exception. We had our Time.zone in Rails set to UTC, but were still getting some EST times. Well, it turns out things like Date.today.beginning_of_day don't respect that setting and use whatever the default system time zone is. After about an hour of investigations and pondering, we eventually decided to replace most uses of Date with the Rails TimeWithZone object, allowing some mildly unsavory naming with the understanding that the decision is going to need to be revisited to account for various user time zones anyway. It seems to me that this is a universal problem for apps that might be used in different time zones. Chris suggested that this might add to the often-quoted-but-rarely-sourced list of hard problems in computer science: cache invalidation and naming* (and off-by-one errors?). So henceforth, time zones are in there too. That's right, you heard it here first: it's three (or four, depending on if you're telling the off-by-one joke) hard problems: cache invalidation, naming, and time zones.

After work Aaron Bedra, Alex Warr, Stuart Halloway, and Stuart Sierra introduced me to my first draft-style game of Magic the Gathering. Yes, that's right, I spent the evening with three of the Clojure/core guys and learned a ton... about Magic. It was fun - I got a TON of help from everybody, especially Alex, and with a lot of help, I actually won a few rounds. We played duels, best-of-3, and I went 2-2 overall in games. And by "I", I mostly mean "the people who told me what I should put down." I heard a rumor on one end of the table that Relevance prefers to hire Magic players over non-players as programmers**, so potential applicants should take note. I don't know that I'll run out and buy a deck, but it's always fun to learn a new game, especially when the cards are named things like "Lord of Shatterskull Pass".

Sounds like I'll be sitting in with Alan Dipert's team tomorrow on some Clojure work, which I'm very much looking forward to.

* Some people on the Internet say the credit belongs to Phil Karlton, so it must be true.

** This sounds made up, and may be.

[UPDATE: fixed retrospective timing description based on Jason's comments below]

Craftsman Swap at Relevance - Day 1

by: colin | July 7th, 2010 | 2 comments »

I'm both honored and humbled to represent 8th Light on the latest Craftsmanship Swap, with Relevance in Durham, NC. I'd first heard about Relevance through Stuart Halloway's excellent book Programming Clojure, and their reputation precedes them in the Clojure community, as well as in the Ruby/Rails world. Jason Rudolph had me come in a few minutes early this morning to give me a quick introduction to the company before their company-wide standup, where the whole company gathered and very efficiently went through the interesting goings-on of the day. The plan for now is for me to spend a couple days working in Ruby/Rails, and a couple on Clojure - I'm excited about both.

I paired up with Chris Redinger to work on a Rails project that's just a few weeks old, but already doing some pretty cool stuff. There's a pretty slick graphing library on the project called Highcharts, which apparently even works on IE6 (gasp!). We did some refactoring, which was a pleasure - Chris has a great eye (nose?) for code smells and writing excellent code (as well as customer relationships), and I often felt like we were on the same wavelength as we made improvements. We ran into a strange issue with the RSpec version we were on: models seem to be loaded twice during our spec runs, which gave us annoying errors about redefining constants (when we were most definitely only defining them once). It turned out the right place for the (non-ActiveRecord) models was actually under lib/ in this case, and that cleared up our problem. Chris thought this may have already been fixed in the latest RSpec 2 beta, but unfortunately we can't upgrade because newer versions are only compatible with Rails 3, which we're not using on this project.

We also got some feature work done, of course. The code is a pleasure to work in - clean, plenty of tests, and several Thundercats references, which is perhaps most important of all. In all seriousness, it was pretty easy to jump in and contribute, and that's been due in large part to the high quality of the code. Bundler, by the way, seems awesome. I've heard mixed reviews, but my experience installing gems today was such a breeze that I'm definitely going to try to use it on my next Rails project. I also sometimes forget how freeing it is to work on a project using Git - all of our greenfield projects use it, but I've gotten used to being on Subversion projects, where making a branch is a big production. Chris uses feature branches regularly, and they seem to work pretty well.

The environment at Relevance is really nice - a big open space with lots of pairing stations, drinks and snacks, and even a keg of Aaron Bedra's delicious home-brewed Belgian-style wheat beer on tap. We had a great lunch of Thai food and very techy conversation - Node.js, Clojure, and EventMachine all got some playing time, and everyone agreed that Aquamacs was very pretty. Wait, maybe that last part went differently... After work, Aaron, his fiancee, Stuart Sierra, Jared Pace and I grabbed some Mexican food (after what was apparently a second failed attempt to eat at a place that serves something called a "Defibralator" - don't blame me, that's how they spell it on the online menu).

Chris and I ended the day working on tracking down a data issue, which we hadn't completely solved by the end of the day, but we narrowed the possible culprits down significantly, and I'm sure it'll be obvious to us in the morning. Looking forward to tomorrow's adventures!

JRuby Helps Us Craft Quality Software

by: doug | April 6th, 2010 | 0 comments »

"It's kind of a Frankenstein." We've used this phrase more than a few times to describe the architecture and technology used to design a product that was recently released after almost a year of development by the software craftsmen of 8th Light. The product is a multi-platform, PC management application for library computer labs written primarily in Ruby, run on the JVM with JRuby. The user interface is written in erb-templated HTML and Javascript, and is displayed in an embedded web browser. JNA (Java Native Access) is used to call operating system specific functions that enable an insane amount of control over the desktop. The various JRuby components communicate with each other over DRb.

But the most amazing thing about this project is that we are using JRuby to deliver a portable, rich desktop app using the best of web technologies. We chose JRuby because it increased portability and performance and reduced development time.

At 8th Light we had tried before to build products for our clients using Ruby, but we always hit a dead end when it came to deployment. For our clients who provide software products instead of software services, the applications must install easily and completely on the end customers machines while still protecting the intellectual property of our client. We tried very hard to make technologies like RubyScript2exe work, but in the end a packing and unpacking solution was too slow and still left our client's source code open for the reading on the installed computer.

As we began the project about a year ago, we discovered that JRuby had reached the point where it could be a viable platform for us to be able to build products that our client's customers could easily deploy in their own environments. Our solution was to bundle all of our JRuby code into jars after it had been encrypted. By patching into the JRuby load service we were able to decrypt the source on the fly and leave the code bundled and encrypted on the disk at all times.

JRuby allows us dirt simple access to Java's huge collection of libraries, along with plenty of custom Java libraries our client had already written. We also go back the other way, implementing Java interfaces in JRuby and using those objects in Java code. This fluidity allows us to always pick the best tool for the job. Early in the project, this also helped to sell the technology to our client. They were wary of a new language that none of their in-house developers knew, but we were able to reassure them that they could easily drop back into Java at any time.

At one point in the project, the client's development team jumped on the project to add a new configuration interface to the server side of the application. The team wanted to use a set of Java tools that they were already familiar with, but needed to make calls into the JRuby side of the application to get and set the configuration values. We built a new Java interface for them and implemented it in JRuby—then from our Ruby initialization code, called a static Java function back on the Main class, passing in the configuration object. This allowed the client's team to initialize their java objects with the instance of our JRuby class that we had provided.

1. //The Java Interface
2. public interface ConfigurationStore
3. {
4. void put(Map<string> pairs);
5. void put(String key, String value);
6. String get(String name);
7. }

1. #The Ruby Implementation
2. class ConfigurationStore
3. include Java::package.ConfigurationStore
4. def get(key)
5. .....
6. end
7. # Notice how we were able to implement the function
8. # overloading present in the interface by
9. # using a variable arguments in one ruby method.
10. def put(*args)
11. if args.size == 2
12. ....
13. elsif args.size == 1
14. .....
15. else
16. raise ArgumentError.new("put() takes a Hash
17. (or a Java Map) or a key and value pair")
18. end
19. end
20. end

We wanted to be extra sure that the class we were writting in Ruby would work from Java before we handed it over, so in addition to the rspecs we wrote to test the Ruby class, we also wanted to write a JUnit Test. Normally, we inject a JRuby object into Java by calling a Java function from a JRuby method and passing along the JRuby object that is to be used. In this case we wanted to actually instantiate the Ruby object in the setup of a JUnit test. We dug around in the JRuby APIs and were able to leverage the fact that the RubyRuntimeAdapter returns the last thing evaluated. We were able to use the toJava() method on the IRubyObject returned from eval and cast the result into the interface class. Then we were free to write JUnit tests exercising the functionality of the JRuby class.

1. // instantiating a Ruby object in Java
2. RubyInstanceConfig config = new RubyInstanceConfig();
3. Ruby runtime = JavaEmbedUtils.initialize(new ArrayList(), config);
4. RubyRuntimeAdapter evaler = JavaEmbedUtils.newRuntimeAdapter();
5. IRubyObject result = evaler.eval(runtime, "require 'configuration_store';" +
6. "config_store = ConfigurationStore.new");
7. store = (ConfigurationStore) result.toJava(ConfigurationStore.class);

We do a lot of things at 8th Light to ensure that we can deliver very high quality code to our clients as fast as is responsible. One of the most important things we do is work in highly productive languages. JRuby enables us to write most of the complicated business logic of our application in the highly productive Ruby language and still provide a viable deployment scheme that allows our clients to successfully sell their product to their customers.

This article appears in the April 6th, 2010 Engine Yard Newsletter

A Software Craftsman's New Year's Resolutions

by: colin | December 30th, 2009 | 1 comments »

There is so much to learn in this field, and the many new languages, frameworks, and APIs that are developed every day only add to our task. Sorry for the title-bait, but no self-respecting iterative-developing programmer would make resolutions for an entire year of learning. I expect to re-evaluate every few months, but making a long list of goals is a fun way to brainstorm paths to improvement.

Let me first take a moment to reflect on the past year. I've learned so much in 2009, beginning with my discovery of the Software Craftsmanship movement and its focus on apprenticeship. I leapt at the opportunity to come to 8th Light as an apprentice under Micah Martin, and was excited to come onboard long-term at the conclusion of my formal apprenticeship. It's important to remember that only a year ago, I wasn't doing TDD or Agile and hadn't really used any other technologies professionally beyond the Rails ecosystem. This year, I've done quite a bit of Java and JRuby, and I've studied Scheme through SICP, as well as some Clojure (mostly exercise-type practice) and Scala (just enough to write a Tic-Tac-Toe program). My progress speaks both to the time and effort I invested in learning (see Uncle Bob's tweet about professionalism) and to the tremendously positive peer pressure that surrounds me here in the Chicago area, the Software Craftsmanship community, and 8th Light specifically.

However, a software developer can't be caught looking backwards for very long. With that in mind, I present my goals for 2010.

---

1. Check email and Twitter less frequently.

Like many other developers (and musicians), I have a bit of an obsessive personality, so I end up interrupting my work and personal life to get my news fix much more often than necessary. Besides, it's going to be illegal to text, email, or tweet while driving in Illinois beginning January 1, so it's absolutely necessary (not that it was safe while driving before).

2. Take more breaks when coding alone.

When programming without a pair, I tend to stay very close to the code and neglect the big picture. With some open-source work over a recent vacation, I found myself coming to an impass nearly every evening, and invariably I'd have a simple solution the following morning. It's less necessary with a pair (like Doug Bradbury) who's able to step back and evaluate bigger-picture ideas, but our minds don't have infinite endurance, even with the labor split up.

3. Timebox spike code.

I'm improving my test-driven development habits, but there are so many unfamiliar areas to me as a programmer that I have a hard time telling whether I'm going in the correct direction until I'm there. I don't write tests for most spikes, so this means that I sometimes end up with a couple hundred lines of working code that I need to retrofit tests onto. Obviously, this ends up being more work than it should, and while I can refactor to make the code more easily testable (especially with automated refactoring tools), I'd prefer to have tests in the first place. And in the worst cases, I might miss some tests that need to be written and allow bugs that should have been caught. So my intention is to put a cap on the amount of time I spend on a spike before I backtrack and start writing tests. Something like 30 minutes seems reasonable. Besides, most of the spike code I write is alone, so I should be taking more breaks anyway.

4. Focus language study.

There's something to be said for the tremendous breadth in the languages I studied this year (Java, Ruby, Scheme, Clojure, Scala, and C), but I don't want to be a jack of all trades, master of none. It's time to get serious and get some real depth. Since my current project is mostly in Java and JRuby, it seems logical to get really in-depth with those two languages. My study at home would be reinforced by what I do daily at work, and vice versa. On the other hand, it also seems smart to go after an up-and-coming language and get ahead of the curve there. So I won't rule out more Clojure or Scala study, but it's my intention to pick one at most.

5. Focus open-source efforts.

I tend to leap around open-source projects a bit, looking into a few projects just enough to catch some low-hanging fruit from the bug or feature lists. I'd like to get more depth on a project or two, ideally in the same languages I'm planning to study more of: Java and Ruby. Limelight is the no-brainer choice, and I've already started a bit of work there. The best news is that I already know Micah, the creator/maintainer, and he's very willing to help me learn and contribute to the project. I highly recommend the project, as well as Nokogiri (the Java branch would be exciting to get working) and JRuby, for anyone with Java and Ruby skills looking for a project.

6. Really learn testing frameworks well.

It's not very often that I run into a testing problem that's limited by my lack of knowledge of the testing framework, but it sure is embarrassing when that does happen. The acceptance tests for my project are in Fitnesse, and the unit tests are in RSpec and JUnit. Following my current rationale, I should get really good at those. I have the longest to go in Fitnesse, where I still feel like a beginner, so I plan to start there.

7. Schedule study / practice.

My evenings often end up being extended mashups of coding, reading, and television. It's nice to have a relaxed feel about practice, but too often, it means less quality time with my wife and dogs. I'd like to schedule my study, both to increase the quality of my work during that time and to increase the amount of quality family time.

---

I'm sure the ways to achieve these will vary, but I think they're specific enough to drive my growth as a developer. I'm excited about continuing the improvements of the past year in 2010, and I hope that by focusing on these "resolutions", I'll even accelerate things a bit. I'd love to hear other ideas people have for improving their skills in the coming year!

Up and running with TDD on Android

by: colin | July 11th, 2009 | 2 comments »

A couple of weeks ago, I happened to be in the right place at the right time (the first ORD Session) when Google hooked a bunch of developers up with an Android Dev Phone 1. I'd been interested in Android for awhile because it's a more open platform than the iPhone, and the code is Java (I've never worked in Objective-C), so I was excited. After my initial excitement of hacking around and getting things to work, I decided to regain my discipline and figure out a workflow for TDD. The good news is that JUnit is built right into the Java framework that Android app have available. The less-good news is that writing and running tests on Android isn't as well-documented as many other facets of the application framework. I'd like to share my setup, which doesn't depend on any specific IDE or text editor. I'm assuming Mac or Linux here, but I'm sure Windows would only require minimal changes. Other assumptions:

• The Android SDK is installed
• Android tools directory is on the PATH
• Ant is on the machine and on the PATH

Note: the "tools" directory is the main one, not the one below "platforms" - in my case: '/Users/colin/lib/android/tools' Let's get started by creating a project with the command-line utility, which will give you the test directory structure and build files that you need (you won't get these with the Eclipse or IntelliJ plugins, for instance). Of course, you can read documentation for the android command to see more details, but here's what we're doing:

1. $ android create project -t 1 -p tictactoe-android -a TicTacToe -k com.colinwjones
2. $ cd tictactoe-android

All of these options are required (-n, the project name, is optional, and I've left it out above)

• -t is the build target platform (1 refers to Android version1.1 on my machine, see the ids in $ android list targets for yours - this is important to remember as you'll need to refer to it later)
• -p is the path of the new project to create (make up a folder name for your new project
• -a is the first Activity that you want to create
• -k is your package name

Next, we want to create an AVD (Android Virtual Device) if one doesn't already exist - it's an emulator for the phone operating system. We can check to see if one exists already first:

1. $ android list avd

If you don't have an AVD already, create one:

1. $ android create avd -n ColinPhone -t 1

You'll be asked if you want a custom hardware profile (I don't, so I just hit Return).

• -n is the name of the new device
• -t is the target platform, as before (and it should match your project's platform)

At this point, let's go ahead and fire up the emulator, making sure to match the name to the one you created. It's good to make sure that our state at this early point is a good one.

1. $ emulator -avd ColinPhone &

Here I'm launching the emulator (AVD) in the background so I don't need to open up a new Terminal window. You'll get some output in your terminal window; if it bothers you, Ctrl-L will freshen it up. Now we'll build and install both of your apps (the test package is really a separate application, which is a good idea to keep the tests out of the eventual deployment package anyway).

1. $ ant debug
2. $ adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug
5. $ adb install -r bin/TicTacToe-debug.apk

The debug target isn't actually defined in either buildfile (build.xml or tests/build.xml), but is available nonetheless (see $ ant help for other targets you might not otherwise find). This takes care of bundling resources, compiling, converting classfiles to Android's .dex format, and packaging. Note that adb is NOT ant - it's the Android Debug Bridge, and it's invaluable for working with the emulator. The -r option to adb install reinstalls the package if necessary. Now, this is pretty redundant-looking, but just remember that the tests directory is a sort of parallel structure with your project directory, and you need both. It's time to run the default test suite that the android create project call has given us (this can be run either from tests directory or from the root of the project):

1. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

Of course, you'll want to substitute your package and activity names for the ones in my examples. It's very important to realize that you've just compiled and installed your software on the emulator and are running tests on it there. In order to do TDD, you'll need to recompile changed code and reinstall on the emulator. I hope that one day there will be a way to avoid going through the emulator each time (or that there already is one that I've been unable to find), but this is the only method I've been able to get working so far. Now we need to actually add a real test to (in my case) tests/src/com/colinwjones/TicTacToeTest.java. Here, an IDE like IntelliJ or Eclipse comes in handy, especially if you're just starting with Android and aren't sure of the methods you might want to use.

1. // with imports at top:
2. import import android.widget.Button;
3. /* some code
4. * ...
5. * ...
6. */
7. // inside your test class:
8. public void testNewGameButtonExists() throws InterruptedException
9. {
10. Button button = (Button) getActivity().findViewById(R.id.new_button);
11. assertEquals("New Game", button.getText());
12. }

Building our test package at this point will fail, since no resource with the new_button id exists yet. Let's do it anyway to see the first failure and guide us to our implementation code (running this from the tests directory):

1. $ ant debug

The error tells us where to go next: we'll implement the button in /layout/main.xml (making sure to set the right ID on the button). Since we changed the main layout, the implementation package needs to be built first, then the test package:

1. $ cd ..
2. $ ant debug && adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug && adb install -r bin/TicTacToe-debug.apk

Now run the tests again:

1. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

Great! Now we have a proper failure:

1. com.colinwjones.TicTacToeTest:
2. Failure in testNewGameButtonExists:
3. junit.framework.AssertionFailedError: expected:<new game> but was:<>

We just need to add the right text in the XML layout: Let's rid of the default "Hello, World" TextView in main.xml while we're at it. Now rebuild, reinstall, and re-run tests

1. $ cd ..
2. $ ant debug && adb install -r bin/TicTacToe-debug.apk
3. $ cd tests
4. $ ant debug && adb install -r bin/TicTacToe-debug.apk
5. $ adb shell am instrument -w com.colinwjones.tests/android.test.InstrumentationTestRunner

This is getting annoying typing all these commands: we're going to want to write a shell script or Ant task to do this for us. But for the time being, we'll plod through (that was the last time, though). Now we're passing:

1. Test results for InstrumentationTestRunner=..
2. Time: 1.009
3. OK (2 tests)

It's a bit strange that the test runner claims we have 2 tests: each test class will add one of its own. Now that we've seen how to get the tests running, let's automate it by adding Ant tasks to the build.xml in the main project directory. You'll need to set the environment variable ANDROID_TOOLS for this exact task to work, or you can provide the full path to adb.

1. <target name="clean">
2. <delete includeemptydirs="true">
3. <fileset dir="bin" includes="**/*" />
4. <fileset dir="tests/bin" includes="**/*" />
5. </delete>
6. </target>
7. <target depends="clean, debug" name="test">
8. <property environment="env" />
9. <property name="android-tools" value="${env.ANDROID_TOOLS}" />
10. <ant dir="tests" antfile="build.xml" inheritall="false" target="debug" />
11. <exec executable="${android-tools}/adb" failonerror="true">
12. <arg line="install -r bin/TicTacToe-debug.apk" />
13. </exec>
14. <exec executable="${android-tools}/adb" failonerror="true">
15. <arg line="install -r tests/bin/TicTacToe-debug.apk" />
16. </exec>
17. <exec executable="${android-tools}/adb">
18. <arg line="shell am instrument" />
19. <arg value="-w" />
20. <arg line="com.colinwjones.tests/android.test.InstrumentationTestRunner" />
21. </exec>
22. </target>

Now we can just run

1. $ ant test

from the project directory (assuming the emulator is already up and running with $ emulator -avd ColinPhone &), and we'll be good to go. Honestly, it's a pretty simple process: the key for me was in using the Android command-line tools rather than IDE plugins. It helped me to understand the build process and get beyond the initial frustration of not having the IDE do the work for me. I imagine things will change a bit for Windows users, so please leave comments if there's anything drastically different (and also if things change for Mac/Linux users as the framework develops). I do hope this will help someone else to get set up and save the headache I had when first discovering Android.

A Functional Refactoring in Scala

by: colin | June 16th, 2009 | 3 comments »

Many of us have heard a lot of talk about functional programming and its benefits, especially when it comes to highly concurrent applications where thread locking and synchronization would be necessary in order to avoid mangling state among processes. Languages like Scala, Erlang, and Clojure are increasingly coming to mind as people look for ways to take advantage of multiple-core processors without the headaches threading can create. While I'm not quite ready to take the plunge and completely eliminate state from my programming, I'm looking for more and more ways to get rid of it when possible. In doing so, I've come across several situations where a simple refactoring to a more functional style can yield rewards in readability and testability, and I'd like to talk about one of them here.

Simple while loops are one of the imperative programmer's most basic tools, and it might seem difficult to eliminate mutable state. However, a little persistence can pay off. Let's say we start with the following (admittedly simplistic) Scala code:

1. def printUpTo(limit: Int): Unit =
2. {
3. var i = 0
4. while(i <= limit)
5. {
6. println("i = " + i)
7. i += 1
8. }
9. }

It's clear that state is changing inside this function, highlighted by the use of Scala's var keyword that specifies a variable that is allowed to change (val variables are unmodifiable). While this state change isn't visible from the outside (a new copy of the variable is created each time the function is entered), var often means there's unnecessary clutter in the code and that it can be simplified. And indeed it can. First of all, any programmer worth her salt would first change this to a for loop. A Java implementation might look like this:

1. void printUpTo(int limit)
2. {
3. for(int i = 0; i <= limit, i++)
4. {
5. System.out.println("i = " + i)
6. }
7. }

Scala's for loops look more like Java's foreach syntax:

In addition to the Java-like iterating structure (left-arrow instead of :); the range specification (0 to limit) is a Scala built-in, but there are also libraries for ranges in Java, Ruby, and plenty of other languages. The code in the last example seems to more clearly encapsulate the structure of the function. The first and last lines in the while-based version of printUpTo are just boilerplate code, and while most any programmer can easily follow the structure of the code, we should always be striving for improvement. In general, I find that I'm less likely to have bugs with a smaller codebase. Certainly there are exceptions (regular expressions comes to mind), but assuming that two pieces of code both read well, I'd rather have less to read.

In the preceding Scala for loop syntax, i is implicitly a val within the context of the loop, which means it can't be reassigned to a new value. This is a great step forward, as we could initially have created an infinite loop by manipulating the value of i inside the while loop. We have now insulated ourselves against this type of change.

Another functional way to write essentially the same loop would be to use Scala's foreach, a method available on Lists and Ranges that takes a function as a parameter.

1. def printUpTo(limit: Int): Unit = {
2. (0 to limit).foreach {
3. i => println("i = " + i)
4. }
5. }

This seems more expressive to me, as it emphasizes the importance of the range object in the loop. It would also have given us the ability to use a shorthand if we'd not needed the "i =" text: (0 to limit).foreach(println _), or even (0 to limit).foreach(println). But there is also something to be said for the familiarity of the more traditional for construct.

The code above is still not completely functional, however, because the println statement depends upon the definition of standard output at the time (imagine redefining it to a java.io.ByteArrayOutputStream, for example). This also makes the function hard to test. Now, if we're going to build an application to work at the console, at some point we're going to have to actually use println if we want the user to know that the application is working. Does this mean that the functional approach completely eschews input and output streams? Certainly not, but we have two problems in the example at hand:

1. printUpTo is responsible for both the construction of each member of a group of output strings, and for the actual printing of those strings. Therefore, the function has two reasons to change. For instance, we might want to change the information that gets printed on each line (e.g. "z = " + i) or its formatting (e.g. two newlines between each line of text). We might also want to change how the output gets to the user (e.g. a GUI such as Swing). This problem is akin to a Single Responsibility Principle violation, but on the function level rather than the class level.
2. printUpTo is awkward to test. In order to verify that the correct output comes goes to the user, we'll need to either find a mocking library to change the way the Console object behaves, or redirect standard output to another stream that we can read to verify its contents afterwards (maybe a ByteArrayOutputStream).

I'd prefer a further refactoring in order to alleviate these issues. The problem of verifying console output in our test code won't be completely eliminated, but we'll at least isolate it to its own test. Here, we're extracting println from the string-building part of the code.

1. def printUpTo(limit: Int): Unit = {
2. applyUpTo(limit, println)
3. }
4. def applyUpTo(limit: Int, action: String => Unit): Unit = {
5. (0 to limit).foreach {
6. i => action("i = " + i)
7. }
8. }

Now we can test applyUpTo by passing our own custom action in, and tests for println-based behavior can be isolated, so that we isolate some of the necessary mocking and redirection. As a word of explanation, the action passed in needs to take a String as an argument and return the Unit value: (). Now we can also move printUpTo to a different class if we so desire (assuming this function is part of a larger application), eliminating our need to recompile the code in printUpTo each time the string format changes. I'd most likely put the applyUpTo call on its own in a main() method if there's really nothing more to the application.

There is another problem with the code above, however: both of the functions are returning Unit values (the equivalent of Java's void). This means that both of these functions rely on side effects to get their jobs done. If we were to pass a side effect-free function into applyUpTo, we wouldn't actually have any output from the applyUpTo to tell us that we did something. A side effect is any effect of a function beyond its return value, so we can say that if a Scala function has a return type of Unit, there are either side effects, or it is a trivial function that we can safely eliminate. We can make the helper function side effect-free like so:

1. def printUpTo(limit: Int): Unit = {
2. buildUpTo(limit).foreach(println)
3. }
4. def buildUpTo(limit: Int): Iterable[String] = {
5. (0 to limit).map("i = " + _)
6. }

In this case, we're relying on a bit of syntactic sugar for transferring the iterating value of foreach to println, and a placeholder (_) for map's iterating value. The buildUpTo function is now more functional and more easily testable (we can do a fairly simple comparison with the returned Iterable), but it's important to note that memory usage might be a concern, as a map of all the values could become pretty large. Another issue with this is speed: in the more functional version, we're iterating through the collection twice rather than just once. We don't often get things for free, and indeed there is a trade-off here between memory/speed and functionalness.

The example here has been on a very small scale, and perhaps a bit contrived, but we can extrapolate the ideas to a wide range of refactorings. We want to remove as much code with side effects and mutable state as we can from the rest of our application. This way we can divide up the functional parts of our application among processors in order to take advantage of the multiple cores that we all have in our machines these days. There are numerous other situations in which functional programming can lead to cleaner code, and there are also places where it might be inappropriate or impossible. It's important to remember the most common reason we might want to write functional code: concurrency. If I'm only going to be running this small application on my local machine, from the command line, and printing is going to be the limiting factor speed-wise (and it probably will in this case), I would most likely stop at the simplest foreach version above and be done with it. It will be efficient with memory and fast. But when the function buildUpTo becomes more complex, or we need to run numerous instances of the application, that's where concurrency, and functional programming, are best applied. Our goal should be to find, and use, the right tool for the job.

########